This paper uses nonlinear differential equations to establish a relationship model between enrollment, education, and employment in vocational education. In this paper, the local stability of the vocational education system is analyzed utilizing nonlinear differential theory and numerical simulation. At the same time, we carried out a numerical simulation of the system’s dynamic behavior. Through numerical simulation research, it is found that the model has peculiar nonlinear behavior and dynamic characteristics when its parameters take some specific values. Finally, we get the critical condition of the stability of the vocational education system according to the model established in the article.