The construction of an intelligent teaching environment is conducive to the improvement of the quality of sports teaching in colleges and universities, and teaching behavior recognition is one of the important means to create and realize an intelligent environment of education. This paper develops an intelligent sports teaching method for colleges and universities based on a preliminary analysis of sports teaching behavior recognition. The method is applied to teaching physical education in a university, and experimental tests are carried out on three dimensions: students’ physical fitness, learning interest, and changes in physical test scores. After the training and validation results on the experimental dataset, it was shown that the improved M-LSTM model proposed in this paper has a good recognition rate. In the change of physical learning interest, students’ physical learning motivation, physical skill learning, after-school physical activity, and physical attention under intelligent teaching are significantly higher than that of traditional teaching methods. It can be concluded that the intelligent physical education teaching method proposed in this paper has good practical effects on the teaching environment in colleges and universities.