This paper analyzes the current situation of physical education teaching in colleges and universities from the perspective of deep learning and innovates the teaching methodology. A multidimensional data array is used to quantify students’ interests and hobbies to create a recommendation module for intelligent sports learning materials. Using the SIFT feature extraction algorithm, identify the extreme value point of the detection target and develop the sports target extraction module. Through distance transformation and morphological features, determine the skeleton image features to construct a college sports teaching model using deep learning. The results show that the model in this paper enables the coverage rate of sports courses to reach over 97.20%, with a fluctuation of no more than 1.5%. Deep learning has contributed to some innovation in college sports teaching.