Using data mining technology to obtain important information in massive data has gradually become an important basis for mathematics teaching strategies in colleges and universities. Combined with data mining technology, this paper proposes the analysis of students’ behavior in mathematics classrooms and personalized mathematics learning strategies and constructs the corresponding method model. The behavior of students in the mathematics classroom is analyzed by using the character behavior recognition technology, and the 7Hu moments of the motion history graph and the motion energy graph are used as the features of behavior recognition. Construct a learning model using the user-item scoring matrix, improve and standardize the null-filling method of scoring, and provide a basis for students’ personalized learning in mathematics. After applying the teaching strategy and the corresponding model to 70 mathematics majors in a university, the student’s performance in mathematics was significantly better than that of the previous semester after the weekly practice 4, and their average score in the midterm examination reached 63.8. The mean values of all dimensions of motivation increased compared with the pre-practice period, and all dimensions were significantly different except the dimension of the learning environment factors (p<0.05). The students demonstrated significant improvement in all dimensions of math performance, with significant differences.