Based on big data technology, this paper first proposes to study and analyze the structural model of teachers’ teaching ability based on multimodal sentiment features of support vector machines. Then the Mel inverse spectral coefficients, wavelet packet coefficients, Fourier coefficient features, dynamic features, and global features are extracted respectively, and the features are dimensionalized using linear support vector machine and Lagrangian function, and the dimensionalized feature parameters are sent to the classifier for emotion recognition. Finally, the structural model evaluation index system of teachers’ teaching ability is constructed, and the current teaching ability development of college teachers in a province is studied and analyzed from two perspectives of teachers’ self-assessment and students’ evaluation based on multimodal emotional features of support vector machine. The results showed that in terms of teaching ability dimensions, teachers had the highest expectation of their teaching ability (4.38) in each dimension, followed in order by teachers’ self-evaluation of teaching ability (4.15), students’ expectation (4.05), and students’ evaluation of teachers’ teaching ability (3.56) was the lowest. This study reveals that teachers’ perceptions of the teaching competencies they should have are biased, and their awareness of self-development is insufficient from the emotional characteristics.