Incorporating American voice singing in singing and vocal music teaching has always been a problem with no standards and excessive teacher subjectivity. In this paper, based on a convolutional neural network, the American voice features are extracted using Mel spectrum, and the CBAM attention module is introduced to refine the American voice features and improve the influence of background noise on American voice extraction. The emotional features extracted under different scale frame lengths are feature fused to build a neural network model of multi-scale feature fusion to quantitatively evaluate the students’ American voice singing. After obtaining the consent of the leadership of a university, it was put into use in its music department to score the students by analyzing the status of their pronunciation of American vowels, consonants, and legato, and output the problems that the students had in a specific bar. The results showed that the more frames in the input vector, the more correct the model was, 89.32 ± 0.21 for 20 frames, increasing to 90.05 ± 1.32 for 40 frames. Student 1 scored 0.654 for American vowel pitch and 0.643 for consonants. Student 10 had the highest vowel pitch with a score of 0.718. Vowel scores were generally around 0.6. Students had the highest wavelength of 0.61, which does not correspond well with the original score. This study shows the direction for the strategy of integrating American singing into the vocal program and promoting the healthy development of vocal teaching.