Sentiment analysis is one of the important applications in the field of natural language processing. With the development of science and technology, sentiment analysis is developing in the direction of multi-feature fusion, and multi-feature fusion plays an important value in application in English spoken emotional expression. In this paper, we design a method for extracting multi-features based on multi-networks and propose a sentiment analysis model, ECISA-MFF model, on the basis of a multi-feature extraction framework and feature fusion scheme to solve the problem of data non-alignment and modal noise, and then further classify the sentiments and optimize the model. The article concludes with a comparison of the relevant performance of the models as well as a case study, and it is found that the F-macro value of the model proposed in this paper rises by 22.2% when compared with the BiLSTM+att+C model, which is significantly effective. After applying this paper’s method for emotion recognition, it was found that the average recognition rate of six emotions (happiness, anger, fear, surprise, sadness, and disgust) increased to 65.52%.