Under the background of the development of network technology, this paper aims to promote rural English teaching and constructs an English teaching model that combines English recognition technology and rural teaching. The main process of speech recognition is examined by analyzing different speech recognition technologies. Using a deep learning network, an English speech recognition model has been established. Combined with the English acoustic features in the network data, fluency of English speech is evaluated. Data embedding is performed on the English sequences in the network, combined with the sequence probability in the English data, so as to determine whether the English speech is correct or not. The Eval value for the English recognition model based on deep learning is 5.49%, while the test value is 5.89%, as per the results. As the English dataset increases, so does the English recognition technique proposed in this paper, and the accuracy remains above 0.6, and when the dataset is 500, the speech recognition accuracy is 0.8. The teaching model that combines speech recognition techniques with English teaching improves students’ English to a certain extent.