In order to effectively improve college students’ English reading ability, this paper introduces multimodal teaching to analyze the way of teaching English reading in college. This paper firstly dismantles the multimodal teaching model, which is divided into multimodal teaching and the incorporation of the convolutional neural network, and analyzes the current situation of college English reading teaching. Secondly, we select the college English reading teaching modality, construct a multimodal fusion model for college English reading teaching and optimize this model to analyze the convergence and time complexity. Through the analysis of the fusion performance, we found that the accuracy and the pick-up degree of the pre-fusion increased by 22.25% and decreased by 18.22% compared with the unimodal fusion, and the number of participants increased by 61.51 Mb, which was not much different from the middle and late comparison, so the pre-fusion was selected to have the highest practicality. Finally, through the survey statistics on the application of multimodal teaching of college English reading, more than 80% of students and teachers think that multimodal teaching of college English reading can improve their English reading ability and teaching ability. So this teaching method improves their reading ability and enriches the diversity of teaching modes at the same time, which is worth promoting.