This paper combines virtual reality technology with end-to-end speech recognition technology to design a digital tourism system to satisfy non-native English speakers (tourists) with the advantage of interactive projection technology. VRML technology is utilized to accelerate network interaction and piggyback on the construction of the digital tourism system. Particle swarm-based hybrid collision algorithm and end-to-end English speech recognition algorithm are proposed respectively to construct a real and complete tourism environment and convert English speech to meet the tourism needs of non-native English speakers. The performance advantages of the improved AABB-MPSO algorithm are detected by combining the exploration of the influence of the size of the search space on the search success rate and search time. Analyze the relationship between the number of modeling units and the recognition error rate of the end-to-end English recognition model with a joint CTC and attention mechanism, using spoken speech of free conversation in real scenes. The seven digital tourism experience dimensions of scenic integrity, scenic service quality, scenic authenticity, scenic infrastructure, degree of commercialization, English recognition and conversion, and cultural connotation embodiment are delineated to explore the key factors of tourism physical examination. Non-native English speakers’ evaluation of the digital tourism experience is generally high, and the mean value of each dimension is greater than “3”, which means that the digital tourism system designed in this paper can meet the tourism needs of non-native English speakers.