The advancement of computer vision technology, coupled with developments in deep learning theory, has facilitated the widespread application of deep learning across various domains. This paper leverages the YOLOv7 deep learning algorithm to incorporate the MobileVITv3 lightweight Transformer architecture and a positional attention mechanism, culminating in the creation of the MobileVIT-YOLO-Tiny algorithm for uncrewed aerial vehicle (UAV) image recognition and detection. Furthermore, utilizing the principles of the Kalman filter algorithm, this study designs an optimized Kalman filter tracking algorithm specifically for UAVs. Subsequent experimental evaluations are conducted to validate the efficacy of the proposed algorithms in UAV image detection and tracking. The results indicate that the overall average precision (AP) of the UAV image detection algorithm is 0.8931, with leading performance in simple and complex background scenarios among participating algorithms. Moreover, the UAV tracking optimization algorithm achieves success rates of 0.827 and 0.629, respectively, fulfilling the real-time operational demands of UAV image tracking. The findings underscore the significant potential for practical applications of the UAV image detection and tracking algorithm developed in this study, as evidenced by the experimental outcomes.