Infrared target detection is widely used in military, life and industrial control industries, so it has been a hot spot of research. This paper optimizes the design and deployment of real-time target detection algorithms for infrared images based on deep learning. Considering the excellent detection accuracy and inference rate of YOLO series target detection networks, this paper introduces the attention mechanism and designs an improved YOLOv5 network based on Transformer network. , the realization of infrared optical image fusion and target detection technology is carried on the embedded platform. Analyzed by detection experiments, the introduction of the CBAM attention mechanism in the backbone network is 0.53% and 0.44% higher in mAP compared to SENet and CA, respectively. The precision and recall of the improved algorithm in this paper reach 85.7% and 81.1%, respectively, which is a significant advantage over other lightweight models of the same type. The optimized model also has an advantage in the comparison of models with large number of parameters, and the mAP of the optimized model in this paper is improved by 2.7% compared to YOLOv5m, and is 2.2 and 2.1 percentage points higher than YOLOv6s and YOLOv8s. Compared with YOLOv7, it is only 0.01% lower, but the number of parameters is 27.06M less than YOLOv7. Meanwhile, the deployment scheme of the embedded platform in this paper has an accuracy of 54.87 and a CPU occupancy of 50.21%, which is in the middle range when compared to other schemes. The running time, memory occupation and number of parameters are all optimal, which verifies the effectiveness of this paper’s infrared image detection model based on deep learning, and it has important application value in the fields of military and national defense, disaster detection, and smart city.