This paper first analyzes the network for 5G communication architecture and builds a framework for signal detection systems for wireless networks based on this architecture. Secondly, to better ensure the reliability of the signal detection system, the AE-ELM algorithm is built for wireless network signal detection by introducing an automatic coding machine and an overrun learning mechanism. Finally, an example is used to verify the reliability of the wireless network signal detection system in real-time. The results show that the signal detection rate of the real-time signal detection system with data volume over 300GB is 93% correct, and the maximum rate of missed rejection is about 6%. When SNR=25dB, the BER of the AE-ELM algorithm fluctuates around 10-5 with a variation of only about 0.1%. This shows that the signal detection system of a wireless network under 5G communication architecture has good reliability.