In this paper, in the process of solving the mathematical model of sports facility optimization, the integer linear programming issue of choosing a sports facility is solved using a branch delimitation approach paired with a recursive matrix-based column-generating method. As part of the process of creating a scientific evaluation system for physical education, a hierarchical gray correlation-based evaluation model for the quality of physical education in colleges and universities is built by combining the hierarchical analysis weight-solving method and the solution method of the gray correlation degree. After the analysis, 81.1% of the teachers and students were satisfied after physical education facilities were optimized with the physical education teaching facilities. The gray correlation coefficient of physical education hardware facilities and software is 0.7, and the gray correlation coefficient of physical education effect is 0.18.