In order to solve the problem of the traditional gray prediction model (GM) during determination of the accuracy of buildings’ energy savings and its poor fitting of data, the idea of a fractional model based on the traditional first-order one-variable GM(1,1) model is applied. We use the GM–backpropagation (GM-BP) neural network to solve the optimal fractional order and establish a fractional GM(1,1) model based on the GM-BP neural network. Example calculation shows that the fractional GM(1,1) model can improve the prediction accuracy of buildings’ energy savings, and selecting the optimal order can further improve the prediction accuracy and decrease the error level when using the GM-BP neural network. This work shows that the fractional GM(1,1) model based on the GM-BP neural network has an important guiding role in the energy savings of buildings.