This paper determines the influencing factors of architectural acoustic environment comfort assessment from the perspective of green building acoustic environment comfort, and constructs a two-by-two comparison judgment matrix for each questionnaire survey result. Through the consistency test of the judgment matrix, the weights of the factors influencing the comfort of the building sound environment are obtained, and the construction of the assessment model of the comfort of the building sound environment is completed. Based on the architectural acoustic environment comfort assessment model, optimization variables are selected and multi-objective optimization is used to determine the objective function and constraints of architectural acoustic environment comfort. The comfort of the acoustic environment of the campus building is evaluated and analyzed through simulation analysis. The results show that the two measurement points, point 8 and point 15, located on the north side of the library and information building and the center courtyard, respectively, have relatively low continuous sound pressure levels Leq of 51.7 dB and 44.4 dB, respectively, which are relatively favorable for the creation of a comfortable acoustic environment. Improving the building arrangement through the multi-objective optimization method under the acoustic environment assessment model can provide good environmental protection for people’s daily lives and work.