Photovoltaic power system is taking a significant percentage of power system and the demands for accurate forecasting of the power outputs is surging. In prior works, the forecasting problem was formulated as a regression problem, however, which most cannot guarantee that the forecasted outputs is nonnegative. To solve this problem, we proposed a novel probabilistic model by using nonlinear regression and Bayesian learning method. In the paper, we present the detailed theoretical derivations and interpretations. The simulation results show the validity and feasibility of the proposed algorithm by comparing with the traditional SVM algorithm.