This paper proposes the general architecture of a multi-scale classification system based on the intelligent imputation method of multi-scale classification. SVM is used to transform the storage structure model of the object, obtain the trained support vector set with corresponding weights, measure the similarity between two data points based on the Hausdorff distance, realize the construction of a similarity matrix, adopt the idea of mean value, blur the data information, and improve the mechanism of on-scale imputation. The bicubic difference method is used as the theoretical basis of scale-down extrapolation, and the scale-down extrapolation algorithm DAMSC is established. Based on the quaternionic linkage number, the dataset data are preprocessed, and the data eigenvalue weights are calculated at the same time. The multi-scale classification validity index evaluates the model’s accuracy in classification and its performance in both upward and downward scale extrapolation. When the categorized data is 7, the MSCVI metric of the model attains a peak of 0.92254 in the segmentation dataset run results, which is more advantageous than other metrics. In the regression test of the model’s imputation results, the correlation coefficient of bus passengers is 0.99, the adjusted R-square is 0.97043, and the coefficient of the regression equation is 1.042, and the model in this paper is valid and reliable for the imputation of passenger drop-off points.