The gymnastic movement recognition system is designed to determine the standard degree of trainers’ movements by accurately grasping their body contours and body positions through motion capture to achieve the training effect. This paper proposes an improved LSI-SVM algorithm based on a support vector machine applied to a gymnastic movement recognition system. Firstly, based on the idea of NMFDA, for each class of samples, its intra-class k-nearest neighbor and inter-class k-nearest neighbor local structure information are mined. Secondly, the obtained structural information is introduced into TSVM to obtain a new classification model. Finally, system operation test experiments are conducted to verify the generalization and accuracy of the proposed LSI-TSVM algorithm for the gymnastic movement recognition system. The experimental results show that the average recognition rate of the LST-TSVM algorithm proposed in this paper is 94.1%, which is 5.9% higher than S-TSVM and 14% higher than SVM, and 8.9% higher compared to SRSVM. The gymnastic action recognition system based on the LST-TSVM algorithm can effectively solve the problem of matching the corresponding frames of gymnastic action sequences on the time axis and effectively improve the action recognition rate, which can better assist trainers in learning gymnastic items.