Applied Mathematics and Nonlinear Sciences
Journal license

Journal

Applied Mathematics and Nonlinear Sciences


Volume
& Issue

Volume 9, Issue 1


Published
on

August 5, 2024


Pages


DOI

Article

Multimodal biometric fusion sentiment analysis of teachers and students based on classroom surveillance video streaming

Check for updates


Authors

Tianxing Zhang Affiliation:
Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, Bangi, Selangor, 43600, Malaysia.
, Hadi Affendy Bin Dahlan Affiliation:
Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, Bangi, Selangor, 43600, Malaysia.
, Zengsheng Xie Affiliation:
FUZHOU INSTITUTE OF EDUCATION RESEARCH, Fuzhou, Fujian, 35001, China.
, Jinfeng Wu Affiliation:
Fuzhou No.8 School, Fuzhou, Fujian, 350000, China.
, Yingping Chen Affiliation:
Wuyi University, Fuzhou, Fujian, 529020, China.
, Qianying Pan Affiliation:
Fujian ChuanZheng Communications College, Fuzhou, Fujian, 35007, China.
and Ying Huang Affiliation:
Fuzhou No.8 School, Fuzhou, Fujian, 350000, China.


Abstract

In the education system, teachers and students as the main body of the classroom; their emotional state in the classroom school is an important indicator of the effectiveness of the classroom. This study first explores biometric recognition, based on the needs of the classroom curriculum and the classroom monitoring as a sensor, to propose a multimodal biometric fusion detection method based on the fusion of face and gait recognition. The PCA algorithm is used to optimize the face recognition as well as the occlusion situation in the classroom to improve gait recognition, and then the face and gait are fused based on the decision layer to achieve the detection and recognition of the identity situation of teachers and students. On this basis, an expression recognition model is established using the attention mechanism, and an emotion analysis system is designed for the classroom curriculum. According to the empirical evidence of multimodal biometric fusion sentiment analysis, the mAP accuracy of this paper’s fusion method is 100% in Euclidean distance, and the accuracy is higher than 99% in cosine distance, which is obviously better than other methods, and the accuracy of this paper’s fusion recognition is above 95% under any condition limitations. At the same time, the correct rate of recognition of emotions such as listening, appreciation, resistance, doubt, and inattention are all higher than 85%, and the five indexes of average absolute error, Pearson correlation coefficient, Accuarcy5, Accuarcy2, and F12 score of this paper’s sentiment analysis have achieved the best results comparing with other sentiment analysis models, which proves the generalization and validity of this paper’s sentiment analysis.


Keywords

Biometric identification, PCA, Fusion algorithm, Expression recognition, Sentiment analysis, 97B20


Citation

Zhang, T., Dahlan, H. A. B., Xie, Z., Wu, J., Chen, Y., Pan, Q., & Huang, Y. (2024). Multimodal biometric fusion sentiment analysis of teachers and students based on classroom surveillance video streaming. Applied Mathematics and Nonlinear Sciences, 9(1). https://doi.org/10.2478/amns-2024-2156

Published by: Engineering Journals

Engineering Journals Logo