Applied Mathematics and Nonlinear Sciences
Journal license

Journal

Applied Mathematics and Nonlinear Sciences


Volume
& Issue

Volume 9, Issue 1


Published
on

November 14, 2024


Pages


DOI

Article

Advances in computer AI-assisted multimodal data fusion techniques

Check for updates


Authors

Pan Fan Affiliation:
College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, 030600, China.
and Qiang Wu Affiliation:
School of History, Culture and Tourism, Gannan Normal University, Ganzhou, Jiangxi, 341000, China.


Abstract

Through the integration of multimodal data fusion technology and computer AI technology, people’s needs for intelligent life can be better met. This paper introduces the alignment and perception algorithm for multimodal data fusion, which is based on combining the multimodal data fusion model. Taking the air pollutant concentration prediction as an example, the time series of air pollutant concentration is obtained through the LSTM model for concentration prediction, and the attention mechanism is introduced to establish the numerical prediction model of air pollution. Different stations are also selected to acquire weather image data, and the TS-Conv-LSTM multimodal spatio-temporal fusion model of air quality images is constructed by utilizing the Conv-LSTM cell as an encoder, and then the TransConv-LSTM cell, which integrates the anti-convolution and the long-short-term memory network cell, as a decoder. The Gaussian regression model was then used to combine numerical prediction and image prediction models, thus achieving the multimodal synergistic prediction of air pollutant concentrations. The RMSE of the ATT-LSTM model on the dataset was reduced to 8.03 compared to the comparison model, and the predictive fit to the image dataset was above 0.75 for all R² values. The lowest MAE value obtained by the multimodal collaborative prediction model is only 3.815, and the highest R² value is up to 0.985. Introducing deep learning techniques into multimodal data fusion helps to explore the value of massive data more deeply and obtain more comprehensive and reliable information about it.


Keywords

Multimodal data fusion, LSTM model, Conv-LSTM, Gaussian regression model, Multimodal cooperative prediction, 03B70


Citation

Fan, P. & Wu, Q. (2024). Advances in computer ai-assisted multimodal data fusion techniques. Applied Mathematics and Nonlinear Sciences, 9(1). https://doi.org/10.2478/amns-2024-3232
2 Total citations
0.41 FWCI
2 Recent citations
(2 years)
28 References
Open Access Yes
View full metrics

Published by: Engineering Journals

Engineering Journals Logo