Applied Mathematics and Nonlinear Sciences
Journal license

Journal

Applied Mathematics and Nonlinear Sciences


Volume
& Issue

Volume 9, Issue 1


Published
on

August 5, 2024


Pages


DOI

Article

EEGVision: Reconstructing vision from human brain signals

Check for updates


Authors

Huangtao Guo Affiliation:
College of Science, China University of Petroleum (East China), Qingdao, Shandong, 266580, China.


Abstract

The intricate mechanisms elucidating the interplay between human visual perceptions and cognitive processes remain elusive. Exploring and reconstructing visual stimuli from cerebral signals could help us better understand the processes by which the human brain generates visual imagery. However, the inherent complexity and significant noise in brain signals limit current efforts to reconstruct visual stimuli, resulting in low-granularity images that miss details. To address these challenges, this paper proposes EEGVision, a comprehensive framework for generating high-quality images directly from brain signals. Leveraging the recent strides in multi-modal models within the realm of deep learning, it is now feasible to bridge the gap between EEG data and visual representation. This process starts with a time-frequency fusion encoder in EEGVision, which quickly pulls out cross-domain and robust features from EEG signals. We then design two parallel pipelines to align EEG embeddings with image features at both perceptual and semantic levels. The process uses a stable diffusion-trained image-to-image pipeline that combines coarse and fine-grained data to get high-quality images back from EEG data. Both quantitative and qualitative assessments affirm that EEGVision surpasses contemporary benchmarks. This network architecture holds promise for further applications in the domain of neuroscience, aiming to unravel the genesis of human visual perception mechanisms. All code is accessible via https://github.com/AvancierGuo/EEGVision.


Keywords

Visual Reconstruction, Human visual perceptions, EEGVision, Multi-modal models, Neuroscience, 68-02


Citation

Guo, H. (2024). Eegvision: Reconstructing vision from human brain signals. Applied Mathematics and Nonlinear Sciences, 9(1). https://doi.org/10.2478/amns-2024-1856

Published by: Engineering Journals

Engineering Journals Logo