| 29 | 0 | 252 |
| 下载次数 | 被引频次 | 阅读次数 |
脑电信号(EEG)图像重建技术在辅助残疾人视觉功能及推动脑机接口(BCI)发展方面具有重要意义。然而，EEG信号噪声大、空间分辨率低的特点使得高精度图像重建面临巨大挑战。因此，本文提出一种基于生成对抗网络(GAN)的双阶段脑电信号图像生成框架，通过双阶段处理流程实现从小规模EEG数据集到高质量图像的端到端重建。首先，采用双向长短期记忆网络(Bi-LSTM)结合多头注意力机制提取EEG信号的特征，并通过三元组损失优化特征空间，增强同类样本的聚集性和异类样本的区分性;其次，利用特征增强的条件生成对抗网络(cGAN)，引入可微分数据增强和模式正则化技术，显著提升生成图像的分辨率(128×128)和多样性。实验结果表明，所提框架在Inception Score(IS)评估指标上达到6.75，优于现有方法，为小规模EEG数据下的图像重建提供新思路。
Abstract：Electroencephalogram( EEG) signal-to-image reconstruction technology holds significant importance in assisting the visual function of individuals with disabilities and advancing the development of brain-computer interfaces( BCIs). However,the inherent challenges of EEG signals—characterized by high noise levels and low spatial resolution—make high-precision image reconstruction extremely difficult. To address this,this study proposes a dual-stage generative adversarial network( GAN) framework for EEG-to-image generation. This framework achieves end-to-end reconstruction of high-quality images from small-scale EEG datasets through a dualstage processing pipeline. Firstly,a Bidirectional Long Short-Term Memory network( Bi-LSTM) combined with a multi-head attention mechanism extract features from EEG signals. The feature space is optimized using triplet loss to enhance intra-class compactness and inter-class distinctiveness. Secondly,a feature-enhanced conditional GAN( c GAN) is employed,incorporating differentiable data augmentation and pattern regularization techniques. This significantly improves the resolution( 128 × 128) and diversity of the generated images. Experimental results demonstrate that the proposed framework achieves an Inception Score( IS) of 6. 75,outperforming existing methods. This study offers a novel approach to image reconstruction under small-scale EEG data constraints.
[1]Bamdad M,Zarshenas H,Auais M A. Application of BCI systems in neurorehabilitation:a scoping review[J]. Disability and Rehabilitation:Assistive Technology,2015,10(5):355-364.
[2]Miyawaki Y,Uchida H,Yamashita O,et al. Visual image reconstruction from human brain activity using a combination of multiscale local image decoders[J]. Neuron,2008,60(5):915-929.
[3]Spampinato C,Palazzo S,Kavasidis I,et al. Deep learning human mind for automated visual classification[C]//In Proceedings of the IEEE conference on computer vision and pattern recognition,2017:6809-6817.
[4]Palazzo S,Spampinato C,Kavasidis I,et al. Decoding brain representations by multimodal learning of neural activity and visual features[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2020,43(11):3833-3849.
[5]Pratiwi M,Wibawa A D,Purnomo M H. EEG-based happy and sad emotions classification using LSTM and bidirectional LSTM[C]//In 2021 3rd international conference on electronics representation and algorithm(ICERA),2021:89-94.
[6]Algarni M,Saeed F,Al-Hadhrami T,et al. Deep learning-based approach for emotion recognition using electroencephalography(EEG)signals using bi-directional long short-term memory(Bi-LSTM)[J]. Sensors,2022,22(8):2976.
[7]Ma S,Cui J,Chen C L,et al. An improved Bi-LSTM EEG emotion recognition algorithm[J]. J. Netw. Intell,2022,7(3):623-639.
[8]Zhao Y,Liang Z,Du J,et al. Multi-head attention-based long short-term memory for depression detection from speech[J]. Frontiers in Neurorobotics,2021,15:684037.
[9]Qu W,Wang Z,Hong H,et al. A residual based attention model for EEG based sleep staging[J]. IEEE journal of biomedical and health informatics,2020,24(10):2833-2843.
[10]Goodfellow I,Pouget-Abadie J,Mirza M,et al. Generative adversarial networks[J]. Communications of the ACM,2020,63(11):139-144.
[11]Radford A,Metz L,Chintala S,et al. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks[EB/OL]. ar Xiv preprint ar Xiv:1511. 06434.(2015-11)/[2024-08-13]. https://arxiv. org/abs/1511. 06434.
[12]Gurumurthy S,Kiran Sarvadevabhatla R,Venkatesh Babu R. Deligan:Generative adversarial networks for diverse and limited data[C]//Proceedings of the IEEE conference on computer vision and pattern recognition,2017:166-174.
[13]Tirupattur P,Rawat Y S,Spampinato C,et al. Thoughtviz:Visualizing human thoughts using generative adversarial network[C]//In Proceedings of the 26th ACM international conference on Multimedia,2018:950-958.
[14]Zhao S,Liu Z,Lin J,et al. Differentiable augmentation for data-efficient gan training[J]. Advances in neural information processing systems,2020,33:7559-7570.
[15]Mao Q,Lee H Y,Tseng H Y,et al. Mode seeking generative adversarial networks for diverse image synthesis[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition,2019:1429-1437.
[16]Kavasidis I,Palazzo S,Spampinato C,et al. Brain2image:Converting brain signals into images[C]//Proceedings of the 25th ACM international conference on Multimedia,2017:1809-1817.
[17]Vivekananthan S. Comparative Analysis of Generative Models:Enhancing Image Synthesis with VAEs,GANs,and Stable Diffusion[EB/OL]. ar Xiv preprint ar Xiv:2408. 08751.(2024-08)/[2024-08-13]. https://arxiv. org/abs/2408. 08751.
[18]Bai Y,Wang X,Cao Y P,et al. Dream Diffusion:Generating High-Quality Images from Brain EEG Signals[EB/OL]. ar Xiv preprint ar Xiv:2306. 16934.(2023-06-29)/[2024-08-13]. https://arxiv. org/abs/2306. 16934.
[19]Zhang X,Chen X,Dong M,et al. Multi-task Generative Adversarial Learning on Geometrical Shape Reconstruction from EEG Brain Signals[EB/OL]. ar Xiv preprint ar Xiv:1907. 13351.(2019-07-31)/[2024-08-13]. https://arxiv. org/abs/1907.13351.
[20]Fares A,Zhong S H,Jiang J. Brain-media:A dual conditioned and lateralization supported GAN(DCLS-GAN)towards visualization of image-evoked brain activities[C]//Proceedings of the 28th ACM International Conference on Multimedia,2020:1764-1772.
[21]Long F,Zhou K,Ou W. Sentiment analysis of text based on bidirectional LSTM with multi-head attention[J]. Ieee Access,2019,7:141960-141969.
[22]Ye Z,Yang L,Zhang Y,et al. See What You See:Self-supervised Cross-modal Retrieval of Visual Stimuli from Brain Activity[EB/OL]. ar Xiv preprint ar Xiv:2208. 03666.(2022-08-07)/[2024-08-13]. https://arxiv. org/abs/2208. 03666.
[23]Zheng X,Chen W,Li M,et al. Decoding human brain activity with deep learning[J]. Biomedical Signal Processing and Control,2020,56:101730.
[24]Vaswani A,Shazeer N,Parmar N,et al. Attention is all you need[J]. Advances in neural information processing systems,2017,30.
[25]Yang X,Guo Y,Li Z,et al. MRDN:A lightweight multi-stage residual distillation network for image super-resolution[J]. Expert Systems with Applications,2022,204:117594.
[26]He K,Zhang X,Ren S,et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016:770-778.
[27]Tao Y,Takagi K,Nakata K. Clustering-friendly Representation Learning via Instance Discrimination and Feature Decorrelation[EB/OL]. ar Xiv preprint ar Xiv:2106. 00131.(2021-06-01)/[2024-08-13]. https://arxiv. org/abs/2106. 00131.
[28]Schroff F,Kalenichenko D,Philbin J. Facenet:A unified embedding for face recognition and clustering[C]//Proceedings of the IEEE conference on computer vision and pattern recognition,2015:815-823.
[29]Lim J H,Ye J C. Geometric gan[J]. ar Xiv preprint ar Xiv:1705. 02894,2017.
[30]Kumar P,Saini R,Roy P P,et al. Envisioned speech recognition using EEG sensors[J]. Personal and Ubiquitous Computing,2018,22:185-199.
[31]Deng J,Dong W,Socher R,et al. Imagenet:A large-scale hierarchical image database[C]//2009 IEEE conference on computer vision and pattern recognition,2009:248-255.
[32]Jin X,Han J. K-means clustering[J]. Encyclopedia of machine learning,2011:563-564.
[33]Van der Maaten L,Hinton G. Visualizing data using t-SNE[J]. Journal of machine learning research,2008,9(11).
[34]Salimans T,Goodfellow I,Zaremba W,et al. Improved techniques for training gans[C]//Advances in neural information processing systems,2016:2163-2899.
[35]Odena A,Olah C,Shlens J. Conditional image synthesis with auxiliary classifier gans[C]//International conference on machine learning. PMLR,2017:2642-2651.
[36]Mishra R,Bhavsar A. Generating Visual Stimuli from EEG Recordings using Transformer-encoder based EEG encoder and GAN[EB/OL]. ar Xiv preprint ar Xiv:2402. 10115.(2024-11-20). https://arxiv. org/abs/2402. 10115.
[37]Mishra R,Sharma K,Jha R R,Bhavsar A. Neuro GAN:image reconstruction from EEG signals via an attention-based GAN[J]//Neural Computing and Applications,2023,35(12):9181-9192.
[38]Rakhmatulin I,Dao M S,Nassibi A,Mandic D. Exploring convolutional neural network architectures for EEG feature extraction[J]. Sensors,2024,24(3):877.
[39]Sutskever I,Vinyals O,Le Q V. Sequence to sequence learning with neural networks[C]//Advances in neural information processing systems. 2014:3104-3112.
基本信息:
DOI：10.20064/j.cnki.2095-347X.2026.01.003
中图分类号:R318;TN911.7;TP391.41
引用信息:
[1]李帅,梁美彦.基于特征增强GAN的脑电信号图像重建[J].网络新媒体技术,2026,15(01):21-32.DOI:10.20064/j.cnki.2095-347X.2026.01.003.
基金信息:
国家自然科学基金项目(编号：11804209); 山西省自然科学基金项目(编号：202303021211014); 山西省国家留学基金委员会(编号：2023-010); 中国博士后科学基金项目(编号：2023M742577); 山西省研究生教育创新计划(编号：2024SJ016)
2026-01-15
2026-01-15