Publicação
From brainwaves to emotions: Emotion recognition in media consumption using EEG and BCI
| Resumo: | The recognition of emotions from Electroencephalogram (EEG) signals captured by Brain-Computer Interface (BCI) is one of the emerging challenges in the field of Artificial Intelligence (AI) applied to digital media consumption. This study aimed to investigate how complete pipelines of pre-processing, feature extraction, and deep learning model training can be used to identify emotional states during the viewing of audiovisual content, focusing not only on obtaining quantitative results but also on better understanding the methodological factors that condition the models’ generalisation capacity. To this end, datasets from the SEED family were analysed and processed from raw signals, applying filtering, resampling, referencing, Independent Component Analysis (ICA), and baseline removal steps. Feature extraction included Differential Entropy (DE), Power Spectral Density (PSD), and Wavelet Transform (WT) in different frequency band schemes. Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and CNN-LSTM models were implemented and evaluated in multiple validation scenarios, including Cross-Subject (Single-Session), Cross-Session, and Cross-Subject (Multi-Session). The results showed that the inclusion of the delta band contributes to better generalisation, while high gamma did not show clear benefits. The combination of DE and PSD proved to be more robust than the isolated use of each feature. In terms of architectures, CNN proved to be the simplest and most computationally efficient solution, while both LSTM and CNN-LSTM showed greater potential for generalisation. It was also found that inadequate evaluation methodologies, such as classic 80/20 splits, can lead to artificially high metrics but without practical validity, reinforcing the importance of rigorous approaches such as Leave-One-Subject-Out (LOSO) validation. Beyond the technical aspect, this dissertation gave rise to a scientific contribution, materialised in the publication of an article proposing a technical-ethical guide for the collection and processing of EEG data, in accordance with the General Data Protection Regulation (GDPR) and the AI Act. The work carried out thus constitutes not only an advance in the critical analysis of emotion recognition pipelines, but also a solid basis for future research, including the continuity ensured within the European DataPACT project. |
|---|---|
| Autores principais: | VIEIRA, JOÃO FILIPE MOREIRA |
| Assunto: | EEG BCI Emotion Recognition Deep Learning Feature Extraction |
| Ano: | 2025 |
| País: | Portugal |
| Tipo de documento: | dissertação de mestrado |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Instituto Politécnico do Porto |
| Idioma: | inglês |
| Origem: | Repositório Científico do Instituto Politécnico do Porto |
| Resumo: | The recognition of emotions from Electroencephalogram (EEG) signals captured by Brain-Computer Interface (BCI) is one of the emerging challenges in the field of Artificial Intelligence (AI) applied to digital media consumption. This study aimed to investigate how complete pipelines of pre-processing, feature extraction, and deep learning model training can be used to identify emotional states during the viewing of audiovisual content, focusing not only on obtaining quantitative results but also on better understanding the methodological factors that condition the models’ generalisation capacity. To this end, datasets from the SEED family were analysed and processed from raw signals, applying filtering, resampling, referencing, Independent Component Analysis (ICA), and baseline removal steps. Feature extraction included Differential Entropy (DE), Power Spectral Density (PSD), and Wavelet Transform (WT) in different frequency band schemes. Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), and CNN-LSTM models were implemented and evaluated in multiple validation scenarios, including Cross-Subject (Single-Session), Cross-Session, and Cross-Subject (Multi-Session). The results showed that the inclusion of the delta band contributes to better generalisation, while high gamma did not show clear benefits. The combination of DE and PSD proved to be more robust than the isolated use of each feature. In terms of architectures, CNN proved to be the simplest and most computationally efficient solution, while both LSTM and CNN-LSTM showed greater potential for generalisation. It was also found that inadequate evaluation methodologies, such as classic 80/20 splits, can lead to artificially high metrics but without practical validity, reinforcing the importance of rigorous approaches such as Leave-One-Subject-Out (LOSO) validation. Beyond the technical aspect, this dissertation gave rise to a scientific contribution, materialised in the publication of an article proposing a technical-ethical guide for the collection and processing of EEG data, in accordance with the General Data Protection Regulation (GDPR) and the AI Act. The work carried out thus constitutes not only an advance in the critical analysis of emotion recognition pipelines, but also a solid basis for future research, including the continuity ensured within the European DataPACT project. |
|---|
Atividades financiadas
Não foram encontrados projetos financiados.