Publicação
Using the body to help predict the brain: integration of body and brain physiological signals to decode visual attention and neural response to objects
| Resumo: | The perception of a sensory stimulus depends on our neural and physiological state at the time of the stimulus. For example, warning stimuli produce a state of expectation that facilitates perceptual decision-making. This state modulates cortical activity, activates the parasympathetic system, which causes the heart to slow down, and activates the sympathetic system, which is reflected in pupil dilation and changes in skin conductance. The goals of this study were two-fold: to investigate how this state of expectation is reflected in changes in body and neural physiological signals; and to check whether this pre-stimulus state modulates cortical processing of visual stimuli and, consequently, visual perception. We used a warned visual discrimination task where participants were required to determine if the presented visual stimulus consisted of a car or a house. Each trial started with an auditory warning cue that alerted participants of the upcoming stimulus. While participants were engaged in the task, we simultaneously acquired electroencephalography (EEG) and body physiological signals (electrocardiography, respiration, pupillography and eye movements). The study of the association between pre-stimulus EEG activity, body physiological signals and visual stimulus detection suggested that both neural activity and pre-stimulus physiological activity are associated with visual performance. To further study the effect of the neural and body state before stimulus onset on visual processing, we used machine learning and tested different types of classifiers – support vector machines and convolutional neural networks. Two types of analyses were conducted using the classifiers. Firstly, the classifiers were used to predict the participant's perception of the stimulus based on the analysis of the body’s preparation for the stimulus. Secondly, we analyzed the classifier’s ability to discriminate the category of the trial using post-stimulus EEG evoked activity, and determine whether the categorization ability was modulated by pre-stimulus physiological activity. The results suggest that the warning stimulus induces changes in the EEG signal and in body physiology. Between the warning and the presentation of the visual stimulus participants showed a negative shift in their EEG signal, cardiac deceleration, a decrease in saccade and blink rate possibly associated with motor inhibition, pupil dilation and an increase in the duration of the breathing cycle. Only pre-stimulus EEG, respiratory activity and pupil behavior contained information that enabled the classifier to predict stimulus detection. However, when combined with EEG activity, none of the physiological signals provided supplementary information to the classifier. Finally, we evaluated whether this pre-stimulus influence modulated stimulus coding and could help the classifier to discriminate between the two classes of images. Prior studies suggest that specific fluctuations in neural activity may favor recognition of different categories. While these studies focused on neural activity, we extended our analyses to include physiological activity as well. The results of this analysis suggest that pre-stimulus activity does not modulate the way the stimulus is decoded. Overall, the results suggest that pre-stimulus activity influences the recognition of upcoming stimuli but that the changes in body physiology observed are already reflected in the pre-stimulus EEG activity and do not modulate the neural representation of the stimulus. |
|---|---|
| Autores principais: | Marques, Ana Rafaela Melanda |
| Assunto: | pre-stimulus neural and body activity attention body-brain interactions Convolutional Neural Network (CNN); Support Vector Machine (SVM) atividade neural e corporal pré-estímulo atenção interações corpo-cérebro Convolutional Neural Network (CNN) Support Vector Machine (SVM) |
| Ano: | 2023 |
| País: | Portugal |
| Tipo de documento: | dissertação de mestrado |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade de Coimbra |
| Idioma: | inglês |
| Origem: | Estudo Geral - Universidade de Coimbra |
| Resumo: | The perception of a sensory stimulus depends on our neural and physiological state at the time of the stimulus. For example, warning stimuli produce a state of expectation that facilitates perceptual decision-making. This state modulates cortical activity, activates the parasympathetic system, which causes the heart to slow down, and activates the sympathetic system, which is reflected in pupil dilation and changes in skin conductance. The goals of this study were two-fold: to investigate how this state of expectation is reflected in changes in body and neural physiological signals; and to check whether this pre-stimulus state modulates cortical processing of visual stimuli and, consequently, visual perception. We used a warned visual discrimination task where participants were required to determine if the presented visual stimulus consisted of a car or a house. Each trial started with an auditory warning cue that alerted participants of the upcoming stimulus. While participants were engaged in the task, we simultaneously acquired electroencephalography (EEG) and body physiological signals (electrocardiography, respiration, pupillography and eye movements). The study of the association between pre-stimulus EEG activity, body physiological signals and visual stimulus detection suggested that both neural activity and pre-stimulus physiological activity are associated with visual performance. To further study the effect of the neural and body state before stimulus onset on visual processing, we used machine learning and tested different types of classifiers – support vector machines and convolutional neural networks. Two types of analyses were conducted using the classifiers. Firstly, the classifiers were used to predict the participant's perception of the stimulus based on the analysis of the body’s preparation for the stimulus. Secondly, we analyzed the classifier’s ability to discriminate the category of the trial using post-stimulus EEG evoked activity, and determine whether the categorization ability was modulated by pre-stimulus physiological activity. The results suggest that the warning stimulus induces changes in the EEG signal and in body physiology. Between the warning and the presentation of the visual stimulus participants showed a negative shift in their EEG signal, cardiac deceleration, a decrease in saccade and blink rate possibly associated with motor inhibition, pupil dilation and an increase in the duration of the breathing cycle. Only pre-stimulus EEG, respiratory activity and pupil behavior contained information that enabled the classifier to predict stimulus detection. However, when combined with EEG activity, none of the physiological signals provided supplementary information to the classifier. Finally, we evaluated whether this pre-stimulus influence modulated stimulus coding and could help the classifier to discriminate between the two classes of images. Prior studies suggest that specific fluctuations in neural activity may favor recognition of different categories. While these studies focused on neural activity, we extended our analyses to include physiological activity as well. The results of this analysis suggest that pre-stimulus activity does not modulate the way the stimulus is decoded. Overall, the results suggest that pre-stimulus activity influences the recognition of upcoming stimuli but that the changes in body physiology observed are already reflected in the pre-stimulus EEG activity and do not modulate the neural representation of the stimulus. |
|---|
Atividades financiadas
Não foram encontrados projetos financiados.