Publication
MERGE Audio: Music Emotion Recognition next Generation – Audio Classification with Deep Learning
| Summary: | The growing Music Emotion Recognition research field is evolving accompanied by an already massive and expanding library of digital music, which raises the need for it to be segmented and organized. Traditional Machine Learning approaches to identify perceived emotion in music are based on carefully crafted features that have dominated this field and brought state-of-the-art results. Our goal was to approach this field with Deep Learning (DL), as it can skip this expensive feature design by automatically extracting features. We propose a Deep Learning approach to the existing static 4QAED dataset, which achieved a state-of-the-art F1-Score of 88.45%. This model consisted in a hybrid approach with a Dense Neural Network (DNN) and a Convolutional Neural Network (CNN) for the features and melspectrograms (converted from audio samples), respectively. Additionally, different methods of data augmentation were experimented with for the static MER problem, using a Generative Adversarial Neural Network (GAN) and classical audio augmentation, which improved the overall performance of the model. Other pre-trained models were also tested (i.e. VGG19 and a CNN trained for music genre recognition). The Music Emotion Variation Detection field was explored as well, with (Bidirectional) Long Short Term Memory layers in combination with pre-trained CNN models, as we consider that the perceived emotion can change throughout the song. This research gave us a good insight into several distinct deep learning approaches resulting in a new state-of-the-art result with the 4QAED dataset, in addition to getting to know the limitations of both datasets. |
|---|---|
| Main Authors: | Sá, Pedro Marques Alegre de |
| Subject: | deep learning audio augmentation music emotion recognition music emotion variation detection deep learning aumento de dados de audio reconhecimento de emoção na música reconhecimento da variação da emoção na música |
| Year: | 2021 |
| Country: | Portugal |
| Document type: | master thesis |
| Access type: | open access |
| Associated institution: | Universidade de Coimbra |
| Language: | English |
| Origin: | Estudo Geral - Universidade de Coimbra |
| Summary: | The growing Music Emotion Recognition research field is evolving accompanied by an already massive and expanding library of digital music, which raises the need for it to be segmented and organized. Traditional Machine Learning approaches to identify perceived emotion in music are based on carefully crafted features that have dominated this field and brought state-of-the-art results. Our goal was to approach this field with Deep Learning (DL), as it can skip this expensive feature design by automatically extracting features. We propose a Deep Learning approach to the existing static 4QAED dataset, which achieved a state-of-the-art F1-Score of 88.45%. This model consisted in a hybrid approach with a Dense Neural Network (DNN) and a Convolutional Neural Network (CNN) for the features and melspectrograms (converted from audio samples), respectively. Additionally, different methods of data augmentation were experimented with for the static MER problem, using a Generative Adversarial Neural Network (GAN) and classical audio augmentation, which improved the overall performance of the model. Other pre-trained models were also tested (i.e. VGG19 and a CNN trained for music genre recognition). The Music Emotion Variation Detection field was explored as well, with (Bidirectional) Long Short Term Memory layers in combination with pre-trained CNN models, as we consider that the perceived emotion can change throughout the song. This research gave us a good insight into several distinct deep learning approaches resulting in a new state-of-the-art result with the 4QAED dataset, in addition to getting to know the limitations of both datasets. |
|---|
Funded activities
No funded projects found.