Joyful for you and tender for us: the influence of individual characteristics and language on emotion labeling and classification (ISMIR 2020)
This work was made with Estefanía Cano, Perfecto Herrera, and Emilia Gómez, and was funded by the TROMPA EU project. It was presented in the 21th International Society for Music Information Retrieval Conference (ISMIR).
Abstract
Tagging a musical excerpt with an emotion label may result in a vague and ambivalent exercise. This subjectivity entangles several high-level music description tasks when the computational models built to address them produce predictions on the basis of a “ground truth”. In this study, we investigate the relationship between emotions perceived in pop and rock music (mainly in Euro-American styles) and personal characteristics from the listener, using language as a key feature. Our goal is to understand the influence of lyrics comprehension on music emotion perception and use this knowledge to improve Music Emotion Recognition (MER) models.
We systematically analyze over 30K annotations of 22 musical fragments to assess the impact of individual differences on agreement, as defined by Krippendorff’s alpha coefficient. We employ personal characteristics to form group-based annotations by assembling ratings with respect to listeners’ familiarity, preference, lyrics comprehension, and music sophistication. Finally, we study our group-based annotations in a two-fold approach: (1) assessing the similarity within annotations using manifold learning algorithms and unsupervised clustering, and (2) analyzing their performance by training classification models with diverse “ground truths”. Our results suggest that a) applying a broader categorization of taxonomies and b) using multi-label, group-based annotations based on language, can be beneficial for MER models.
You can see a presentation of our work in the following video:
Links to paper and poster and open source code.