The Brain Representational Dynamics of Perceived Voice Emotions

Post by Anna Cranston

What's the science?

There are two main theories on the way humans perceive emotional stimuli. The first theory is that we perceive emotions as discrete categories (e.g. an angry or happy voice), and the other theory suggests that we perceive emotions along several continuous dimensions (e.g. an intense and negative voice (angry) or an intense and positive voice (happy). However, which of these theories provides the truest picture of perception of emotional and vocal stimuli is yet to be fully understood. This week in Nature Human Behaviour, Giordano and colleagues investigate the exact mechanistic underpinnings of our perception of human vocal emotion.

How did they do it?

The authors recruited ten healthy adult participants with normal hearing. The participants were exposed to synthetic voice samples generated by morphing between recordings of brief affective bursts (from the Montreal Affective Voices database). These affective bursts portrayed either an emotionally neutral intention, expressed anger, disgust, fear, or pleasure. The authors combined pairs of these emotional expressions with different weights, resulting in 39 voice samples that represented a wide range of perceived emotions. Changing or ‘morphing’ these weights resulted in variation in the category of perceived emotion. Participants were asked to rate the dissimilarity between the samples — in other words, their perception of the varying degrees of emotion in the recorded vocalizations. They were also asked to rate their response to each vocalization on arousal (low to high), valence (negative to positive), and emotional intensity for four emotions (anger, disgust, fear and pleasure, low to high). 

fMRI and magnetoencephalography (MEG) neuroimaging data were collected from each participant during each behavioral session. The authors utilized a modulation transfer function (MTF), which is an acoustics-driven computational model of the cortical representation of complex sounds, to model the brain’s response to varying vocal stimuli in participants. The authors then used representation dissimilarity matrices (RDMs) to measure the acoustic specification of perceived emotions at specific time points. Finally, the authors applied a specific type of analysis known as representation similarity analysis (RSA) to assess the spatio-temporal representation of acoustics in the brain of each participant using the fMRI and MEG data in response to each perceived emotion.

What did they find?

The authors first examined the behavioural data (ratings of dissimilarity) using representational dissimilarity matrices, and found that both categories and dimensions influence the perception of voices, however, a categorical approach better accounted for perceived differences in the emotional vocalizations. When the authors used representational dissimilarity matrices to map the cerebral activation or ‘geometry’ for each participant using data obtained via fMRI and MEG recordings, they found that: a) brain activity immediately (~115 milliseconds) after sound onset was best explained by the categories the participant perceived, and b) later (>0.5 seconds after onset) brain activity in different brain regions was best explained by dimensions. Finally, the authors also identified correlations in perception and brain activation in response to acoustics, suggesting that key cerebral regions are activated in response to distinct emotional stimuli.

What's the impact?

The authors found that through a combined neuroimaging and behavioral assessment, they were able to spatiotemporally map the cerebral activation in the brain in response to vocal emotions as either distinct categories or dimensional states. This study sheds new light on the mechanistic underpinnings of vocal emotions, and our understanding of cerebral dynamics in relation to our perception of vocal stimuli.

Giordano et al. The representational dynamics of perceived voice emotions evolve from categories to dimensions. Nature Human Behaviour (2021). Access the original scientific publication here