15th International Congress of Phonetic Sciences (ICPhS-15)

Barcelona, Spain
August 3-9, 2003

Interpretation of Emotions in Natural Speech - A Comparison Between Written, Auditive and Gestural Information

Åsa Abelin

Göteborg University, Sweden

This study concerns the interpretation of emotion in natural dialogue in Swedish. The aim of this study is to investigate the role of different types of modalities. The modalities studied were: visual/written (lexico-grammatical information), auditive (adding prosodic information), and visual (adding body language). 32 listeners followed 3,5 minutes of dialogue between four persons and interpreted the emotions expressed. The results are: 1) interpretations are similar across modalities for some utterances, but vary for others. However, all listeners do not make the same interpretation in the different modalities. 2) considering all interpretations, listeners made more interpretations in the written and auditive modalites than in the visual, but the interpretations in the auditive and the visual modalities are more similar. The four most common classifications in all three modalities were very similar. 3) some interpretations were very frequent in one modality but infrequent or absent in others.

Full Paper

Bibliographic reference.  Abelin, Åsa (2003): "Interpretation of emotions in natural speech - a comparison between written, auditive and gestural information", In ICPhS-15, 837-840.