15th International Congress of Phonetic Sciences (ICPhS-15)

Barcelona, Spain
August 3-9, 2003

Perception Difficulties and Errors in Multimodal Speech: The Case of Consonants

Azra N. Ali

University of Huddersfield, UK

This study focuses on the perception difficulties and errors of audiovisual presentations of talking heads, and examines the effect of noise and temporal misalignment of channels for two groups of consonants; onsets and codas in monosyllabic words. The study confirms that incongruence of audio and visual channels cases McGurk fusion even in the linguistic context of monosyllabic words. The results show that, in coda position, incongruent consonant stimuli elicit fusion more readily than those in the onset position. In all cases with incongruence, subjects are found to take more time to reach decisions than with congruent stimuli.
   It is argued that decision times - which we measure in two different ways - are indicators of perceptual difficulty experienced by subjects when interpreting a stimulus.

Full Paper

Bibliographic reference.  Ali, Azra N. (2003): "Perception difficulties and errors in multimodal speech: the case of consonants", In ICPhS-15, 2317-2320.