15th International Congress of Phonetic Sciences (ICPhS-15)

Barcelona, Spain
August 3-9, 2003

Evaluation of a Multilingual Synthetic Talking Face as a Communication Aid for the Hearing Impaired

Catherine Siciliano (1), Geoff Williams (1), Jonas Beskow (2), Andrew Faulkner (1)

(1) University College London, UK
(2) KTH, Sweden

The Synface project is developing a synthetic talking face to aid the hearing impaired in telephone conversation. This report investigates the gain in intelligibility from the synthetic talking head when controlled by hand-annotated speech. Audio from Swedish, English and Dutch sentences was degraded to simulate the information losses that arise in severe-to-profound hearing impairment. 12 normal-hearing native speakers for each language took part. Auditory signals were presented alone, with the synthetic face, and with a video of the original talker. Purely auditory intelligibility was low. With the addition of the synthetic face, average intelligibility increased by 20%. Scores with the synthetic face were significantly lower than for the natural face for English and Dutch, but not Swedish. Visual identification of English consonants showed that the synthetic face fell short of a natural face on both place and manner of articulation. This information will be used to improve the synthesis.

Full Paper

Bibliographic reference.  Siciliano, Catherine / Williams, Geoff / Beskow, Jonas / Faulkner, Andrew (2003): "Evaluation of a multilingual synthetic talking face as a communication aid for the hearing impaired", In ICPhS-15, 131-134.