Title: The effect of onset asynchrony in audio-visual speech and the Uncanny Valley in virtual characters

Authors: Angela Tinwell; Mark Grimshaw; Deborah Abdel Nabi

Addresses: Research and Graduate School, The University of Bolton, Deane Road, Bolton, BL3 5AB, UK ' Aalborg University, Institute for Communication, Rensburggade 14, 9000 Aalborg, Denmark ' Faculty of Wellbeing and Social Sciences, The University of Bolton, Deane Road, Bolton, BL3 5AB, UK

Abstract: This study investigates if the Uncanny Valley phenomenon is increased for realistic, human-like characters with an asynchrony of lip movement during speech. An experiment was conducted in which 113 participants rated, a human and a realistic, talking-head, human-like, virtual character over a range of onset asynchronies for both perceived familiarity and human-likeness. The results show that virtual characters were regarded as more uncanny (less familiar and human-like) than humans and that increasing levels of asynchrony increased perception of uncanniness. Interestingly, participants were more sensitive to the uncanny in characters when the audio stream preceded the visual stream than with asynchronous footage where the video stream preceded the audio stream. This paper considers possible psychological explanations as to why the magnitude and direction of an asynchrony of speech dictates magnitude of perceived uncanniness and the implications of this in character design.

Keywords: onset asynchrony; audio-visual speech; Uncanny Valley; virtual characters; realism; human-like; lip-sync; speech asynchrony; lip movement; 3D environments; video games; strangeness; familiarity; audio streaming; visual streaming; viewer perception; digital human; psychology.

DOI: 10.1504/IJMRS.2015.068991

International Journal of Mechanisms and Robotic Systems, 2015 Vol.2 No.2, pp.97 - 110

Accepted: 15 Nov 2012
Published online: 09 May 2015 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article