Title: Designing user interfaces with gestures and sound: Towards the performance and appeal of voice mail browsing

Authors: Marco C. Rozendaal, David V. Keyson

Addresses: School of Industrial Design Engineering, Delft University of Technology, Landbergstraat 15, 2628 CE Delft, The Netherlands. ' School of Industrial Design Engineering, Delft University of Technology, Landbergstraat 15, 2628 CE Delft, The Netherlands

Abstract: This paper explores the use of sound and gestures within a tangible user interface, towards influencing the performance and appeal of browsing through a voice mail collection. Three browsers were developed, in which speech and non-speech sounds were projected in a sound-scape above a tangible interface. For each of the browsers, users were given the task of locating voice mail messages. Search behaviour in terms of gestures, task time, and errors, were recorded and categorised. Users| experience was measured in terms of appeal, consisting of hedonic and ergonomic qualities. Results showed that users| explorative behaviour changed, depending on the specific type of browser. Subjects were observed to utilise non-speech sound cues effectively in searching for target voice mail messages. Subjective results showed that ergonomic and hedonic qualities of appeal were found to interact in an averaging way, such that overall appeal was similar across conditions. Implications towards designing multimodal interfaces are discussed.

Keywords: multimodal interaction; sound; gestures; tangible interfaces; representation; user experience; user interfaces; interface design; voice mail browsing; ergonomics; human factors; hedonic; cues; multimodal interfaces; content access.

DOI: 10.1504/JDR.2006.010797

Journal of Design Research, 2006 Vol.5 No.1, pp.96 - 115

Published online: 04 Sep 2006 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article