Title: Multimethod assessment of affective experience and expression during deep learning

Authors: Sidney K. D'Mello, Scotty D. Craig, Art C. Graesser

Addresses: Department of Computer Science, University of Memphis, Memphis, TN 38152, USA. ' Department of Psychology, University of Memphis, Memphis, TN 38152, USA. ' Department of Psychology, University of Memphis, Memphis, TN 38152, USA

Abstract: Inquiries into the link between affect and learning require robust methodologies to measure the learner|s affective states. We describe two studies that utilised either an online or offline methodology to detect the affective states of a learner during a tutorial session with AutoTutor. The online study relied on self-reports for affect judgements, while the offline study considered the judgements by the learner, a peer and two trained judges. The studies also investigated the relationships between facial features, conversational cues and emotional expressions in an attempt to scaffold the development of computer algorithms to automatically detect learners| emotions. Both methodologies showed that boredom, confusion and frustration are the prominent affective states during learning with AutoTutor. For both methodologies, there were also some relationships involving patterns of facial activity and conversational cues that were diagnostic of emotional expressions.

Keywords: learning technology; student affect; learner emotions; dialogue; facial features; confusion; frustration; boredom; emote-aloud; judgements; facial features; conversational cues; emotional expressions.

DOI: 10.1504/IJLT.2009.028805

International Journal of Learning Technology, 2009 Vol.4 No.3/4, pp.165 - 187

Published online: 03 Oct 2009 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article