Title: Using cluster analysis to explore students' interactions with automated feedback in an online Earth science task

Authors: Mengxiao Zhu; Ou Lydia Liu; Hee-Sun Lee

Addresses: Educational Testing Service, 660 Rosedale Road, Princeton, NJ 08541, USA ' Educational Testing Service, 660 Rosedale Road, Princeton, NJ 08541, USA ' Concord Consortium, 25 Love Ln, Concord, MA 01742, USA

Abstract: Digitally delivered tasks can provide students opportunities to interact with disciplinary content while their interactions with the tasks can be recorded as time-stamped log events in the system server. Through post-hoc analysis of log data, we can re-enact and discover patterns in students' activities. This study addresses the online Earth science module where students engaged in writing and revising scientific arguments in a structured format. We adopted natural language processing (NLP) techniques to analyse students' responses, which enabled us to provide immediate feedback to students on their responses and revisions. Cluster analyses were conducted on the action sequences in four argumentation tasks embedded in the module. For each task, the cluster analyses identified two clusters of students who showed different revision patterns with allocation of time on different items. In addition, students in those two clusters also differed in their initial item scores and item score changes after revision.

Keywords: log data analysis; automated scoring; automated feedback; cluster analysis; scientific argumentation.

DOI: 10.1504/IJQRE.2020.111452

International Journal of Quantitative Research in Education, 2020 Vol.5 No.2, pp.111 - 135

Received: 13 Jan 2019
Accepted: 06 Oct 2019

Published online: 27 Nov 2020 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article