Title: Using cluster analysis to explore students' interactions with automated feedback in an online Earth science task
Authors: Mengxiao Zhu; Ou Lydia Liu; Hee-Sun Lee
Addresses: Educational Testing Service, 660 Rosedale Road, Princeton, NJ 08541, USA ' Educational Testing Service, 660 Rosedale Road, Princeton, NJ 08541, USA ' Concord Consortium, 25 Love Ln, Concord, MA 01742, USA
Abstract: Digitally delivered tasks can provide students opportunities to interact with disciplinary content while their interactions with the tasks can be recorded as time-stamped log events in the system server. Through post-hoc analysis of log data, we can re-enact and discover patterns in students' activities. This study addresses the online Earth science module where students engaged in writing and revising scientific arguments in a structured format. We adopted natural language processing (NLP) techniques to analyse students' responses, which enabled us to provide immediate feedback to students on their responses and revisions. Cluster analyses were conducted on the action sequences in four argumentation tasks embedded in the module. For each task, the cluster analyses identified two clusters of students who showed different revision patterns with allocation of time on different items. In addition, students in those two clusters also differed in their initial item scores and item score changes after revision.
Keywords: log data analysis; automated scoring; automated feedback; cluster analysis; scientific argumentation.
DOI: 10.1504/IJQRE.2020.111452
International Journal of Quantitative Research in Education, 2020 Vol.5 No.2, pp.111 - 135
Received: 13 Jan 2019
Accepted: 06 Oct 2019
Published online: 27 Nov 2020 *