Open Access Article

Title: A framework for effectively utilising human grading input in automated short answer grading

Authors: Andrew Kwok-Fai Lui; Sin-Chun Ng; Stella Wing-Nga Cheung

Addresses: School of Science and Technology, Hong Kong Metropolitan University, Hong Kong SAR, China ' School of Science and Technology, Hong Kong Metropolitan University, Hong Kong SAR, China ' School of Science and Technology, Hong Kong Metropolitan University, Hong Kong SAR, China

Abstract: Short answer questions are effective for recall knowledge assessment. Grading a large amount of short answers is costly and time consuming. To apply short answer questions on MOOCs platforms, the issues of scalability and responsiveness must be addressed. Automated grading uses a computing process and a machine learning grading model to classify answers into correct, wrong, and other levels of correctness. The divide-and-grade approach is proven effective in reducing the annotation effort needed for the learning the grading model. This paper presents an improvement on the divide-and-grade approach that is designed to increase the utility of human actions. A novel short answer grading framework is proposed that addresses the selection of impactful answers for grading, the injection of the ground-truth grades for steering towards purer final clusters, and the final grade assignments. Experiment results indicate the grading quality can be improved with the same level of human actions.

Keywords: automated short answer grading; clustering; semi-supervised clustering; MOOCs; automated grading.

DOI: 10.1504/IJMLO.2022.10048071

International Journal of Mobile Learning and Organisation, 2022 Vol.16 No.3, pp.266 - 286

Received: 04 Nov 2020
Accepted: 08 Mar 2021

Published online: 09 Jun 2022 *