Title: Short answer scoring using W-Bleu for regular assessment in vocational high school

Authors: Feddy Setio Pribadi; Teguh Bharata Adji; Adhistya Erna Permanasari; Takashi Ninomiya

Addresses: Department of Electrical Engineering and Information Technology, Universitas Gadjah Mada, Yogyakarta, Indonesia; Department of Electrical Engineering, Universitas Negeri Semarang, Semarang, Indonesia ' Department of Electrical Engineering and Information Technology, Universitas Gadjah Mada, Yogyakarta, Indonesia ' Department of Electrical Engineering and Information Technology, Universitas Gadjah Mada, Yogyakarta, Indonesia ' Department of Computer Science, Ehime University, Japan

Abstract: The assessment process in a vocational school needs tools that can integrate both knowledge assessment and skill assessment, among which are the construct response question. One example of the construct response question is automated short answer scoring. However, there is a problem with automated short-answer scoring that is how to overcome the difference in sentence length between student answer and reference answer that has the same meaning. The difference in the length of these sentences occurs because each student provides answers with different additional information. This paper proposes a method for handling the problem by weighting the key words that make up the sentence in the reference answer. This weighting process is done by modifying the Bleu method into weighted bleu (W-Bleu). The proposed method produces a process of automated short answer scoring that obtains a correlation value of 0.64 and root mean square error of 1.01.

Keywords: short answer scoring; vocational high school; weighted bleu; automatic assessment.

DOI: 10.1504/IJIL.2018.095368

International Journal of Innovation and Learning, 2018 Vol.24 No.4, pp.437 - 447

Received: 10 Jan 2018
Accepted: 06 Feb 2018
Published online: 03 Oct 2018 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article