Title: Prediction of item psychometric indices from item characteristics automatically extracted from the stem and option text

Authors: Carmen Garcia, Vicente Ponsoda, Alejandro Sierra

Addresses: Facultad de Psicologia, Universidad Autonoma de Madrid, C/ Ivan Pavlov, 6, Campus de Cantoblanco, 28049 Madrid, Spain. ' Facultad de Psicologia, Universidad Autonoma de Madrid, C/ Ivan Pavlov, 6, Campus de Cantoblanco, 28049 Madrid, Spain. ' Escuela Politecnica Superior, Universidad Autonoma de Madrid, C/ Francisco Tomas y Valiente, 11, Campus de Cantoblanco, 28049 Madrid, Spain

Abstract: This study has four main parts: first, some of the 31 guidelines proposed by Haladyna et al. (2002) were converted to 39 features that can be automatically extracted from a text file containing the test stem and options. Second, text files of a few multiple choice university exams were collected and the 39 features were extracted. Third, the difficulty and discrimination indices of each item were computed. Finally, linear regression was applied to find which features predict the items| psychometric indices. The number of special adverbs (always, never, …) in the distracters and the discrimination index show the strongest link.

Keywords: multiple choice items; item writing guidelines; item difficulty; item discrimination; item parameter prediction; automatic item reading; psychometric indices; item characteristics; stem texts; option texts; Thomas Haladyna; text files; test stems; university exams; examinations; higher education; universities; difficulty indices; discrimination indices; linear regression; special adverbs; distracters; psychology students; Spain; continuing education; life-long learning; automatic free-text evaluation.

DOI: 10.1504/IJCEELL.2011.040199

International Journal of Continuing Engineering Education and Life-Long Learning, 2011 Vol.21 No.2/3, pp.210 - 221

Published online: 16 Oct 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article