Title: Traditional fit indices utility in new psychometric model: cognitive diagnostic model

Authors: Roofia Galeshi; Gary Skaggs

Addresses: School of Education, Virginia Polytechnic Institute and State University, 434 Durham Hall, 1145 Perry Street, Blacksburg, VA 24061, USA ' School of Education, Virginia Polytechnic Institute and State University, 434 Durham Hall, 1145 Perry Street, Blacksburg, VA 24061, USA

Abstract: Among compensatory cognitive diagnostic models (CDMs), the compensatory reparameterised unified model (CRUM) has shown an accurate item parameter recovery in simulation studies. This article presents the results of multiple simulation studies examining the performance of the commonly used relative fit indices in determining the model to data fit for CRUM. Its objectives were two-fold: first, to provide a degree of assurance for CRUM's plausible accuracy in practice, and second, to provide a direction for future research. We evaluated the sensitivity of the AIC, BIC, and ssaBIC in identifying model misfit/selection for six sample sizes of 10,000, 5,000, 1,000, 500, 100, and 50 with various test length/number of attribute under two extreme Q-matrix misspecifications - over-fit and complete reverse Q-matrices. The results indicated that the BIC and AIC indices performed similarly for larger datasets (N ≥ 500) but varied for smaller datasets (N < 500) - suggesting a superior performance for BIC.

Keywords: compensatory reparameterised unified model; CRUM; traditional fit indices; Q-matrix misspecification; cognitive diagnostic models; CDM; psychometric modelling; parameter recovery; simulation.

DOI: 10.1504/IJQRE.2014.064388

International Journal of Quantitative Research in Education, 2014 Vol.2 No.2, pp.113 - 132

Received: 24 Jun 2013
Accepted: 26 Nov 2013

Published online: 30 Aug 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article