Authors: Roofia Galeshi; Gary Skaggs
Addresses: School of Education, Virginia Polytechnic Institute and State University, 434 Durham Hall, 1145 Perry Street, Blacksburg, VA 24061, USA ' School of Education, Virginia Polytechnic Institute and State University, 434 Durham Hall, 1145 Perry Street, Blacksburg, VA 24061, USA
Abstract: Among compensatory cognitive diagnostic models (CDMs), the compensatory reparameterised unified model (CRUM) has shown an accurate item parameter recovery in simulation studies. This article presents the results of multiple simulation studies examining the performance of the commonly used relative fit indices in determining the model to data fit for CRUM. Its objectives were two-fold: first, to provide a degree of assurance for CRUM's plausible accuracy in practice, and second, to provide a direction for future research. We evaluated the sensitivity of the AIC, BIC, and ssaBIC in identifying model misfit/selection for six sample sizes of 10,000, 5,000, 1,000, 500, 100, and 50 with various test length/number of attribute under two extreme Q-matrix misspecifications - over-fit and complete reverse Q-matrices. The results indicated that the BIC and AIC indices performed similarly for larger datasets (N ≥ 500) but varied for smaller datasets (N < 500) - suggesting a superior performance for BIC.
Keywords: compensatory reparameterised unified model; CRUM; traditional fit indices; Q-matrix misspecification; cognitive diagnostic models; CDM; psychometric modelling; parameter recovery; simulation.
International Journal of Quantitative Research in Education, 2014 Vol.2 No.2, pp.113 - 132
Received: 24 Jun 2013
Accepted: 26 Nov 2013
Published online: 19 Aug 2014 *