Authors: Roofia Galeshi; Gary Skaggs
Addresses: Mathematics Education Department, School of Teacher Education and Leadership, Radford University, P.O. Box 6959, Radford, VA 24142, USA ' School of Education (0302), Virginia Tech, Blacksburg, VA 24061, USA
Abstract: Although item response theory is a valuable summative assessment tool, cognitive diagnostic models could potentially be a valuable instructional and formative assessment instrument. Using simulation methods, we have examined one of the existing additive models - the compensatory reparameterised unified model (CRUM) - for its parameter recovery and classification accuracy under various research designs. In order for models such as the CRUM to be useful, it is essential to establish their parameter recovery and classification accuracy. This study simulated data for sample sizes of 50, 100, 500, 1,000, 5,000, and 10,000 examinees with various attribute-item combinations: 7 items with 3 attributes, 15 items with 4 attributes, 31 items with 5 attributes, and 35 items with 3 attributes to be evaluated for its parameter recovery. The results suggest that the accuracy of measuring examinees' attribute mastery depends heavily on the combination of the number of items and attributes as well as the sample size. Item parameter estimation was almost equally accurate for datasets of sample sizes greater than 100. The more complex items required larger sample sizes to maintain its consistency of parameter estimation.
Keywords: cognitive modelling; cognitive diagnostic model; CDM; parameter recovery; compensatory reparameterised unified model; CRUM; simulation; classification accuracy; research design.
International Journal of Quantitative Research in Education, 2016 Vol.3 No.4, pp.223 - 241
Accepted: 19 Aug 2015
Published online: 22 Feb 2017 *