Article: Parameter recovery of a cognitive diagnostic model: evidence from a simulation study Journal: International Journal of Quantitative Research in Education (IJQRE) 2016 Vol.3 No.4 pp.223 - 241 Abstract: Although item response theory is a valuable summative assessment tool, cognitive diagnostic models could potentially be a valuable instructional and formative assessment instrument. Using simulation methods, we have examined one of the existing additive models - the compensatory reparameterised unified model (CRUM) - for its parameter recovery and classification accuracy under various research designs. In order for models such as the CRUM to be useful, it is essential to establish their parameter recovery and classification accuracy. This study simulated data for sample sizes of 50, 100, 500, 1,000, 5,000, and 10,000 examinees with various attribute-item combinations: 7 items with 3 attributes, 15 items with 4 attributes, 31 items with 5 attributes, and 35 items with 3 attributes to be evaluated for its parameter recovery. The results suggest that the accuracy of measuring examinees' attribute mastery depends heavily on the combination of the number of items and attributes as well as the sample size. Item parameter estimation was almost equally accurate for datasets of sample sizes greater than 100. The more complex items required larger sample sizes to maintain its consistency of parameter estimation. Inderscience Publishers - linking academia, business and industry through research

Title: Parameter recovery of a cognitive diagnostic model: evidence from a simulation study

Authors: Roofia Galeshi; Gary Skaggs

Addresses: Mathematics Education Department, School of Teacher Education and Leadership, Radford University, P.O. Box 6959, Radford, VA 24142, USA ' School of Education (0302), Virginia Tech, Blacksburg, VA 24061, USA

Abstract: Although item response theory is a valuable summative assessment tool, cognitive diagnostic models could potentially be a valuable instructional and formative assessment instrument. Using simulation methods, we have examined one of the existing additive models - the compensatory reparameterised unified model (CRUM) - for its parameter recovery and classification accuracy under various research designs. In order for models such as the CRUM to be useful, it is essential to establish their parameter recovery and classification accuracy. This study simulated data for sample sizes of 50, 100, 500, 1,000, 5,000, and 10,000 examinees with various attribute-item combinations: 7 items with 3 attributes, 15 items with 4 attributes, 31 items with 5 attributes, and 35 items with 3 attributes to be evaluated for its parameter recovery. The results suggest that the accuracy of measuring examinees' attribute mastery depends heavily on the combination of the number of items and attributes as well as the sample size. Item parameter estimation was almost equally accurate for datasets of sample sizes greater than 100. The more complex items required larger sample sizes to maintain its consistency of parameter estimation.

Keywords: cognitive modelling; cognitive diagnostic model; CDM; parameter recovery; compensatory reparameterised unified model; CRUM; simulation; classification accuracy; research design.

DOI: 10.1504/IJQRE.2016.082386

International Journal of Quantitative Research in Education, 2016 Vol.3 No.4, pp.223 - 241

Accepted: 19 Aug 2015
Published online: 22 Feb 2017 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article

Title: Parameter recovery of a cognitive diagnostic model: evidence from a simulation study

Keep up-to-date