Title: Posterior predictive model checks for cognitive diagnostic models

Authors: Jung Yeon Park; Matthew S. Johnson; Young-Sun Lee

Addresses: Department of Human Development, Teachers College, Columbia University, Box 118, 525 W120th St., New York, NY 10027, USA ' Department of Human Development, Teachers College, Columbia University, Box 118, 525 W120th St., New York, NY 10027, USA ' Department of Human Development, Teachers College, Columbia University, Box 118, 525 W120th St., New York, NY 10027, USA

Abstract: Cognitive diagnostic models (CDMs; DiBello et al., 2007) have received increasing attention in educational measurement for the purpose of diagnosing examinees' strengths and weaknesses of their latent attributes. Despite the current popularity of a number of diagnostic models, research on assessing model-data fit has been limited. The current study applies one of the Bayesian model checking methods, namely the posterior predictive model check (PPMC) method (Rubin, 1984) to investigate model misfit. Specifically, we aim to employ the technique to investigate model-data misfit from various diagnostic models using real data and a simulation study. An important issue with the application of PPMC is the choice of discrepancy measure. This study examines the performance of three discrepancy measures for assessing different aspects of model fit: observed total-scores distribution, association of item pairs, and correlation of attribute pairs as adequate measures for the diagnostic models.

Keywords: posterior predictive model checking; cognitive diagnostic models; CDM; deterministic-input noisy-and-gate; DINA; discrepancy measures; general diagnostic modelling; psychometrics; examinee ability; Bayesian model checking; model misfit; model-data misfit; simulation; model fit.

DOI: 10.1504/IJQRE.2015.071738

International Journal of Quantitative Research in Education, 2015 Vol.2 No.3/4, pp.244 - 264

Received: 07 Nov 2014
Accepted: 09 Jun 2015
Published online: 16 Sep 2015 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article