Title: A DIF detection procedure in multidimensional item response theory framework using MCMC technique

Authors: Lihua Yao; Feiming Li

Addresses: Defense Manpower Data Center Monterey Bay, 400 Gigling Rd, Seaside, Ca 93955, USA ' University of North Texas Health Science Center, North Texas Health Science Center, 3500 Camp Bowie Blvd., Fort Worth, TX 76107, USA

Abstract: DIF detection procedures are available not only in the classical test theory framework, but also in the IRT-based framework, such as Lord's (1980) chi-square method, Raju's (1988) area measures, and the likelihood ratio test. These methods are very useful in detecting DIF, however, little progress has been made in understanding the causes of DIF. Benign DIF caused by auxiliary dimensions enhances construct validity of a test, while adverse DIF resulting from nuisance dimensions lowers construct validity. Benign DIF items cannot be detected by conducting an additional DIF analysis in which all construct-relevant dimensions are modelled and included in the conditioning variable. Adverse DIF, however, can be eliminated only by deleting the item or by revising it. Therefore, it is very important to have a procedure for investigating the cause of DIF and detecting adverse DIF only. In this study, we developed a DIF detection procedure in multidimensional item response theory framework using MCMC to flag only those items that have adverse DIF. Items of benign DIF detected by other procedures will not be flagged. The DIF detection procedure proposed in this study was applied to both real data and simulated data and was found to be successful.

Keywords: BMIRT; DIF detection; Markov chain Monte Carlo; MCMC; multidimensional IRT; item response theory; MIRT; multi-group; adverse DIF; examinee ability; psychometrics.

DOI: 10.1504/IJQRE.2015.071734

International Journal of Quantitative Research in Education, 2015 Vol.2 No.3/4, pp.285 - 304

Received: 22 Aug 2014
Accepted: 05 May 2015

Published online: 16 Sep 2015 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article