Title: Bayesian modelling of differential item functioning: type I error and power rates in the presence of non-normal ability distributions, impact, and anchor set contamination

Authors: W. Holmes Finch; Brian F. French

Addresses: Department of Educational Psychology, Ball State University, TC 521, Muncie, IN 47306, USA ' Department of Educational Leadership and Counseling Psychology, Cleveland Hall, Washington State University, Pullman, Washington, 99164, USA

Abstract: Differential item functioning (DIF) refers to the case where performance on an item differs between groups of examinees when the underlying latent trait being assessed is held constant. The DIF literature supports the conclusion that standard detection methods are effective across many conditions; however, there are other situations in which these methods lack accuracy. There has been increased interest in the use of Bayesian estimation across a wide area of statistical practice, including in educational measurement. Bayesian methods may be able to assist with the lack of accuracy. The goal of this simulation study was to examine the performance of Bayesian models with informative priors for detecting DIF under a variety of conditions. Results illustrate a positive performance of two such methods, Bayesian versions of logistic regression, and the MIMIC model. Type I error rates for these approaches were controlled when the distribution of θ differed between groups, while power rates were comparable across methods and conditions. This paper concludes with a discussion of the implications for practice, and directions for future research.

Keywords: differential item functioning; DIF; Bayesian modelling; validity; type I error; power rates; non-normal ability distributions; anchor set contamination; simulation.

DOI: 10.1504/IJQRE.2013.058305

International Journal of Quantitative Research in Education, 2013 Vol.1 No.4, pp.341 - 363

Received: 05 Nov 2012
Accepted: 24 May 2013

Published online: 29 Apr 2014 *

Full-text access for editors Full-text access for subscribers Purchase this article Comment on this article