International Journal of Quantitative Research in Education (8 papers in press)
Psychometric examination of the Academic Motivation Scale using a Vietnamese university student sample
by Lan Luong, Arthur Poropat, Helen Klieve, Kate Thompson
Abstract: Massification of higher education in Vietnam has brought about both achievements and various issues. Efforts have been made to improve the quality of teaching and learning in higher education, but very little attention has been paid to the issue of student motivation. This study aims to contribute to this area of knowledge by testing the applicability of the Academic Motivation Scale (AMS) in assessing Vietnamese university students motivation. This was achieved via evaluation of the scales psychometric properties using data obtained from 648 first year students from a high-ranking university. Using a different approach in model testing, results indicated that the revised seven factor AMS with 23 items best fitted the data. All subscales had satisfactory reliabilities. Thus, the revised AMS can be used to study Vietnamese university students motivation.
Keywords: academic motivation; Academic Motivation Scale; Vietnam; university; self-determination theory.
A Comparison of Linear and Equipercentile Equating and IRT Equating with FIPC Across Multidimensional Test Forms for Nonequivalent Groups
by Ki Cole, Sohee Kim, Mwarumba Mwavita
Abstract: The aim of this study is to compare linear equating procedures (i.e., Levine and Tucker methods), equipercentile equating, and the fixed item parameter calibration (FIPC) with true and observed score equating methods when applied non-equivalent groups taking forms which have various multidimensional structures: equal or unequal total test difficulty and similar or dissimilar difficulty within dimensions across forms. This situation may be common for large-scale test forms that are composed of multiple sub-content areas and are being administered to multiple samples at different times. Within the specifications of this study, when forms differ in total difficulty but do not have confounded difficulty within dimensions, the linear methods may be preferred over the equipercentile and FIPC methods. When forms differ in average item difficulty within dimensions, regardless of equal or unequal total test difficulty, the FIPC method is favored over the equipercentile method, when data were correlated.
Keywords: Equating; Psychometrics; Test construction.
The Analysis of MCQs in a Newly Developed Reading Comprehension Ability Test: A Study on Yemeni undergraduate EFL learners
by Iftikhar Al-Ariqi, Jagannath Dange, Mohsin Mir
Abstract: Studies have shown that the best way to test the students ability in reading comprehension is the Multiple-choice questions (MCQs) for its validity and reliability. The efficiency of MCQs as an efficient tool for evaluation solely rests upon their quality which is best assessed by item and test analysis. This paper tries to assess items and test quality in order to explore the relationship between difficulty index (p-value) and discrimination indices (DI) with distractor efficiency (DE). The study was conducted among 134 second year Yemeni EFL students in Sanaa University, Yemen. Twenty MCQs, after checking their reliability and validity, were analysed for p-value, DI and DE. Results indicated that the mean score was 9.49 with S.D 2.82. Internal consistency reliability of the test as per KR20 was 0.7. Mean p-value and DI were 61.92
Keywords: Difficulty Index; Discrimination Index; Distractor Efficiency; Item Analysis; Multiple Choice Questions; Non-functional Distractor (NFD).
Student Academic Performance System (SAPS): Quantitative Approaches to Evaluating and Monitoring Student Progress
by Samantha Robinson, Joon Jin Song
Abstract: Monitoring student performance throughout the course of an academic semester can have a positive impact for students and educators alike in terms of motivating invaluable course redesign, effective student intervention, and practical methodologies for classroom enrichment. Student Academic Performance Systems (SAPS), analytical tools to track student progress, can enhance both learning and academic development. However, these monitoring systems need to be effective, easy to implement, clear to interpret, and based on a framework flexible enough to easily adjust and suit any educational level or course style. We propose a SAPS system designed to monitor student performance, demonstrate that the SAPS system is an efficient and complementary tool for educators, and argue that some form of a SAPS system should be incorporated into every classroom at all instructional levels.
Keywords: Educational Strategies; Student Performance Monitoring; Student Evaluation; Statistical Surveillance; Statistical Classification Techniques; Hybrid Classification; Logistic Regression; Classification and Regression Trees; CART Methods.
Leveraging Psychometric Isomorphism in Assessment Development
by Katie Kunze, Roy Levy, Vandhana Mehta
Abstract: Two studies were conducted to examine ways in which isomorph item families can aid in the creation of exam forms and the assessment of student learning. Methods for selecting isomorph item families for specific uses are described. Study 1 examined the use of isomorphs on high-stakes final exam forms. Study 2 explored using isomorphs for lower-stakes comparisons between pretests and posttests. Results of this work highlight the benefits of using isomorph item families and provide implications for both operational assessments in the Cisco Networking Academy Program, where this work takes place, and for the assessment community at large.
Keywords: isomorphs; assessment development; test assembly.
Academic Performance Analysis to Support Proactive Student Advising for an Electrical Engineering Program
by Richelle Adams, Cathy Ann Radix
Abstract: Using correlation, regression and hierarchical clustering methods, the authors examined three consecutive graduating cohorts of students in an Electrical and Computer Engineering undergraduate programme to determine which courses (or groups of courses) were the best predictors of graduation GPA. The aim was to develop predictive models that support a consistent proactive advising experience. The main impact of this study is the methodology which can be applied to other programmes with similar weighted GPA schemes and with limited data sources. Other impacts were: the model identified which types of courses impacted GPA performance most, bringing clarity as to where cohort-wide intervention may be required; and the model can help us identify earlier at-risk and exceptional students.
Keywords: Proactive Advising; Student Performance; Prediction; Engineering Curriculum.
Estimation of Nonlinear Structural Equation Models with Dichotomous Indicator Variables: A Monte Carlo Comparison of Methods
by Holmes Finch
Abstract: Nonlinear structural equation models (SEMs), which include interactions among latent predictors, as well as quadratic or higher order terms, have been the focus of research over the last three decades, beginning with Kenny and Judd (1984). The great majority of that work has focused on the case where the indicator variables are continuous in nature. However, in practice many nonlinear SEMs will involve the use of responses to items on scales, which are categorical. The focus of the current simulation study was on comparing several methods for modeling nonlinear SEMs when indicator variables were dichotomous. Results of the study showed that a Bayesian approach, as well as a method based on 2-stage least squares, provided the most accurate parameter estimates, the highest power, and the best control over the Type I error rate for the interaction effect. Implications of these findings for practice are discussed.
Keywords: Structural equation model; interaction; Bayesian estimation; 2-stage least squares.
Binomial Logistic Modeling for Aggregate Binary Data: Application to Preschoolers' Alphabet Knowledge
by Seongah Im, Barbara DeBaryshe
Abstract: This study investigated the use of different binomial logistic models as alternatives to the normal model when analysing non-normal aggregate outcomes that are sums of correlated binary responses. The outcome variables provided in the two illustrative examples were preschoolers uppercase and lowercase letter naming knowledge with different shapes of non-normal distributions. The binomial, beta-binomial, and mixed binomial models with logit links were examined and compared to each other and to the normal linear model. Results were consistent in both examples. Among the models compared, the beta-binomial and mixed binomial models with overdispersion parameters captured interdependence among correlated binary responses. In addition, the mixed binomial model further explained remaining overdispersion and best fitted the data. Implications including advocating for the use of the binomial models with overdispersion parameters for clustered data were further discussed.
Keywords: Correlated Binary Responses; Non-Normal; Aggregate Data; Overdispersion; Beta-Binomial; Mixed Binomial; Test Scores; Alphabet Knowledge.