International Journal of Quantitative Research in Education (6 papers in press)
Psychometric examination of the Academic Motivation Scale using a Vietnamese university student sample
by Lan Luong, Arthur Poropat, Helen Klieve, Kate Thompson
Abstract: Massification of higher education in Vietnam has brought about both achievements and various issues. Efforts have been made to improve the quality of teaching and learning in higher education, but very little attention has been paid to the issue of student motivation. This study aims to contribute to this area of knowledge by testing the applicability of the Academic Motivation Scale (AMS) in assessing Vietnamese university students motivation. This was achieved via evaluation of the scales psychometric properties using data obtained from 648 first year students from a high-ranking university. Using a different approach in model testing, results indicated that the revised seven factor AMS with 23 items best fitted the data. All subscales had satisfactory reliabilities. Thus, the revised AMS can be used to study Vietnamese university students motivation.
Keywords: academic motivation; Academic Motivation Scale; Vietnam; university; self-determination theory.
A Comparison of Linear and Equipercentile Equating and IRT Equating with FIPC Across Multidimensional Test Forms for Nonequivalent Groups
by Ki Cole, Sohee Kim, Mwarumba Mwavita
Abstract: The aim of this study is to compare linear equating procedures (i.e., Levine and Tucker methods), equipercentile equating, and the fixed item parameter calibration (FIPC) with true and observed score equating methods when applied non-equivalent groups taking forms which have various multidimensional structures: equal or unequal total test difficulty and similar or dissimilar difficulty within dimensions across forms. This situation may be common for large-scale test forms that are composed of multiple sub-content areas and are being administered to multiple samples at different times. Within the specifications of this study, when forms differ in total difficulty but do not have confounded difficulty within dimensions, the linear methods may be preferred over the equipercentile and FIPC methods. When forms differ in average item difficulty within dimensions, regardless of equal or unequal total test difficulty, the FIPC method is favored over the equipercentile method, when data were correlated.
Keywords: Equating; Psychometrics; Test construction.
The Analysis of MCQs in a Newly Developed Reading Comprehension Ability Test: A Study on Yemeni undergraduate EFL learners
by Iftikhar Al-Ariqi, Jagannath Dange, Mohsin Mir
Abstract: Studies have shown that the best way to test the students ability in reading comprehension is the Multiple-choice questions (MCQs) for its validity and reliability. The efficiency of MCQs as an efficient tool for evaluation solely rests upon their quality which is best assessed by item and test analysis. This paper tries to assess items and test quality in order to explore the relationship between difficulty index (p-value) and discrimination indices (DI) with distractor efficiency (DE). The study was conducted among 134 second year Yemeni EFL students in Sanaa University, Yemen. Twenty MCQs, after checking their reliability and validity, were analysed for p-value, DI and DE. Results indicated that the mean score was 9.49 with S.D 2.82. Internal consistency reliability of the test as per KR20 was 0.7. Mean p-value and DI were 61.92
Keywords: Difficulty Index; Discrimination Index; Distractor Efficiency; Item Analysis; Multiple Choice Questions; Non-functional Distractor (NFD).
Student Academic Performance System (SAPS): Quantitative Approaches to Evaluating and Monitoring Student Progress
by Samantha Robinson, Joon Jin Song
Abstract: Monitoring student performance throughout the course of an academic semester can have a positive impact for students and educators alike in terms of motivating invaluable course redesign, effective student intervention, and practical methodologies for classroom enrichment. Student Academic Performance Systems (SAPS), analytical tools to track student progress, can enhance both learning and academic development. However, these monitoring systems need to be effective, easy to implement, clear to interpret, and based on a framework flexible enough to easily adjust and suit any educational level or course style. We propose a SAPS system designed to monitor student performance, demonstrate that the SAPS system is an efficient and complementary tool for educators, and argue that some form of a SAPS system should be incorporated into every classroom at all instructional levels.
Keywords: Educational Strategies; Student Performance Monitoring; Student Evaluation; Statistical Surveillance; Statistical Classification Techniques; Hybrid Classification; Logistic Regression; Classification and Regression Trees; CART Methods.
Leveraging Psychometric Isomorphism in Assessment Development
by Katie Kunze, Roy Levy, Vandhana Mehta
Abstract: Two studies were conducted to examine ways in which isomorph item families can aid in the creation of exam forms and the assessment of student learning. Methods for selecting isomorph item families for specific uses are described. Study 1 examined the use of isomorphs on high-stakes final exam forms. Study 2 explored using isomorphs for lower-stakes comparisons between pretests and posttests. Results of this work highlight the benefits of using isomorph item families and provide implications for both operational assessments in the Cisco Networking Academy Program, where this work takes place, and for the assessment community at large.
Keywords: isomorphs; assessment development; test assembly.
Academic Performance Analysis to Support Proactive Student Advising for an Electrical Engineering Program
by Richelle Adams, Cathy Ann Radix
Abstract: Using correlation, regression and hierarchical clustering methods, the authors examined three consecutive graduating cohorts of students in an Electrical and Computer Engineering undergraduate programme to determine which courses (or groups of courses) were the best predictors of graduation GPA. The aim was to develop predictive models that support a consistent proactive advising experience. The main impact of this study is the methodology which can be applied to other programmes with similar weighted GPA schemes and with limited data sources. Other impacts were: the model identified which types of courses impacted GPA performance most, bringing clarity as to where cohort-wide intervention may be required; and the model can help us identify earlier at-risk and exceptional students.
Keywords: Proactive Advising; Student Performance; Prediction; Engineering Curriculum.