International Journal of Quantitative Research in Education
These articles have been peer-reviewed and accepted for publication but are pending final changes, are not yet published and may not appear here in their final order of publication until they are assigned to issues. Therefore, the content conforms to our standards but the presentation (e.g. typesetting and proof-reading) is not necessarily up to the Inderscience standard. Additionally, titles, authors, abstracts and keywords may change before publication. Articles will not be published until the final proofs are validated by their authors.
Forthcoming articles must be purchased for the purposes of research, teaching and private study only. These articles can be cited using the expression "in press". For example: Smith, J. (in press). Article Title. Journal Title.
Articles marked with this shopping trolley icon are available for purchase - click on the icon to send an email request to purchase.
International Journal of Quantitative Research in Education (9 papers in press)
Leveraging Psychometric Isomorphism in Assessment Development by Katie Kunze, Roy Levy, Vandhana Mehta Abstract: Two studies were conducted to examine ways in which isomorph item families can aid in the creation of exam forms and the assessment of student learning. Methods for selecting isomorph item families for specific uses are described. Study 1 examined the use of isomorphs on high-stakes final exam forms. Study 2 explored using isomorphs for lower-stakes comparisons between pretests and posttests. Results of this work highlight the benefits of using isomorph item families and provide implications for both operational assessments in the Cisco Networking Academy Program, where this work takes place, and for the assessment community at large. Keywords: isomorphs; assessment development; test assembly.
Academic Performance Analysis to Support Proactive Student Advising for an Electrical Engineering Program by Richelle Adams, Cathy Ann Radix Abstract: Using correlation, regression and hierarchical clustering methods, the authors examined three consecutive graduating cohorts of students in an Electrical and Computer Engineering undergraduate programme to determine which courses (or groups of courses) were the best predictors of graduation GPA. The aim was to develop predictive models that support a consistent proactive advising experience. The main impact of this study is the methodology which can be applied to other programmes with similar weighted GPA schemes and with limited data sources. Other impacts were: the model identified which types of courses impacted GPA performance most, bringing clarity as to where cohort-wide intervention may be required; and the model can help us identify earlier at-risk and exceptional students. Keywords: Proactive Advising; Student Performance; Prediction; Engineering Curriculum.
Estimation of Nonlinear Structural Equation Models with Dichotomous Indicator Variables: A Monte Carlo Comparison of Methods by Holmes Finch Abstract: Nonlinear structural equation models (SEMs), which include interactions among latent predictors, as well as quadratic or higher order terms, have been the focus of research over the last three decades, beginning with Kenny and Judd (1984). The great majority of that work has focused on the case where the indicator variables are continuous in nature. However, in practice many nonlinear SEMs will involve the use of responses to items on scales, which are categorical. The focus of the current simulation study was on comparing several methods for modeling nonlinear SEMs when indicator variables were dichotomous. Results of the study showed that a Bayesian approach, as well as a method based on 2-stage least squares, provided the most accurate parameter estimates, the highest power, and the best control over the Type I error rate for the interaction effect. Implications of these findings for practice are discussed. Keywords: Structural equation model; interaction; Bayesian estimation; 2-stage least squares.
Binomial Logistic Modeling for Aggregate Binary Data: Application to Preschoolers' Alphabet Knowledge by Seongah Im, Barbara DeBaryshe Abstract: This study investigated the use of different binomial logistic models as alternatives to the normal model when analysing non-normal aggregate outcomes that are sums of correlated binary responses. The outcome variables provided in the two illustrative examples were preschoolers uppercase and lowercase letter naming knowledge with different shapes of non-normal distributions. The binomial, beta-binomial, and mixed binomial models with logit links were examined and compared to each other and to the normal linear model. Results were consistent in both examples. Among the models compared, the beta-binomial and mixed binomial models with overdispersion parameters captured interdependence among correlated binary responses. In addition, the mixed binomial model further explained remaining overdispersion and best fitted the data. Implications including advocating for the use of the binomial models with overdispersion parameters for clustered data were further discussed. Keywords: Correlated Binary Responses; Non-Normal; Aggregate Data; Overdispersion; Beta-Binomial; Mixed Binomial; Test Scores; Alphabet Knowledge.
State Expenditures to Public Higher Education and Regional Funding Norms: A Panel Data Analysis by Gabriel Serna, Joshua M. Cohen Abstract: Using well understood regional indicators we seek to understand if and how region influences state expenditures to public higher education. We employ an econometric technique that allows for estimation of parameter estimates on time-invariant regressors (Census Divisions) from 199596 through 20072008. Additionally, we are able to provide solid evidence that region matters while also including other well-known drivers of state expenditures to public higher education. We conclude that these relationships are often overlooked in higher education economics research, for better or for worse, and hence warrant further investigation since implications exist for future and prospective policy. Keywords: higher education expenditures; higher education policy; higher education economics; higher education finance; panel data analysis.
Mapworks survey for student retention: Who declines to respond? by Youqin Pan Abstract: Surveys of students are among the primary data sources for research in higher education. Student surveys suffer from increasing level of nonresponse. This paper investigated the demographic factors that have contributed to nonresponses in Mapworks student survey. The study utilized t-test and logistic regression to analyze the data collected from undergraduate students. The findings showed that high school GPA, gender, campus residency, race/ethnicity were significant predictors of survey nonresponse, and there were significant differences between respondents and non-respondents. Implications for academic administrators were discussed. Keywords: response rates; nonresponse bias; Mapworks survey; on-campus residence; retention.
Special Issue on: Assessment in the Era of Big Data Examining the Potential of Process Data and Paradata in Evaluating Educational Assessments
Hints, Multiple Attempts, and Learning Outcomes in a Computer-based Formative Assessment System by Jinnie Choi, Miko?aj Bogucki Abstract: In formative assessment, providing feedback, such as hints and multiple opportunities to answer a question, are important features that support learning. Increasingly in computer-based systems that support formative assessment, the controls are given to instructors over the settings that encourage or discourage students hint opening behavior or multiple attempts. Different settings may affect how students (a) access hints differently and (b) make multiple attempts to improve answers or give up. Ultimately, these settings may encourage effortful behaviors and eventually, learning. This study analyzed empirical data and examined under which settings students (a) persist with multiple attempts to reach correct answers; (b) conscientiously submit all problems assigned to them; and (c) perform better in terms of getting a correct answer on their first try. Evidence showed that the settings that encourage more hint opening and multiple attempts were associated with higher persistence and conscientiousness, but with lower performance. Keywords: formative assessment; technology integration in assessment; learning outcomes; process data.
Screening for aberrant school performances in high-stakes assessments using influential analysis by Andrés Christiansen Abstract: A method is proposed to screen for aberrant school performances in large-scale, high-stakes assessments using influential analysis under a Bayesian approach. Proportions of low and high achievers within a school were modeled via the beta inflated mean regression model (Bayes & Valdivieso, 2016) using school performances in previous years as predictors. The general measure of phi-divergence proposed by Peng and Dey (1995) was used to determine aberrancy. A simulation study revealed that the method could recover previously distorted school performances as aberrant. The proposed technique was applied to a Peruvian national reading assessment in Grade 4th of primary education for which the government provided a school performance incentive bonus. Keywords: influential analysis; aberrant school performance; high-stakes assessments,
beta inflated mean regression.
Using Cluster Analysis to Explore Students Interactions With Automated Feedback in an Online Earth Science Task by Mengxiao Zhu, Ou Lydia Liu, Hee-Sun Lee Abstract: Digitally delivered tasks can provide students opportunities to interact with disciplinary content while their interactions with the tasks can be recorded as time-stamped log events in the system server. Through post-hoc analysis of log data, we can reenact and discover patterns in students activities. This study addresses the online earth science module where students engaged in writing and revising scientific arguments in a structured format. We adopted natural language processing (NLP) techniques to analyze students responses, which enabled us to provide immediate feedback to students on their responses and revisions. Cluster analyses were conducted on the action sequences in four argumentation tasks embedded in the module. For each task, the cluster analyses identified two clusters of students who showed different revision patterns with allocation of time on different items. In addition, students in those two clusters also differed in their initial item scores and item score changes after revision. Keywords: log data analysis; automated scoring and feedback; cluster analysis; scientific argumentation.