Paper Accepted to AERA 2021- Establishing Statistical Significance at Scale for Pattern-based Items (Stroup and Petrosino, 2021)

The American Educational Research Association (AERA), a national research society, strives to advance knowledge about education, to encourage scholarly inquiry related to education, and to promote the use of research to improve education and serve the public good. The Association was founded in 1916, is concerned with improving the educational process by encouraging scholarly inquiry related to education and evaluation and by promoting the dissemination and practical application of research results. This will be the 29th consecutive year I have presented research findings at the annual conference. Below is the acceptance letter received recently along with an abstract of the accepted paper. -Dr. Petrosino 

Dr. Anthony Petrosino,

We are pleased to inform you that your paper, "Establishing Statistical Significance at Scale for Pattern-based Items" submitted for consideration for the 2021 Virtual AERA Annual Meeting has been accepted. Congratulations on this accomplishment. AERA received more than 10,000 submissions this year. To ensure the highest quality papers presented at the Annual Meeting, your submission was reviewed by highly qualified reviewers serving on a review panel constituted by the Division D - Measurement and Research Methodology/Division D - Section 1: Educational Measurement, Psychometrics, and Assessment. Reviewers' comments are now available on the Online Annual Meeting Program Portal (All Academic) System for your use.

The AERA meeting offers a number of session formats to feature high quality research and scholarly work. Your paper has been placed in a Poster Session session titled, “Psychometrics and Educational Measurement: Poster Session 2”. Each format involves different modes of presentation. The program schedule will be available on January 15, 2021.

Establishing Statistical Significance at Scale for Pattern-based Items (accepted- 2021)

Abstract: Tests of statistical significance often play a decisive role in establishing the empirical warrant of evidence-based research in education. The results from pattern-based assessment items, as introduced in this paper, are categorical and multimodal and do not immediately support the use of measures of central tendency as typically related to interpretations of measures of statistical significance. Using results from the implementation of pattern-based items (PBIs) at scale with more than 400,000 students as part of the statewide Interim Assessment Program (IAP) in Texas, as well from a prior pilot implementations of identical items at schools in central Texas, this paper is meant to address how statistical significance can be established across scale – results from an individual classroom compared to statewide results – and across grade level – a repeat item used in subsequent grades. We illustrate the use of a Fisher’s Exact Test or a Pearson's Chi-squared Test to establish the level of statistical significance for differences in outcomes for pattern-based items. Then, subsequently, projections onto any axis of relative performance – e.g., assigning partial credit -- can be used for comparisons of achievement-related outcomes for groups of students.