Air Reading: A Randomized Evaluation of a Virtual Tutoring Model for K-6 Grade Students across Two Cohorts
Saved in:
| Title: | Air Reading: A Randomized Evaluation of a Virtual Tutoring Model for K-6 Grade Students across Two Cohorts |
|---|---|
| Language: | English |
| Authors: | Nathan Storey, Amanda J. Neitzel, Society for Research on Educational Effectiveness (SREE) |
| Source: | Society for Research on Educational Effectiveness. 2025. |
| Availability: | Society for Research on Educational Effectiveness. 2040 Sheridan Road, Evanston, IL 60208. Tel: 202-495-0920; e-mail: contact@sree.org; Web site: https://www.sree.org/ |
| Peer Reviewed: | Y |
| Publication Date: | 2025 |
| Document Type: | Reports - Research |
| Education Level: | Elementary Education |
| Descriptors: | Elementary School Students, Intelligent Tutoring Systems, Reading Skills, Reading Achievement, Reading Difficulties, Reading Instruction, Teaching Methods, Gender Differences, Racial Differences, Ethnicity, English Learners, Socioeconomic Status, Program Implementation, Fidelity, Reading Tests, National Competency Tests |
| Assessment and Survey Identifiers: | National Assessment of Educational Progress |
| Abstract: | Background: Theoretical Framework--A child's reading skills play an enormous role in their development as learners and as members of society. Students reading below grade level as they enter middle school are likely to fall further behind academically each year (Lewkowicz, 2000; Lyon, et al., 2001). The 2024 NAEP reading assessment found that fourth graders were scoring four points lower than when the NAEP started in 1992 (The Nation's Report Card, 2024). Students considered to be struggling or striving readers are often from minoritized groups (Biancarosa & Snow, 2006), creating a gap in learning opportunities. Tutoring programs have repeatedly proven to be a powerful tool for supporting student learning and closing achievement gaps, particularly for students from low-SES backgrounds (Baye et al., 2019; Dietrichson et al., 2017; Neitzel et al., 2021). With limited funds, it is essential to develop evidence of effective programs. This study aims to provide rigorous research on one such tutoring initiative, Air Reading. Programmatic Focus: Air Reading is an assessment-driven virtual tutoring program designed to improve students' foundational reading skills in kindergarten through eighth grade. Groups of up to four students are paired with a consistent, trained tutor in 40-minute sessions four times a week. Comprehensive, one-on-one diagnostics identify students' learning needs and inform group placement, and bi-weekly assessments track student progress. Qualified, paid tutors deliver explicit, skill-based instruction using a systematic reading curriculum. Ongoing training and support are provided to tutors all year. Sessions are tracked and monitored on an electronic platform to maintain consistent standards. Purpose: This study used a randomized controlled trial with two cohorts of students to examine--1. What are the one-year and two-year (follow-up) effects of Air Reading on reading achievement for students reading below grade level in comparison to similar students receiving business-as-usual teaching? 2. How do the effects of Air Reading Tutoring differ by gender, ethnicity, English-learner status, grade level, economic status, special education status, and baseline reading achievement? 3. To what extent is fidelity of implementation associated with better student outcomes? Method: The first cohort of this study took place in a small, rural school district in the central U.S. During the 2023-24 school year, six elementary schools in the district took part in the study to assess reading outcomes for students in grades 1-6. The second cohort took place in a larger, more urban school district in the southern U.S. During the 2024-25 school year, five schools in the district took part in a new cohort of randomization for students in grades 1-4. At each participating school, students needing reading intervention were identified, and based on fall NWEA MAP scores, were flagged for Tier 2 or Tier 3 support. Air Reading organized the students by available tutoring slots, then randomly assigned them to treatment (Air Reading) or control conditions using the "randomizr" package (Coppock, 2023). Stratified by scheduling slots, students were randomly assigned within the strata to their groups. The first cohort was randomized in January 2024, and the second cohort was randomized in fall 2024. Demographics for the first cohort's analytic sample are in Table 1. Measures: Data sources and measures for the current study included student achievement data from the Northwest Evaluation Association's Measures of Academic Progress (MAP) end-of-year reading scores and Air Reading usage data, both provided by Air Reading. Regarding usage, in cohort 1, we analyzed one three-category variable (low, medium, and high), based on Air Reading's recommended dosage guidelines: 48+ sessions=full dosage (high); 37-47 sessions=acceptable dosage (medium); and 0-36 sessions=not enough dosage (low). Data Collection: At the conclusion of tutoring, data on student achievement, student demographics, and program dosage was merged and deidentified by Air Reading and shared with the research team for analysis. Analytic Plan: To confirm randomization effectiveness, we assessed baseline equivalence using student reading achievement on the NWEA MAP (Table 2). Hierarchical linear modeling (HLM) compared achievement between Air Reading and control students, with students nested within schools. We controlled for pretest scores and adjusted for remaining group differences. We also tested for differential effects of Air Reading based on student demographics by adding interaction terms between the treatment indicator and covariates. For the dosage analysis, we replaced the treatment indicator with a categorical measure of treatment dosage (high/medium/low). All covariates were grand mean centered to facilitate interpretation of the intercept. All analyses were conducted using R statistical software (R Core Team, 2024) and the "lmerTest" package (Kuznetsova et al., 2017). Findings: In the first cohort, Air Reading students (M = 174.41) outperformed control students (M = 172.40) on the NWEA MAP, with a significant effect size of +0.12 (p = 0.05) (Table 3). No differential impacts were found across student subgroups (See Table 4). Analysis of Air Reading program usage revealed that treatment students attended an average of 39.8 tutoring sessions, with much usage variation, as attendance ranged from 0 to 63 sessions. However, no significant relationship was found between session attendance and reading outcomes (Table 5). Conclusions: The results from the first cohort indicate that Air Reading had a positive impact on student reading achievement, with an effect size of +0.12, suggesting an additional 1.12 months of learning for students performing below grade level. The program showed consistent effectiveness across demographic groups, highlighting its equity in supporting a diverse range of students. Additionally, while there was no significant link between the number of sessions attended and reading outcomes, students attending 48 or more sessions experienced the largest effect size (+0.21), underscoring the importance of consistent program usage. The second round of analysis will build on the findings from the first cohort by examining how students have progressed over two years of tutoring. In addition, it will examine the one-year impact of the second cohort in a new setting. These findings will provide insight into the long-term impact of the Air Reading program, as well as its effects after one year of participation in the first cohort. |
| Abstractor: | As Provided |
| Entry Date: | 2026 |
| Access URL: | https://www.sree.org/2025-conference |
| Accession Number: | ED677661 |
| Database: | ERIC |
Be the first to leave a comment!
