Improving the Design of Evaluations That Include Students, Teachers, and Schools: An Empirical Investigation of Key Design Parameters
Saved in:
| Title: | Improving the Design of Evaluations That Include Students, Teachers, and Schools: An Empirical Investigation of Key Design Parameters |
|---|---|
| Language: | English |
| Authors: | Dea Mulolli (ORCID |
| Source: | AERA Open. 2025 11(1). |
| Availability: | SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: journals@sagepub.com; Web site: http://sagepub.com |
| Peer Reviewed: | Y |
| Page Count: | 20 |
| Publication Date: | 2025 |
| Sponsoring Agency: | National Science Foundation (NSF) |
| Contract Number: | 2000388 |
| Document Type: | Journal Articles Reports - Research |
| Education Level: | Early Childhood Education Elementary Education Grade 3 Primary Education Grade 4 Intermediate Grades Grade 5 Middle Schools Grade 6 Grade 7 Junior High Schools Secondary Education Grade 8 |
| Descriptors: | Grade 3, Grade 4, Grade 5, Grade 6, Grade 7, Grade 8, Educational Environment, Demography, Reading Achievement, Mathematics Achievement, Science Achievement, Statistical Analysis, Evaluation Methods, Environmental Influences, Material Development, Randomized Controlled Trials, Students, Teachers, Schools |
| Geographic Terms: | North Carolina, Kentucky, Michigan, Maryland |
| ISSN: | 2332-8584 |
| Abstract: | The learning environment is a complex environment for conducting evaluations. Students, teachers, and schools all play a role in shaping the environment, which can lead to differences in student achievement. Hence designing an evaluation with the capacity to detect the effectiveness of an educational intervention on student achievement requires careful planning. A key part of the planning is a statistical power analysis that considers the multilevel nature of the design. This study provides empirical estimates of design parameters necessary for planning adequately powered cluster randomized trials that include the student, teacher, and school level and are focused on reading, mathematics, or science achievement. The sample in our study includes administrative state datasets from Michigan, North Carolina, Kentucky, and Maryland for grades 3 through 8. The results showed that, with few exceptions, the variance in student test scores is larger between teachers within schools than between schools. |
| Abstractor: | As Provided |
| Notes: | https://www.openicpsr.org/openicpsr/project/215781/version/V1/view |
| Entry Date: | 2026 |
| Accession Number: | EJ1494762 |
| Database: | ERIC |
| Abstract: | The learning environment is a complex environment for conducting evaluations. Students, teachers, and schools all play a role in shaping the environment, which can lead to differences in student achievement. Hence designing an evaluation with the capacity to detect the effectiveness of an educational intervention on student achievement requires careful planning. A key part of the planning is a statistical power analysis that considers the multilevel nature of the design. This study provides empirical estimates of design parameters necessary for planning adequately powered cluster randomized trials that include the student, teacher, and school level and are focused on reading, mathematics, or science achievement. The sample in our study includes administrative state datasets from Michigan, North Carolina, Kentucky, and Maryland for grades 3 through 8. The results showed that, with few exceptions, the variance in student test scores is larger between teachers within schools than between schools. |
|---|---|
| ISSN: | 2332-8584 |
