Making the Grade: Accounting for Course Selection in High School Transcripts with Item Response Theory. EdWorkingPaper No. 24-1109
Saved in:
| Title: | Making the Grade: Accounting for Course Selection in High School Transcripts with Item Response Theory. EdWorkingPaper No. 24-1109 |
|---|---|
| Language: | English |
| Authors: | Kenneth A. Shores, Sanford R. Student, Annenberg Institute for School Reform at Brown University |
| Source: | Annenberg Institute for School Reform at Brown University. 2024. |
| Availability: | Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/ |
| Peer Reviewed: | N |
| Page Count: | 68 |
| Publication Date: | 2024 |
| Sponsoring Agency: | Institute of Education Sciences (ED) |
| Contract Number: | R305S210008 |
| Document Type: | Reports - Research |
| Education Level: | High Schools Secondary Education Higher Education Postsecondary Education |
| Descriptors: | High School Seniors, Academic Records, Course Selection (Students), Grades (Scholastic), Difficulty Level, Grade Point Average, College Entrance Examinations, Test Bias, Admission Criteria, Open Enrollment, Grade Inflation |
| Geographic Terms: | Delaware |
| Assessment and Survey Identifiers: | SAT (College Admission Test) |
| Abstract: | We use student-level administrative data from Delaware for 43,767 high school students across five 12th grade cohorts from 2017 to 2021. We apply Item Response Theory (IRT) to high school transcript data, treating courses as items and grades as ordered responses, to estimate both student transcript strength ([theta]) and course difficulty. We prove, via construct and predictive validation and simulation, that [theta] improves upon GPA because it accounts for ability selection into courses with variable difficulty. Compared to the SAT, [theta] shows smaller racial/ethnic gaps but substantially larger gender gaps that indicate boys underperform in their courses relative to their standardized test scores. We conclude by discussing significant methodological--such as grade-inflation and cross-school heterogeneity in course offerings--and practical challenges that remain before such measures could be considered for high-stakes applications. |
| Abstractor: | As Provided |
| IES Funded: | Yes |
| Entry Date: | 2025 |
| Accession Number: | ED671123 |
| Database: | ERIC |
| Abstract: | We use student-level administrative data from Delaware for 43,767 high school students across five 12th grade cohorts from 2017 to 2021. We apply Item Response Theory (IRT) to high school transcript data, treating courses as items and grades as ordered responses, to estimate both student transcript strength ([theta]) and course difficulty. We prove, via construct and predictive validation and simulation, that [theta] improves upon GPA because it accounts for ability selection into courses with variable difficulty. Compared to the SAT, [theta] shows smaller racial/ethnic gaps but substantially larger gender gaps that indicate boys underperform in their courses relative to their standardized test scores. We conclude by discussing significant methodological--such as grade-inflation and cross-school heterogeneity in course offerings--and practical challenges that remain before such measures could be considered for high-stakes applications. |
|---|
