Do Test Scores Misrepresent Test Results? An Item-by-Item Analysis. EdWorkingPaper No. 25-1343
Saved in:
| Title: | Do Test Scores Misrepresent Test Results? An Item-by-Item Analysis. EdWorkingPaper No. 25-1343 |
|---|---|
| Language: | English |
| Authors: | Jesse Bruhn, Michael Gilraine, Jens Ludwig, Sendhil Mullainathan, Annenberg Institute for School Reform at Brown University |
| Source: | Annenberg Institute for School Reform at Brown University. 2025. |
| Availability: | Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/ |
| Peer Reviewed: | N |
| Page Count: | 85 |
| Publication Date: | 2025 |
| Document Type: | Reports - Research Numerical/Quantitative Data |
| Education Level: | Elementary Secondary Education |
| Descriptors: | Scores, Test Results, Achievement Tests, Test Items, Elementary Secondary Education, Academic Achievement, Teacher Effectiveness |
| Geographic Terms: | Texas |
| Assessment and Survey Identifiers: | State of Texas Assessments of Academic Readiness (STAAR) |
| Abstract: | Much of the data collected in education is effectively thrown away. Students answer individual test questions, but administrators and researchers only see aggregate performance. All the item-level data are lost. Ex ante it is not clear this destroys much useful information, since the aggregate might be a sufficient statistic. Using data from Texas for 5 million students and 1.31 billion student-item responses, we show that in fact aggregation does destroy a great deal of valuable information in education: (1) Even conditional on a summary test measure, there is additional information in the item-level data; (2) This additional information is relevant for the student outcomes that education decisions seek to optimize; and (3) This information can be made practically useful for schools. Given how inexpensive storing, transmitting and analyzing such data would be, large gains could be had in education by simply using all the data we currently collect. [This report was funded by the Student Upward Mobility Initiative, Manny Roman, and the Center for Applied AI at the University of Chicago Booth School of Business.] |
| Abstractor: | As Provided |
| Entry Date: | 2026 |
| Accession Number: | ED678317 |
| Database: | ERIC |
| Abstract: | Much of the data collected in education is effectively thrown away. Students answer individual test questions, but administrators and researchers only see aggregate performance. All the item-level data are lost. Ex ante it is not clear this destroys much useful information, since the aggregate might be a sufficient statistic. Using data from Texas for 5 million students and 1.31 billion student-item responses, we show that in fact aggregation does destroy a great deal of valuable information in education: (1) Even conditional on a summary test measure, there is additional information in the item-level data; (2) This additional information is relevant for the student outcomes that education decisions seek to optimize; and (3) This information can be made practically useful for schools. Given how inexpensive storing, transmitting and analyzing such data would be, large gains could be had in education by simply using all the data we currently collect. [This report was funded by the Student Upward Mobility Initiative, Manny Roman, and the Center for Applied AI at the University of Chicago Booth School of Business.] |
|---|
