View in EDS

Do Test Scores Misrepresent Test Results? An Item-by-Item Analysis. EdWorkingPaper No. 25-1343

Saved in:
Bibliographic Details
Title: Do Test Scores Misrepresent Test Results? An Item-by-Item Analysis. EdWorkingPaper No. 25-1343
Language: English
Authors: Jesse Bruhn, Michael Gilraine, Jens Ludwig, Sendhil Mullainathan, Annenberg Institute for School Reform at Brown University
Source: Annenberg Institute for School Reform at Brown University. 2025.
Availability: Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/
Peer Reviewed: N
Page Count: 85
Publication Date: 2025
Document Type: Reports - Research
Numerical/Quantitative Data
Education Level: Elementary Secondary Education
Descriptors: Scores, Test Results, Achievement Tests, Test Items, Elementary Secondary Education, Academic Achievement, Teacher Effectiveness
Geographic Terms: Texas
Assessment and Survey Identifiers: State of Texas Assessments of Academic Readiness (STAAR)
Abstract: Much of the data collected in education is effectively thrown away. Students answer individual test questions, but administrators and researchers only see aggregate performance. All the item-level data are lost. Ex ante it is not clear this destroys much useful information, since the aggregate might be a sufficient statistic. Using data from Texas for 5 million students and 1.31 billion student-item responses, we show that in fact aggregation does destroy a great deal of valuable information in education: (1) Even conditional on a summary test measure, there is additional information in the item-level data; (2) This additional information is relevant for the student outcomes that education decisions seek to optimize; and (3) This information can be made practically useful for schools. Given how inexpensive storing, transmitting and analyzing such data would be, large gains could be had in education by simply using all the data we currently collect. [This report was funded by the Student Upward Mobility Initiative, Manny Roman, and the Center for Applied AI at the University of Chicago Booth School of Business.]
Abstractor: As Provided
Entry Date: 2026
Accession Number: ED678317
Database: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED678317
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED678317
AccessLevel: 3
PubType: Report
PubTypeId: report
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Do Test Scores Misrepresent Test Results? An Item-by-Item Analysis. EdWorkingPaper No. 25-1343
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Jesse+Bruhn%22">Jesse Bruhn</searchLink><br /><searchLink fieldCode="AR" term="%22Michael+Gilraine%22">Michael Gilraine</searchLink><br /><searchLink fieldCode="AR" term="%22Jens+Ludwig%22">Jens Ludwig</searchLink><br /><searchLink fieldCode="AR" term="%22Sendhil+Mullainathan%22">Sendhil Mullainathan</searchLink><br /><searchLink fieldCode="AR" term="%22Annenberg+Institute+for+School+Reform+at+Brown+University%22">Annenberg Institute for School Reform at Brown University</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Annenberg+Institute+for+School+Reform+at+Brown+University%22"><i>Annenberg Institute for School Reform at Brown University</i></searchLink>. 2025.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: N
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 85
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Reports - Research<br />Numerical/Quantitative Data
– Name: Audience
  Label: Education Level
  Group: Audnce
  Data: <searchLink fieldCode="EL" term="%22Elementary+Secondary+Education%22">Elementary Secondary Education</searchLink>
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Scores%22">Scores</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Results%22">Test Results</searchLink><br /><searchLink fieldCode="DE" term="%22Achievement+Tests%22">Achievement Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Items%22">Test Items</searchLink><br /><searchLink fieldCode="DE" term="%22Elementary+Secondary+Education%22">Elementary Secondary Education</searchLink><br /><searchLink fieldCode="DE" term="%22Academic+Achievement%22">Academic Achievement</searchLink><br /><searchLink fieldCode="DE" term="%22Teacher+Effectiveness%22">Teacher Effectiveness</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Texas%22">Texas</searchLink>
– Name: SubjectThesaurus
  Label: Assessment and Survey Identifiers
  Group: Su
  Data: <searchLink fieldCode="SU" term="%22State+of+Texas+Assessments+of+Academic+Readiness+%28STAAR%29%22">State of Texas Assessments of Academic Readiness (STAAR)</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Much of the data collected in education is effectively thrown away. Students answer individual test questions, but administrators and researchers only see aggregate performance. All the item-level data are lost. Ex ante it is not clear this destroys much useful information, since the aggregate might be a sufficient statistic. Using data from Texas for 5 million students and 1.31 billion student-item responses, we show that in fact aggregation does destroy a great deal of valuable information in education: (1) Even conditional on a summary test measure, there is additional information in the item-level data; (2) This additional information is relevant for the student outcomes that education decisions seek to optimize; and (3) This information can be made practically useful for schools. Given how inexpensive storing, transmitting and analyzing such data would be, large gains could be had in education by simply using all the data we currently collect. [This report was funded by the Student Upward Mobility Initiative, Manny Roman, and the Center for Applied AI at the University of Chicago Booth School of Business.]
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED678317
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED678317
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 85
    Subjects:
      – SubjectFull: Scores
        Type: general
      – SubjectFull: Test Results
        Type: general
      – SubjectFull: Achievement Tests
        Type: general
      – SubjectFull: Test Items
        Type: general
      – SubjectFull: Elementary Secondary Education
        Type: general
      – SubjectFull: Academic Achievement
        Type: general
      – SubjectFull: Teacher Effectiveness
        Type: general
      – SubjectFull: Texas
        Type: general
      – SubjectFull: State of Texas Assessments of Academic Readiness (STAAR)
        Type: general
    Titles:
      – TitleFull: Do Test Scores Misrepresent Test Results? An Item-by-Item Analysis. EdWorkingPaper No. 25-1343
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Annenberg Institute for School Reform at Brown University
      – PersonEntity:
          Name:
            NameFull: Jesse Bruhn
      – PersonEntity:
          Name:
            NameFull: Michael Gilraine
      – PersonEntity:
          Name:
            NameFull: Jens Ludwig
      – PersonEntity:
          Name:
            NameFull: Sendhil Mullainathan
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 11
              Type: published
              Y: 2025
          Titles:
            – TitleFull: Annenberg Institute for School Reform at Brown University
              Type: main
ResultId 1