View in EDS

Do Test Scores Misrepresent Test Results? An Item-by-Item Analysis. Discussion Paper #2025.13

Saved in:
Bibliographic Details
Title: Do Test Scores Misrepresent Test Results? An Item-by-Item Analysis. Discussion Paper #2025.13
Language: English
Authors: Jesse Bruhn, Michael Gilraine, Jens Ludwig, Sendil Mullainathan, Massachusetts Institute of Technology (MIT), Blueprint Labs, National Bureau of Economic Research (NBER)
Source: Blueprint Labs. 2025.
Availability: Blueprint Labs. 30 Wadsworth Street, Cambridge, MA 02142. e-mail: contact@mitblueprintlabs.org; Web site: https://blueprintlabs.mit.edu/
Peer Reviewed: N
Page Count: 85
Publication Date: 2025
Document Type: Reports - Research
Descriptors: Testing, Tests, Scores, Test Results, Response Style (Tests), Data Use, Testing Problems, Item Analysis, Item Response Theory, Data, Data Interpretation
Geographic Terms: Texas
Abstract: Much of the data collected in education is effectively thrown away. Students answer individual test questions, but administrators and researchers only see aggregate performance. All the item-level data are lost. Ex ante it is not clear this destroys much useful information, since the aggregate might be a sufficient statistic. Using data from Texas for 5 million students and 1.31 billion student-item responses, the researchers show that in fact aggregation does destroy a great deal of valuable information in education: (1) Even conditional on a summary test measure, there is additional information in the item-level data; (2) This additional information is relevant for the student outcomes that education decisions seek to optimize; and (3) This information can be made practically useful for schools.
Abstractor: As Provided
Entry Date: 2026
Access URL: https://blueprintlabs.mit.edu/research/do-test-scores-misrepresent-test-results-an-item-by-item-analysis/
Accession Number: ED678493
Database: ERIC
FullText Text:
  Availability: 0
Header DbId: eric
DbLabel: ERIC
An: ED678493
AccessLevel: 3
PubType: Report
PubTypeId: report
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Do Test Scores Misrepresent Test Results? An Item-by-Item Analysis. Discussion Paper #2025.13
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Jesse+Bruhn%22">Jesse Bruhn</searchLink><br /><searchLink fieldCode="AR" term="%22Michael+Gilraine%22">Michael Gilraine</searchLink><br /><searchLink fieldCode="AR" term="%22Jens+Ludwig%22">Jens Ludwig</searchLink><br /><searchLink fieldCode="AR" term="%22Sendil+Mullainathan%22">Sendil Mullainathan</searchLink><br /><searchLink fieldCode="AR" term="%22Massachusetts+Institute+of+Technology+%28MIT%29%2C+Blueprint+Labs%22">Massachusetts Institute of Technology (MIT), Blueprint Labs</searchLink><br /><searchLink fieldCode="AR" term="%22National+Bureau+of+Economic+Research+%28NBER%29%22">National Bureau of Economic Research (NBER)</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Blueprint+Labs%22"><i>Blueprint Labs</i></searchLink>. 2025.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Blueprint Labs. 30 Wadsworth Street, Cambridge, MA 02142. e-mail: contact@mitblueprintlabs.org; Web site: https://blueprintlabs.mit.edu/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: N
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 85
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2025
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Testing%22">Testing</searchLink><br /><searchLink fieldCode="DE" term="%22Tests%22">Tests</searchLink><br /><searchLink fieldCode="DE" term="%22Scores%22">Scores</searchLink><br /><searchLink fieldCode="DE" term="%22Test+Results%22">Test Results</searchLink><br /><searchLink fieldCode="DE" term="%22Response+Style+%28Tests%29%22">Response Style (Tests)</searchLink><br /><searchLink fieldCode="DE" term="%22Data+Use%22">Data Use</searchLink><br /><searchLink fieldCode="DE" term="%22Testing+Problems%22">Testing Problems</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Analysis%22">Item Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Item+Response+Theory%22">Item Response Theory</searchLink><br /><searchLink fieldCode="DE" term="%22Data%22">Data</searchLink><br /><searchLink fieldCode="DE" term="%22Data+Interpretation%22">Data Interpretation</searchLink>
– Name: Subject
  Label: Geographic Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Texas%22">Texas</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Much of the data collected in education is effectively thrown away. Students answer individual test questions, but administrators and researchers only see aggregate performance. All the item-level data are lost. Ex ante it is not clear this destroys much useful information, since the aggregate might be a sufficient statistic. Using data from Texas for 5 million students and 1.31 billion student-item responses, the researchers show that in fact aggregation does destroy a great deal of valuable information in education: (1) Even conditional on a summary test measure, there is additional information in the item-level data; (2) This additional information is relevant for the student outcomes that education decisions seek to optimize; and (3) This information can be made practically useful for schools.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 2026
– Name: URL
  Label: Access URL
  Group: URL
  Data: <link linkTarget="URL" linkTerm="https://blueprintlabs.mit.edu/research/do-test-scores-misrepresent-test-results-an-item-by-item-analysis/" linkWindow="_blank">https://blueprintlabs.mit.edu/research/do-test-scores-misrepresent-test-results-an-item-by-item-analysis/</link>
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED678493
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED678493
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 85
    Subjects:
      – SubjectFull: Testing
        Type: general
      – SubjectFull: Tests
        Type: general
      – SubjectFull: Scores
        Type: general
      – SubjectFull: Test Results
        Type: general
      – SubjectFull: Response Style (Tests)
        Type: general
      – SubjectFull: Data Use
        Type: general
      – SubjectFull: Testing Problems
        Type: general
      – SubjectFull: Item Analysis
        Type: general
      – SubjectFull: Item Response Theory
        Type: general
      – SubjectFull: Data
        Type: general
      – SubjectFull: Data Interpretation
        Type: general
      – SubjectFull: Texas
        Type: general
    Titles:
      – TitleFull: Do Test Scores Misrepresent Test Results? An Item-by-Item Analysis. Discussion Paper #2025.13
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Massachusetts Institute of Technology (MIT), Blueprint Labs
      – PersonEntity:
          Name:
            NameFull: National Bureau of Economic Research (NBER)
      – PersonEntity:
          Name:
            NameFull: Jesse Bruhn
      – PersonEntity:
          Name:
            NameFull: Michael Gilraine
      – PersonEntity:
          Name:
            NameFull: Jens Ludwig
      – PersonEntity:
          Name:
            NameFull: Sendil Mullainathan
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 11
              Type: published
              Y: 2025
          Titles:
            – TitleFull: Blueprint Labs
              Type: main
ResultId 1