View in EDS

The Impact of the Failure to Account for KS2 Cohort Effects on Awarding at GCSE. Research Report

Saved in:
Bibliographic Details
Title: The Impact of the Failure to Account for KS2 Cohort Effects on Awarding at GCSE. Research Report
Language: English
Authors: Tim Gill, Cambridge University Press and Assessment (United Kingdom)
Source: Cambridge University Press & Assessment. 2025.
Availability: Cambridge University Press & Assessment. Shaftesbury Road Cambridge CB2 8EA. Tel: 44-1223-553311; e-mail: directcs@cambridge.org; Web site: https://www.cambridgeassessment.org.uk/
Peer Reviewed: N
Page Count: 35
Publication Date: 2025
Document Type: Reports - Research
Education Level: Secondary Education
Descriptors: Foreign Countries, Exit Examinations, Academic Ability, Individual Differences, Cohort Analysis, Prediction, Grades (Scholastic), Outcomes of Education, Matrices, Intellectual Disciplines, Regression (Statistics), Scores, Accuracy, Secondary School Students, Models, English Literature, Mathematics, Biology, Chemistry, History, Religion Studies, Music
Geographic Terms: United Kingdom (England)
Abstract: What is this report about? This report investigates a limitation of the current GCSE awarding process in England, which relies on a 'comparable outcomes' approach using Key Stage 2 (KS2) prior attainment data. Specifically, it examines how failing to account for cohort effects -- where students perform differently depending on the average ability of their peers -- can lead to systematic under- or over-prediction of outcomes for certain awarding organisations (AOs). The research explores whether incorporating centre-level KS2 data into prediction models can improve fairness and accuracy in grade awarding. What did we do? We analysed GCSE outcomes in seven subjects (English literature, maths, biology, chemistry, history, religious studies, and music) using data from the National Pupil Database for 2023 and 2024. For each subject, we compared three methods for generating predicted grade distributions: 1. Prediction matrices based on current awarding procedures. 2. Logistic regression excluding centre-level KS2 scores, using only candidate-level data. 3. Logistic regression including centre-level KS2 scores, accounting for both individual and cohort effects. Candidates from independent/selective schools and those not in Year 11 were excluded, to mirror current awarding practices. We identified AO specifications with candidates who had high or low average KS2 attainment and assessed how predictions differed across methods. What did we find? Across all subjects, the logistic regression models including centre-level KS2 scores consistently produced more accurate predictions for specifications with high-attaining cohorts. These regression models revealed significant cohort effects: students in higher-attaining centres tended to outperform predictions based solely on individual KS2 scores. The differences in predicted outcomes were most pronounced at grades 7 and 4, with some specifications seeing shifts of up to 3 percentage points. In contrast, predictions from the logistic regression model excluding centre-level data were nearly identical to those from the prediction matrix method, indicating that the observed improvements stemmed from accounting for cohort effects rather than the statistical method itself. What are the implications? The findings suggest that the current GCSE awarding process may systematically disadvantage specifications with high-attaining cohorts and advantage those with lower-attaining cohorts. This misalignment could result in grade distributions that are either too harsh or too lenient. Incorporating centre-level KS2 data into prediction models would lead to fairer and more consistent outcomes across specifications. As GCSE awarding in 2026 will again use a "common centres" approach, it is crucial to investigate and correct these discrepancies to ensure equitable standards. We recommend considering the use of logistic regression methods in future awarding processes and reviewing past awards for potential misalignment.
Abstractor: As Provided
Entry Date: 2026
Accession Number: ED678111
Database: ERIC
Description
Abstract:What is this report about? This report investigates a limitation of the current GCSE awarding process in England, which relies on a 'comparable outcomes' approach using Key Stage 2 (KS2) prior attainment data. Specifically, it examines how failing to account for cohort effects -- where students perform differently depending on the average ability of their peers -- can lead to systematic under- or over-prediction of outcomes for certain awarding organisations (AOs). The research explores whether incorporating centre-level KS2 data into prediction models can improve fairness and accuracy in grade awarding. What did we do? We analysed GCSE outcomes in seven subjects (English literature, maths, biology, chemistry, history, religious studies, and music) using data from the National Pupil Database for 2023 and 2024. For each subject, we compared three methods for generating predicted grade distributions: 1. Prediction matrices based on current awarding procedures. 2. Logistic regression excluding centre-level KS2 scores, using only candidate-level data. 3. Logistic regression including centre-level KS2 scores, accounting for both individual and cohort effects. Candidates from independent/selective schools and those not in Year 11 were excluded, to mirror current awarding practices. We identified AO specifications with candidates who had high or low average KS2 attainment and assessed how predictions differed across methods. What did we find? Across all subjects, the logistic regression models including centre-level KS2 scores consistently produced more accurate predictions for specifications with high-attaining cohorts. These regression models revealed significant cohort effects: students in higher-attaining centres tended to outperform predictions based solely on individual KS2 scores. The differences in predicted outcomes were most pronounced at grades 7 and 4, with some specifications seeing shifts of up to 3 percentage points. In contrast, predictions from the logistic regression model excluding centre-level data were nearly identical to those from the prediction matrix method, indicating that the observed improvements stemmed from accounting for cohort effects rather than the statistical method itself. What are the implications? The findings suggest that the current GCSE awarding process may systematically disadvantage specifications with high-attaining cohorts and advantage those with lower-attaining cohorts. This misalignment could result in grade distributions that are either too harsh or too lenient. Incorporating centre-level KS2 data into prediction models would lead to fairer and more consistent outcomes across specifications. As GCSE awarding in 2026 will again use a "common centres" approach, it is crucial to investigate and correct these discrepancies to ensure equitable standards. We recommend considering the use of logistic regression methods in future awarding processes and reviewing past awards for potential misalignment.