Statistically Guided Grading Judgements: Contextualisation or Contamination?
Saved in:
| Title: | Statistically Guided Grading Judgements: Contextualisation or Contamination? |
|---|---|
| Language: | English |
| Authors: | Louise Badham (ORCID |
| Source: | Oxford Review of Education. 2025 51(1):17-35. |
| Availability: | Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals |
| Peer Reviewed: | Y |
| Page Count: | 19 |
| Publication Date: | 2025 |
| Document Type: | Journal Articles Reports - Research |
| Descriptors: | Advanced Placement Programs, Grading, Interrater Reliability, Evaluative Thinking, Evaluation Criteria, Statistical Data, Teacher Attitudes, Error of Measurement, Scoring, Scoring Rubrics, Context Effect, Bias |
| DOI: | 10.1080/03054985.2023.2290640 |
| ISSN: | 0305-4985 1465-3915 |
| Abstract: | Different sources of assessment evidence are reviewed during International Baccalaureate (IB) grade awarding to convert marks into grades and ensure fair results for students. Qualitative and quantitative evidence are analysed to determine grade boundaries, with statistical evidence weighed against examiner judgement and teachers' feedback on examinations. A trial was conducted to explore how examiners' grading decisions were influenced by having access to statistical evidence. Grade awards were replicated in nine exams across five subjects, with examiners accessing all available evidence in one model, and only scripts and grade descriptors in the other. Preliminary findings suggest that both approaches lead to broadly comparable grading outcomes. Focus group feedback indicates that examiners consider judging the grade-worthiness of student work to be their primary role in grade award. Whilst they found item-level data helpful for prioritising questions for review, participants reported that access to evidence such as statistically recommended boundaries can cloud their judgement or encourage strategic grading. This study also raises further questions about the purposes and uses of different forms of statistical evidence, as well as how and when they should be integrated with qualitative evidence in grade awarding. |
| Abstractor: | As Provided |
| Entry Date: | 2025 |
| Accession Number: | EJ1458475 |
| Database: | ERIC |
|
Full text is not displayed to guests.
Login for full access.
|
|
| Abstract: | Different sources of assessment evidence are reviewed during International Baccalaureate (IB) grade awarding to convert marks into grades and ensure fair results for students. Qualitative and quantitative evidence are analysed to determine grade boundaries, with statistical evidence weighed against examiner judgement and teachers' feedback on examinations. A trial was conducted to explore how examiners' grading decisions were influenced by having access to statistical evidence. Grade awards were replicated in nine exams across five subjects, with examiners accessing all available evidence in one model, and only scripts and grade descriptors in the other. Preliminary findings suggest that both approaches lead to broadly comparable grading outcomes. Focus group feedback indicates that examiners consider judging the grade-worthiness of student work to be their primary role in grade award. Whilst they found item-level data helpful for prioritising questions for review, participants reported that access to evidence such as statistically recommended boundaries can cloud their judgement or encourage strategic grading. This study also raises further questions about the purposes and uses of different forms of statistical evidence, as well as how and when they should be integrated with qualitative evidence in grade awarding. |
|---|---|
| ISSN: | 0305-4985 1465-3915 |
| DOI: | 10.1080/03054985.2023.2290640 |
