The 2020 GCSE and A-level 'exam grades fiasco': A secondary data analysis of students' grades and Ofqual's algorithm
ESRC Secondary Data Analysis Initiative grant
- Grant Code: ES/W000555/1
- Duration: 01/10/2021 - 30/09/2023
- Amount Awarded: £299,000
People
- George Leckie, Principal Investigator
- Lucy Prior, Co-Investigator
Summary
The awarding of the 2020 GCSE and A-Level exam grades in England was widely viewed as a ‘fiasco’. When COVID-19 forced the cancellation of exams, DfE and Ofqual asked centres (schools and colleges) to submit Centre Assessment Grades (CAGs) and rankings. Namely, the grades and rank orders within their centres that teachers thought students would have achieved had they sat their exams. Ofqual, tasked with preventing grade inflation and ensuring grading consistency , viewed students’ CAGs as overly optimistic and so replaced them with calculated grades predicted via their Direct Centre-level Performance (DCP) algorithm. The result was that 40% of CAGs were downgraded by one or more grades. There was immediate public outcry that students were ‘robbed’ of the grades they deserved. The media quickly reported that the calculated grades were systematically biased against various students and schools. Others argued that they were not reliable enough, with predictive accuracy especially low in smaller centres. The furore resulted in a government U-turn, Prime Minister Boris Johnson declaring Ofqual’s DCP approach a ‘mutant algorithm’, and Education Secretary Gavin Williamson instructing Ofqual to revert to the original CAGs. In January 2021, the government announced that the 2021 exams will also be cancelled with CAGs used in their place.
Students are accepted into universities and employment based on their GCSE and A-level grades. Their grades directly impact their immediate future. It is therefore vitally important for society to understand the extent to which students’ grades were unfairly awarded in 2020 and 2021 with biases potentially varying across individual centres and by student and school characteristics. It is also crucial to learn from the fiasco to help inform DfE and Ofqual responses when CAGs might again be needed in place of exam grades (e.g., due to future pandemics, teacher strikes, exam boycotts, leaked exam papers, centre malpractice, technology failures with onscreen assessments). More generally, our findings will be relevant to those calling for a reintroduction of coursework and other non-exam assessments at GCSE and A-level and especially those calling for a removal of exams altogether, since this would imply a permanent reliance on school and college assessments.
Our overarching aim is to therefore conduct an independent and rigorous secondary data analysis of the 2020 and 2021 GCSE and A-level exam grades to explore not just what went wrong statistically, but to identify what could be improved statistically when predicting grades in future years.
Videos
Journal articles
- Prior, L. and Leckie, G. (2023). School differences on whether and where students apply to university. British Journal of Sociology of Education. Article.
- Prior, L., Evans, C.R., Merlo, J., and Leckie, G., (2024). Sociodemographic inequalities in student achievement: An intersectional multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA). Sociology of Race and Ethnicity. Article.
- Prior, L. and Leckie, G. (2024). Student intersectional sociodemographic and school variation in GCSE final grades in England following Covid-19 examination cancellations. Oxford Review of Education, Article
- Leckie, G. and Maragkou, K. (2024). Student sociodemographic and school type differences in teacher-predicted vs. achieved grades for university admission. Higher Education. Article
Manuscripts in preparation for peer reviewed journals
- MAIHDA for quantitative intersectionality: A review and new extensions with application to student achievement
Working papers
- Leckie, G. and Maragkou, K., (2024). Tracing the origins of gender bias in teacher grades. Cambridge Working Papers in Economics. CWPE2457. Article.
- Leckie, G. and Maragkou, K. (2023). Student Sociodemographic and School Type Differences in Teacher-Predicted vs. Achieved Grades for University Admission. Bristol Working Papers in Education Series. Working Paper: 04/2023. Article. Published, see above.
- Prior, L. and Leckie, G. (2023). School differences on whether and where students apply to university. Bristol Working Paper in Education Series. Working Paper: 01/2023. Article. Published, see above.
- Prior, L., and Leckie, G. (2023). An investigation of student intersectional sociodemographic and school variation in GCSE final grades in England in 2020. Bristol Working Paper in Education Series. Working Paper: 03/2023. Article. Published, see above.
- Prior, L., Evans, C., Merlo, J., Leckie, G. (2022). Sociodemographic inequalities in student achievement: an intersectional multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA) with application to students in London, England. Bristol Working Papers in Education Series. Working Paper: 06/2022. Article. Published, see above.
- Prior, L., Evans, C., Merlo, J., Leckie, G. (2022). Sociodemographic inequalities in student achievement: an intersectional multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA) with application to students in London, England. arXiv:2211.06321 [cs.CY]. Article. Published, see above.
Conference sessions
- Special Session "Administrative data for education research: building bridges using the GRADE data sharing project" at Royal Economic Society 2023 Annual Conference, April 2023.
Conference presentations
- 'Student and school differences in UCAS predicted vs. achieved grades' at Royal Economic Society 2023 Annual Conference
- 'School differences in whether and where students apply to university' at The European Conference on Education 2023, July 2023.
- ADR UK Conference, Birmingham, November 2023.
- Workshop on administrative data for public policy research GS5, Faculty of Education, University of Cambridge, November 2023.
- ADR UK Event UWE: Research Development Workshop "Better Data. Better Research. Better Policy." University of the West of England, Bristol, May 2024.
- QRM Conference 2024, Gothenburg, June 2024.
- Higher Education in a Cold Climate, University of Bristol, Bristol, July 2024.
Invited seminars
- CREATE - Centre for Research on Equality in Education, Oslo, December 2024
- Victoria University, Melbourne (Zoom), October 2024
- Sheffield Methods Institute and School of Education, University of Sheffield, July 2024
- Student and school differences in UCAS predicted vs. achieved grades, Edinburgh, June 2023
- Student and school differences in UCAS predicted vs. achieved grades, UCAS, June 2023
- Student and school differences in UCAS predicted vs. achieved grades, Education Policy Institute (EPI), May 2023
- School differences in whether and where students apply to university, Education Policy Institute (EPI), May 2023
Invited speakers hosted
- Dennis Sherwood 'The Great Grading Scandal', 18th October 2022. Bristol Conversations in Education.
Research discussions
- Discussion with Tom Moseley about the 2020 GCSE and A-level exams fiasco, BBC Radio 4, You and Yours - October 2024
- Discussions with UCAS Modelling and Research team, September - January 2023
- Discussions with UCAS Modelling and Research team, March - July 2023
- Meeting with James Zuccollo of Education Policy Institute (EPI) on school effects on longer term outcomes, February 2023.
- Discussion around research ideas with Dennis Sherwood, October 2022
- Discussion with Andrew Boyle Director of Quantitative Research at AlphaPlus and Analysis and Ben Smith Senior Statistician at AlphaPlus about 2020 CAGs, June 2022
- Discussion with Alex Jones Ofsted Director for Insights and Research, May 2022
- Discussion around research ideas with Professor Robin Shields, School of Education, University of Bristol, October 2021
GRADE events attended
- GRADE dataset roundtable: data-mine and discussion on the art of the possible, November 2021
- GRADE Applicant Webinar, September 2021
Other events attended related to the grant
- ADR UK Showcase for Ambassadors, London, 21 November 2022
- Longitudinal Education Outcomes (LEO) data: what next? London, 17 October 2022.
- Education Symposium: Administrative Data for Education Research: Building a sustainable future. Sheffield, 16 June 2022.
- Meeting with UKRI ESRC, MoJ, DfE and academics about ‘Building admin data communities’, Zoom, April 2022
Other activities related to the grant
- ADR UK Ambassador for increasing and improving the use of administrative data in academia for research to inform policy
- Submitted a PhD proposal for the ESRC linked administrative data studentship call, October 2022.