The 2020 GCSE and A-level 'exam grades fiasco': A secondary data analysis of students' grades and Ofqual's algorithm
ESRC Secondary Data Analysis Initiative grant
- Grant Code: ES/W000555/1
- Duration: 01/10/2021 - 30/09/2023
- Amount Awarded: £299,000
People
- George Leckie, Principal Investigator
- Lucy Prior, Research Associate
Summary
The awarding of the 2020 GCSE and A-Level exam grades in England was widely viewed as a ‘fiasco’. When COVID-19 forced the cancellation of exams, DfE and Ofqual asked centres (schools and colleges) to submit Centre Assessment Grades (CAGs) and rankings. Namely, the grades and rank orders within their centres that teachers thought students would have achieved had they sat their exams. Ofqual, tasked with preventing grade inflation and ensuring grading consistency , viewed students’ CAGs as overly optimistic and so replaced them with calculated grades predicted via their Direct Centre-level Performance (DCP) algorithm. The result was that 40% of CAGs were downgraded by one or more grades. There was immediate public outcry that students were ‘robbed’ of the grades they deserved. The media quickly reported that the calculated grades were systematically biased against various students and schools. Others argued that they were not reliable enough, with predictive accuracy especially low in smaller centres. The furore resulted in a government U-turn, Prime Minister Boris Johnson declaring Ofqual’s DCP approach a ‘mutant algorithm’, and Education Secretary Gavin Williamson instructing Ofqual to revert to the original CAGs. In January 2021, the government announced that the 2021 exams will also be cancelled with CAGs used in their place.
Students are accepted into universities and employment based on their GCSE and A-level grades. Their grades directly impact their immediate future. It is therefore vitally important for society to understand the extent to which students’ grades were unfairly awarded in 2020 and 2021 with biases potentially varying across individual centres and by student and school characteristics. It is also crucial to learn from the fiasco to help inform DfE and Ofqual responses when CAGs might again be needed in place of exam grades (e.g., due to future pandemics, teacher strikes, exam boycotts, leaked exam papers, centre malpractice, technology failures with onscreen assessments). More generally, our findings will be relevant to those calling for a reintroduction of coursework and other non-exam assessments at GCSE and A-level and especially those calling for a removal of exams altogether, since this would imply a permanent reliance on school and college assessments.
Our overarching aim is to therefore conduct an independent and rigorous secondary data analysis of the 2020 and 2021 GCSE and A-level exam grades to explore not just what went wrong statistically, but to identify what could be improved statistically when predicting grades in future years.
Videos
Journal articles
- Prior, L. and Leckie, G. (2023). School differences on whether and where students apply to university. British Journal of Sociology of Education.
Manuscripts in preparation for peer reviewed journals
- MAIHDA for quantiative intersectionality: A review and new extensions with application to student achievement
Manuscripts under review at peer reviewed journals
- Leckie, G., & Maragkou, K. (2023). Student Sociodemographic and School Type Differences in Teacher-Predicted vs. Achieved Grades for University Admission.
- Prior, L., Evans, C., Merlo, J., Leckie, G. (2022). Sociodemographic inequalities in student achievement: an intersectional multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA) with application to students in London, England.
- Prior, L., & Leckie, G. Student intersectional sociodemographic and school variation in GCSE final grades in England following COVID-19 examination cancellations.
Working papers
- Leckie, G., & Maragkou, K. (2023). Student Sociodemographic and School Type Differences in Teacher-Predicted vs. Achieved Grades for University Admission. Bristol Working Papers in Education Series. Working Paper: 04/2023. Article. Under review, see above.
- Prior, L. and Leckie, G. (2023). School differences on whether and where students apply to university. Bristol Working Paper in Education Series. Working Paper: 01/2023. Article. Published, see above.
- Prior, L., & Leckie, G. (2023). An investigation of student intersectional sociodemographic and school variation in GCSE final grades in England in 2020. Bristol Working Paper in Education Series. Working Paper: 03/2023. Article. Under review, see above.
- Prior, L., Evans, C., Merlo, J., Leckie, G. (2022). Sociodemographic inequalities in student achievement: an intersectional multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA) with application to students in London, England. Bristol Working Papers in Education Series. Working Paper: 06/2022. Article. Under review, see above.
- Prior, L., Evans, C., Merlo, J., Leckie, G. (2022). Sociodemographic inequalities in student achievement: an intersectional multilevel analysis of individual heterogeneity and discriminatory accuracy (MAIHDA) with application to students in London, England. arXiv:2211.06321 [cs.CY]. Article. Under review, see above.
Conference sessions
- Special Session "Administrative data for education research: building bridges using the GRADE data sharing project" at Royal Economic Society 2023 Annual Conference, April 2023.
Conference presentations
- 'Student and school differences in UCAS predicted vs. achieved grades' at Royal Economic Society 2023 Annual Conference
- 'School differences in whether and where students apply to university' at The European Conference on Education 2023, July 2023.
Invited seminars
- Student and school differences in UCAS predicted vs. achieved grades, Edinburgh, June 2023
- Student and school differences in UCAS predicted vs. achieved grades, UCAS, June 2023
- Student and school differences in UCAS predicted vs. achieved grades, Education Policy Institute (EPI), May 2023
- School differences in whether and where students apply to university, Education Policy Institute (EPI), May 2023
Invited speakers hosted
- Dennis Sherwood 'The Great Grading Scandal', 18th October 2022. Bristol Conversations in Education.
Research discussions
- Discussions with UCAS Modelling and Resaerch team, September - January 2023
- Discussions with UCAS Modelling and Resaerch team, March - July 2023
- Meeting with James Zuccollo of Education Policy Institute (EPI) on school effects on longer term outcomes, February 2023.
- Discussion around research ideas with Dennis Sherwood, October 2022
- Discussion with Andrew Boyle Director of Quantitative Research at AlphaPlus and Analysis and Ben Smith Senior Statistician at AlphaPlus about 2020 CAGs, June 2022
- Discussion with Alex Jones Ofsted Director for Insights and Research, May 2022
- Discussion around research ideas with Professor Robin Shields, School of Education, University of Bristol, October 2021
GRADE events attended
- GRADE dataset roundtable: data-mine and discussion on the art of the possible, November 2021
- GRADE Applicant Webinar, September 2021
Other events attended related to the grant
- ADR UK Showcase for Ambassadors, London, 21 November 2022
- Longitudinal Education Outcomes (LEO) data: what next? London, 17 October 2022.
- Education Symposium: Administrative Data for Education Research: Building a sustainable future. Sheffield, 16 June 2022.
- Meeting with UKRI ESRC, MoJ, DfE and academics about ‘Building admin data communities’, Zoom, April 2022
Other activities related to the grant
- ADR UK Ambassador for increasing and improving the use of administrative data in academia for research to inform policy
- Submitted a PhD proposal for the ESRC linked administrative data studentship call, October 2022.