Changing public understanding of league tables

Our improved school performance measure is reframing school accountability policy debates.

The challenge

The Progress 8 performance measure, used to inform England’s school league tables, is biased in favour of schools with socially and educationally advantaged pupils, and therefore punishes and rewards the wrong schools. Progress 8 needs to adjust for pupil socioeconomic background to provide fairer and more meaningful insight into the impact that schools have on learning.

Research impact - Informing debate around school accountability measures

In this research, we critique the Government’s current Progress 8 secondary school league tables for ignoring differences in pupil sociodemographic characteristics when holding schools to account for their GCSE results. We discuss the policy context which saw a move away from the contextualised measures of school performance used by the Department for Education (DfE) between 2006-2010, and explore the arguments for and against adjusting for pupil background, arguing that the case for adjusting is far more compelling.

We have discussed these findings with the Office for Students (OfS) with reference to their work on measuring University “learning gains”, and in March 2019, we had a rich and varied discussion with Department for Education (DfE) policy advisors regarding the future direction of school performance measures in England. The University of Bristol press release of our work led to national radio, newspaper and online coverage, increasing public awareness and understanding of the issue. George Leckie gave radio interviews at TalkSport and Heart FM, and was invited to appear on BBC Radio 4’s World At One. Newspaper coverage appeared in The Guardian, The Independent, The Times, and the Times Educational Supplement (TES).

Following this media attention, we discussed our work at a private meeting with the Education Select Committee, and with teachers at the National Education Union. We have since had follow-on discussions with Emma Hardy, Labour MP and Education Select Committee member, regarding alternatives to Progress 8 and data-driven school accountability following Labour’s recent policy to scrap Key Stage 2 tests.

Our work prompted the Co-op Multi-Academy Trust to issue their own press release, leading to articles in the TES and Schools Week calling for Progress 8 to be adjusted, citing our work and encouraging other schools to contact us. We have since corresponded with many head-teachers who have requested their Progress 8 scores adjusted for pupil background according to our findings, and asked to be involved in future research. The Director of the Co-op MAT also discussed our work in a meeting with MP Angela Rayner (then Shadow Education Secretary) and 25 Labour MPs.

We have also collaborated extensively with the Northern Powerhouse Partnership (NPP) who have championed our work at a second Education Select Committee meeting on “Education in the North” and in a meeting with Shadow cabinet members on education and skills. Partnering with the NPP, we have published Adjusted Progress 8 scores for all schools in England in a report and on a dedicated interactive website, as a way of encouraging further debate around concerns with Progress 8 and data-driven accountability, and to place more pressure on Government to engage on these issues. Education Select Committee MP Lucy Powell has described this as “a ground-breaking piece of work by the Northern Powerhouse Partnership with Bristol University”; while the subsequent NPP press release led to further media coverage by the BBC, The Independent, the Financial Times, and the TES.

This impact of this research continues to build as we are currently in ongoing discussions with a national newspaper with a view to publication of schools’ 2019 Adjusted Progress 8 scores. It has also informed our advisory work with government education departments in both Sweden and Trinidad & Tobago.

Underpinning research

In this research, we reanalyse the Government’s own data to show that schools’ Progress 8 scores change substantially when the true nature of their cohorts is fully acknowledged. We show that the national league table rankings would change drastically if pupil background were taken into account, as was the case in the ‘contextual value-added’ progress measures which were previously used by the DfE.

The main driver of these results is that Progress 8 penalises schools teaching greater proportions of white British pupils and those entitled to free school meals (FSM). We recommend adjusting Progress 8 for pupil background and better explaining its importance when comparing schools. More broadly, we argue for less emphasis on all school performance measures, as a failure to adjust for pupil background is just one of many statistical and more general concerns with the high-stakes testing and league tables regime in England.

Our research has been informed from the start by knowledge exchange meetings with multiple stakeholders as part of our ongoing ESRC grant. Feedback from Ofsted and Fischer Family Trust (an organisation which provides school performance monitoring tools to schools) has also informed and improved our work.

Key facts

The Government’s current Progress 8 measure of school progress ignores differences in school intake, while earlier measures used by the DfE between 2006-2010 took pupil background into account.
Adjusting Progress 8 for pupil background would lead to the national league table rankings of over one-fifth of schools changing by over 500 places.
Adjusting Progress 8 for pupil background would lead to 40% of schools judged ‘underperforming’ to achieve a higher banding than under the current system.