The Government’s Progress 8 school performance measure needs to account for pupil background
Progress 8 is the Department for Education’s (DfE) headline measure of the average academic progress pupils make in each school over secondary schooling. The measure adjusts pupils’ GCSE results for their end of primary schooling Key Stage 2 test results. The DfE and Ofsted both rely heavily on Progress 8 to hold schools to account.
About the research
The DfE argue that Progress 8 is a fair measure as it accounts for school intake attainment differences in pupils’ Key Stage 2 test scores. However, Progress 8 ignores school intake differences in all other pupil background characteristics, yet these also predict why some schools score higher at GCSE than others.
This research compared the DfE’s 2016 Progress 8 measure with an ‘Adjusted Progress 8’ version which accounts for pupil age, gender, ethnicity, English as an additional language, special education needs, free school meal status, and residential deprivation.
The results show that schools’ Progress 8 scores, differences in average scores between regions and different school types all change dramatically once adjustments are made for pupil background. This leads to very different interpretations and conclusions about education in England.
Policy implications
• The many well-known statistical issues with all attempts to measure school performance, not to mention more general concerns with perverse incentives and gaming behaviours introduced by high-stakes testing, suggest the DfE and Ofsted should place far less
emphasis on Progress 8 when holding schools to account.
• Given the importance of pupil background in driving schools’ scores, the Government should revise their current school league tables to include an adjusted Progress 8 measure side-by-side with Progress 8 to present a more informative picture of school performance.
• In this case, the DfE should provide users with greater insight as to why schools achieve the scores they do, accompanied with a more detailed explanation as to the limitations of using such scores for school accountability.
• Adjusting for pupil background would see the national league table rankings of over one-fifth of schools change by over 500 places.
• Adjusting for pupil background would lead 40% of schools judged ‘underperforming’ under Progress 8 to move up out of this banding.
Key findings
The DfE’s decision to ignore pupil background when comparing schools is in stark contrast to both the academic literature and practitioner commentaries, both of which argue that such adjustments should be made when holding schools to account.
• The high average Progress 8 score seen in London more than halves when we adjust for pupil background. This is principally due to these schools teaching high proportions of high progress ethnic groups. In contrast, the low average Progress 8 score seen in the North East increases substantially after adjustment due to the high proportions of poor pupils taught in this region.
• Other dramatic changes are seen for grammar schools and faith schools whose high average Progress 8 scores reduce substantially once the educationally advantaged nature of their pupils is considered. In contrast, the low average pupil progress seen in sponsored academies increases once the disadvantaged nature of their pupils is recognised.
• Progress 8 effectively punishes schools teaching high proportions of disadvantaged pupils for the national underperformance of these groups.
• Progress 8 can, therefore, be argued to give too much emphasis to schools rather than Government or society, as primarily responsible for the national underperformance of these groups. In contrast, adjusted versions of Progress 8 can be argued to stress more that society and Government rather than schools are primarily responsible.
Policy Briefing 66: January 2019
Progress 8 school performance measure needs to account for pupil background (PDF, 93kB)
Further information
A video recording of a presentation of this work is available. The slides are also available, as is a working paper version of this work.
Funding
This work was funded by an ESRC Standard grant (ES/R010285/1). Further details can be found on the grant website.
Contact the researchers
Dr George Leckie, Reader in Social Statistics, School of Education, University of Bristol: g.leckie@bristol.ac.uk
Professor Harvey Goldstein, Professor of Social Statistics, School of Education, University of Bristol: h.goldstein@bristol.ac.uk
Authors
Dr George Leckie and Professor Harvey Goldstein, University of Bristol
Related briefings
Transnationalizing Modern Languages: Reframing language education
Measures of school performance are biased by genetic differences between students
