Value added for primary schools
Value added – a commentary on the KS1-KS2, KS2-KS3 league tables December 2003 and KS3-KS4 January 2004.
Following last year's value added secondary results the government has this year released value added data for KS1 – KS2 results along with the raw, unadjusted, KS2 results and also data for KS2-KS3 and KS3-KS4.
According to schools minister David Milliband (December 4 2003) speaking about the KS1-KS2 results;
"We have always said that we will listen to the views of heads, teachers and parents about how the performance tables can provide a more comprehensive and rounded picture of school performance. Including value added information does just that. It shows the rates of progress that children make between 7 and 11 in different schools".
Unfortunately, this year's tables are no advance on the inadequate information that was produced before; Milliband may have listened but he doesn't seem to have learnt much.
As I have pointed out before there is no emphasis on uncertainty intervals that would allow proper comparisons between schools, and this is a more serious problem for primaries which tend to be smaller than secondaries. Mention of such intervals is confined to a technical note and even this gives the minimal information so that only an experienced statistician would be able to apply the information to schools generally. Thus, for example, they quote an interval for schools with 30 or 50 pupils in the calculation but not for a school with just 20 where the interval will be approximately 98.9 to 101.1, etc. It would be a simple matter to have given the interval in the value added table itself, but this helpful bit of information is not provided. Furthermore, we are nowhere told what the intervals are for the raw results – a key piece of information that has never been provided in these tables, and which it seems impossible to calculate from the published data.
Moreover, even this information is absent for the KS2-KS3 and KS3-KS4 results. Instead we are provided by the DfES with a ranking of the 'top 100 schools' or top 5% with no indication about uncertainty (confidence intervals) at all.
I have also drawn attention to the positive correlation between the value added and raw scores as probably due to a misspecification of the statistical procedure being used, but no response to this has appeared. There is also no recognition that there can be any differential effectiveness, as is found routinely in those LEAs which produce their own, carefully constructed and presented value added systems, such as Hampshire. Nor is there any recognition of the increasing evidence about the perverse consequences that so often result from these league tables. In the case of the KS2-KS3 results this model misspecification leads to Grammar schools tending to do well on both raw and value added tables; a point that has been given extensive publicity by sections of the media. For the KS3-KS4 an analysis carried out by the DfES, likewise shows the average value added for selective schools to be higher than that for comprehensives.
In short, despite proclaiming that they are really trying hard to provide the best possible information, the evidence is that this is little more than the kind of Government spin that we are now very used to. Scotland, Wales and Northern Ireland have realised how counterproductive league tables of all kinds are, and have called a halt to them. England is now the only place in the UK that insists on promoting them and one wonders how long it will take before the DfES too is forced to drop them.
Harvey Goldstein, January 15 2004