Labour party literacy task group report - a critique
This report from the task group chaired by Professor Michael Barber, 'A Reading Revolution', was published amid much publicity on 27 February 1997. It proposed a 10 year programme to raise standards of literacy, making specific suggestions about the amount of time that should be devoted to it in schools and how schools should work towards achieving targets, as well as recommendations for government action.
This critique concentrates on two aspects of the report. The first concerns the report's espousal of the publication of school test results and teacher assessments: the second concerns its aim of raising the literacy standard of every child to 'level 4' by the year 2006.
Publishing school results at Key Stage 2
The following quotes are taken from the report:
PARA 15. 'Teacher assessment results should also be made publicly available along with the results from the tests and used by schools....'
PARA 18. 'It is important that information about primary school performance is published and reported to parents.'
PARA 19. 'we believe that each primary school needs to be given data which enable it to compare its performance in both reading and English as a whole to national performance, to other local schools and to schools with comparable intakes.'
PARA 27 'In other words, the best schools in this group are already achieving our target while some comparable schools are falling far below it'.
Given the quite inadequate criterion for comparability (adjusting for uptake of free school meals) and the commitments in paras 15,18 and 19 this sets the stage for what are effectively league tables. While the authors have gone to some trouble to some trouble to expunge any mention of league tables from the report: the problem is that they appear not to have not realised that the proposals are precisely supportive of such league tables. Furthermore, no menti on at all of the problem that precise comparisons among schools are very difficult since the sampling variation is so large: this has been demonstrated over and over again (see Goldstein and Speigelhalter, 1996 for a review). In other words the report fails to take note of the research evidence from school effectiveness research about the problems of school comparisons based on the publication of test scores. In its enthusiasm for setting 'targets' it fails to recognise the severe dangers posed by crude, uncontextualised, comparisons with no indication of their limitations.
The second issue concerns the desire to compare literacy achievements against a fixed standard, in this case level 4 of the National Curriculum. The report demonstrates its naiveté here by supposing that this really is possible. Yet all the research from the examination boards, from the Assessment of Performance Unit and from the US National Assessment of Educational Progress, is that it is virtually impossible to maintain such an absolute standard. The tests change,language use changes and it is a matter for judgment whether what is deemed 'level 4' in 1998 is the same as that deemed 'level 4' in 2006. The real danger of setting such an unrealistic target is that it then becomes a political slogan, devoid of useful meanings and possibly distorting the teaching and learning process.
This report illustrates a disquieting recent trend in educational research and policy; namely the unwillingness to pay close attention to evidence, or the distortion of evidence in pursuit of particular policy goals (see web document by Mortimore and Goldstein, for a detailed discussion of this). Those goals are not, in the long term, well served by ignoring evidence, however unwelcome.
Reference
Goldstein, H. and Spiegelhalter, D. J. (1996). League tables and their limitations: statistical issues in comparisons of institutional performance. Journal of the Royal Statistical Society, A. 159: 385-443.
Harvey Goldstein, March 1997