School performance and extra-curricular provision
by Professor M. Barber et al., Institute of Education and Keele University (Report for DFEE, January 1997)
This report looks at a number of out-of-school activities. Based upon 14 schools it basically provides case study information about
The report received a great deal of publicity just before and on its release (14.1.97) because of what it purported to show about the effect of homework on pupil performance. Some of the political comment around this time, especially the Labour Party proposals for mandatory periods of homework for every pupil, were received with well-deserved derision, but the 'findings' of the survey appear to have gone unchallenged. In fact, the study turns out to be poorly designed and with little to add to the debater about the efficacy of homework.
The researchers chose 7 schools which OFSTED had rated as 'worthy of special recognition ' and 7 other schools not so rated but 'matched' for socio-economic factors. The report assumes that the OFSTED judgment can be equated with 'effectiveness', and because these schools had more homework, they conclude that giving more homework leads to 'success'.
Apart from the naiveté of making such an inference from cross-sectional and observational data, the report makes the quite erroneous assumption that OFSTED judgments can indeed be equated with effectiveness. As Mortimore and Goldstein (1996) point out, to judge effectiveness requires long term longitudinal data so that progress or 'value added estimates' can be made: a single inspection report cannot do this. It seems particularly unfortunate that this basic point has been misunderstood in this report.
A second problem with the report is that the number of schools studied is much too small to come to any secure conclusions, and did not even constitute a random sample from a meaningful population. At best the data provide interesting case study evidence, but generalizations are inadmissible.
All the results are presented in terms of percentages of children. What is required is information at the school level: for example are there just one or two schools with particular patterns?
Finally, this research, and especially the manner of its release reflects a worrying trend whereby poorly controlled, badly designed and often inadequately interpreted research studies are released with a blaze of publicity and political comment. As in the case of the OFSTED research criticised by Mortimore and Goldstein and the present study, little or no prior exposure of the research to peers via seminars, refereeing etc. takes place. This tends to undermine educational research generally and is an issue which, for example BERA should consider.
Harvey Goldstein, Institute of Education, 16.1.97