Behind every quantitative research project, there's a statistician waiting to make sense of the data, and they could well be using software developed at Bristol's Centre for Multilevel Modelling to do it.

By Hilary Brown

This article first appeared in the University of Bristol's Nonesuch magazine, Autumn 2011

Statistics

The very word is apt to strike fear into the heart of any failed GCSE maths student. But there's no getting away from the blighters - they crop up in all aspects of everyday life, says Fiona Steele FBA, OBE, Professor of Social Statistics and Co-Director of the Centre for Multilevel Modelling (CMM). Quantitative data - information gathered to summarise the experiences of large groups of people, make comparisons between groups, and track changes among them over time - are used to inform all social and economic policies, from health and education to housing and work.

But analysing such data is a tricky business, because individual behaviour depends on a number of factors that interact in complex ways. Take Steele's work on housing demographics, where, as part of a project funded by the Economic and Social Research Council (ESRC) she is building a mathematical model to predict whether someone will move house in a given year. "People move for a number of reasons, depending on whether they are single or co-habiting, whether they have children, the ages of those children, and whether they are owner-occupiers or living in rented properties," she explains. "The challenge is how to represent that complexity in a statistical model."

Layer upon layer

As if that weren't complicated enough, for these models to be truly representative, you have to allow for the fact that people are acting within hierarchical structures - at the lowest level you've got the individual, at the second level are households, and at the top level are the areas where those households are clustered.

And that's where multilevel modelling comes in. "The attraction of this method in quantitative research is that it allows for the fact that people aren't operating independently, that their behaviour is influenced by other people and by social groupings," says Steele. In CMM, statisticians like Steele produce new multilevel methods for analysing these sorts of data structures, and develop software to apply these methods to research questions.

Blazing a trail

Among the different types of statistical analysis software used by the international research community, CMM's MLwiN is up there with the best, thanks in part to the initial work of Professor Harvey Goldstein and the late Professor Jon Rasbash.

These two pioneers of multilevel modelling brought the centre to Bristol six years ago from the Institute of Education in London. Since then, CMM researchers have further developed the software and made it more accessible through training workshops and online materials, so that it is now used by colleagues around the world. It is also routinely used by national bodies, such as the Higher Education Funding Council for England, the Department for Education and the Office for National Statistics. Chris Charlton, CMM's senior software engineer, and Professor Bill Browne (see below) continue to develop and maintain MLwiN, while also developing new software as part of a larger team supported by ESRC funding.

One of the major strengths of multilevel modelling is its versatility: it is used in education, medical science, demography, economics and many other areas. CMM researchers are drawn from the Graduate School of Education, the School of Geographical Sciences and the School of Veterinary Science, and collaborate with colleagues across the University. If you thought that school league tables and chicken welfare had nothing in common, read on.

The challenge is how to represent complexity in a statistical model

The education specialist Dr George Leckie (BSc 2003, PhD 2009), Lecturer in Social Statistics, researches various aspects of education and school effectiveness, including the quality of marking of England's national curriculum key stage educational tests, and social and ethnic segregation among schools and neighbourhoods. He is perhaps best known for his work with Professor Harvey Goldstein highlighting the limitations of England's school league tables.

School league tables are considered by government to be a vital tool in driving up standards and are eagerly scanned by parents hoping to find the best school for their child. But, says Leckie, they don't really give a reliable picture of how well a school is performing.

"In the 1990s, league tables were only based on the percentage of children getting five A* to C grades at GCSE, but that is an unfair measure of school quality, as schools differ hugely in the ability of their student intakes, with some schools starting off with much higher-achieving pupils than others," he explains. "You can't use the raw exam results as a measure of school quality, because you're not starting with a level playing field."

As a result, the government used multilevel models to introduce a 'value-added' system that takes account of the differing achievements of students entering the school, as well as adjusting for other 'contextual' factors such as eligibility for free school meals and lack of spoken English at home.

So far, so good - but still not perfect, particularly when it comes to school choice. Leckie and Goldstein have shown that there is great statistical imprecision in the league tables - meaning that you struggle to separate schools' performances from one another.

"This is because the numbers of children in any particular calculation in a given year (say, 200 pupils per school) are too small to make precise comparisons", says Leckie. "And if you're a parent, you're not interested in last year's exam results, on which the tables are based, but in those six years ahead when your child will be taking exams. So what you need to know is how well last year's results predict those in the future." And Leckie and Goldstein's models show that when you do those calculations, you can barely distinguish between schools: 'Statistical uncertainty is a fundamental aspect of communicating school performances, but is all too often ignored by the media and public.'

The biostatistician

Bill Browne is Professor of Biostatistics in the School of Veterinary Science and Co-Director of CMM. His research includes statistical methodology and software development in the fields of veterinary epidemiology and bird ecology. He has worked with Professor Christine Nicol in the Animal Welfare and Behaviour research group on a project (funded by the Biotechnology and Biological Sciences Research Council) on chicken welfare.

On the face of it, the Vet School seems an unlikely partner in CMM, but statistical modelling methods are as relevant in animal as in human populations. "Nested hierarchies in environments like schools - individual pupils, pupils within classes, classes within schools - can be found in, say, poultry operations, where you have individual chickens that are housed in groups within pens, which are clustered on farms," says Browne.

The chicken welfare models developed by Nicol and Browne draw on two types of data - welfare indicators and results from motivational priority experiments - and examine how these factors relate to each other.

Welfare indicators are measurable factors that scientists assume are associated with the animal's underlying welfare. For example the animal's body condition or the exhibition of specific behaviours may be indicative of good or bad welfare. To collect behaviour data researchers observe and record the frequency and duration of various behaviours - such as sitting, standing alert, feeding, feather-pecking and wing-stretching in the case of chickens - and they can then examine how they relate to other indicators.

Welfare indicators can be measured in different environments and the animal's preference between the environments can be established using motivational priority, or 'choice', experiments. "In the chicken example, this might involve housing a group of chickens in pens with, say, wire floors for six weeks, followed by a period in an enriched environment containing perches and nesting boxes," explains Browne. "You then allow the chickens to choose between the two environments."

As with the school leagues table example, it's essential to factor in statistical imprecision that stems from the fact that the chickens are housed together in groups and used repeatedly in a series of choice experiments. And this, says Browne, is one of things that multilevel models do best.

CMM vital statistics

• 25 years since Goldstein's seminal paper on multilevel modeling^*
• 18,000+ MLwiN users worldwide
• 2,200+ journal articles cite MLwiN
• 22 years old MLwiN software and its precursors
• 6,000+ users of CMM's online multilevel modelling course

*References

• Leckie, G. and Goldstein, H. (2011) 'Understanding uncertainty in school league tables' Fiscal Studies, 32, 207-224
• Nicol, C.J., Caplen, G., Edgar, J. and Browne, W.J. (2009) 'Associations between welfare indicators and environmental choice in laying hens' Animal Behaviour, 78, 413-424
• Goldstein, H. (1986) 'Multilevel mixed linear model analysis using iterative generalized least squares', Biometrika, 73, 43-56