One question I get asked a lot is: “You say that Frewmanackshire is a terrible local authority. How do you know? Do you know what we are working with?” etc etc. It is true that schools with radically different intakes cannot be usefully compared. So I thought I would let you in on how I benchmark schools, and supply you with two jolly new maps.

What I do for secondary schools, is run a simple regression – that is to say, I fit a simple line through all the pupils’ school results in the country after asking it to account for the children’s ethnicity, poverty and prior test results. Unlike other models, the regression contains precisely zero information about the schools – only data about the children.

I work out what FT score each child would get if their fate were the national average for that kind of child. Then you can see who is under- or over-performing. By definition, this kind of approach is zero-sum (the computer will run its lines through the middle of the pack). Someone will always be behind. But it’s helpful to see who is weaker and stronger.

This is a neighbourhood map. It shows where children live. In blue areas, they do better than the model would predict: they are beating the model. In red areas, they are behind. Remember: it is performance against the prediction – not in absolute terms. A poor child who does badly – but not as badly as the model predicts – would show up as a positive.

London is great, obviously. The problems in south Yorkshire and north Lincolnshire are particularly obvious:

/

For the massively nerdy, the R^2 is about 70 per cent, and the model runs off ethnic group, KS2 scores (including squares and cubes), home language, poverty measures and special needs. The geographical units are MSOAs – see here for more details. The script to write the file needed to calculate the FT score is here, the MSOA list is here, and the very basic model is here.

Some care needs to be taken in interpreting the map: schools are not the only missing factor in the model. Things weigh on pupils and schools. It is harder to recruit teachers and motivate children in Hull than Southwark. That said, I have never found a school with poor results on this measure that I think is being cruelly misrepresented. The model works.

This is a map for reformers. It is the kind of exercise one hopes the Department of Education is undertaking to identify small areas where schools can be rapidly improved with a small intervention (look at Corby) and areas where school improvement needs more work (see top righthand side of the country). It also tells you where reform will have diminishing returns (London).

Below, I have included the same sort of map, but for English and maths results only at the age of 11. This is generally a less robust measure. The inputs are the same, but I do not use any prior attainment data (I do not ever use Key Stage 1 data). The R^2 is only at about 40 per cent on this model.

/