Britain’s happiest places have been mapped by scientists, according to the BBC: Edinburgh is the most miserable place in the country, and they were overbrimming with technical details on exactly how miserable we are in each area of Britain. The story struck a chord, and was lifted by journalists throughout the nation, as we cheerfully castigated ourselves. ‘Misera-Poole?’ asked the Dorset Echo. ‘No Smiles in Donny’, said Doncaster Today.
From the Bromley Times, through Bexley, Dartford and Gravesham, to the Hampshire Chronicle, everyone was keen to analyse and explain their ranking. ‘Basingstoke lacks any sense of community or heart,’ said Reverend Dr Derek Overfield, industrial chaplain for the area. And so on.
Exactly what kind of data is the good reverend explaining there? The Times had some methodological information: ‘Researchers at Sheffield and Manchester universities based their findings on more than 5,000 responses from the annual British Household Panel Survey.’ According to the BBC it was presented in a lecture at some geographical society. ‘However,’ it said quietly, ‘the researchers stress that the variations between different places in Britain are not statistically significant.’
Here, nestled away, halfway through the gushing barrage of data and facts, was an unmarked confession: this entire news story was based on nothing more than random variation.
There are many reasons why you might see differences between different areas in your survey data on how miserable people are, and people being differently miserable is only one explanation. There might also be, of course, the play of chance: 5,000 people in 274 areas doesn’t give you many in each town – fewer than twenty, in fact – so you might just happen to have picked out more miserable people in Edinburgh, and miss the fact that misery is uniformly distributed throughout the country.
This is called sampling error, and it quietly undermines almost every piece of survey data ever covered in any newspaper. Although the phenomenon has spawned a fiendish area of applied maths called ‘statistics’, the basic principles are best understood with a simple game.
Dr W. Edwards Deming was a charismatic American management guru who railed against performance-related pay on the grounds that it arbitrarily rewarded luck. Working in a theatrical field, he demonstrated his ideas with a simple piece of stagecraft he called the Red Bead Experiment.
Deming would appear at management conferences with a big trough containing thousands of beads which were mostly white, but 20 per cent were red. Eight volunteers were then invited up on stage from the audience of management drones: three to be managers, and five to be workers. ‘Your job,’ Deming explained solemnly, ‘is to make white beads.’
He then produced a paddle with fifty holes cut into it, which was passed to each ‘worker’ in turn. They dipped the paddle into the trough, wiggled it around, and tried to produce as many white beads as they could manage through this entirely random process.
‘Go and show the inspectors,’ Deming would say sternly.
‘Only five red beads, well done! Fourteen red beads? I think we need to re-evaluate your skill set.’ Workers were sacked, promoted, retrained and redeployed, to great amusement.
We ignore basic principles like sampling error at our peril, because the illusion of control, which we all carry around for the sake of sanity, is more powerful than we think, and countless workers have had their lives turned to misery for the simple crime of pulling out fifteen red beads.
Back in the world of misery, were the journalists blameless, and guilty only of ignorance? For any individual, nobody can tell. But Dr Dimitris Ballas, the academic who did the research, has a clue: ‘I tried to explain issues of significance to the journalists who interviewed me. Most,’ he says, ‘did not want to know.’