I can think of nothing an audience won’t understand. The only problem is to interest them; once they are interested they understand anything in the world.
—ORSON WELLES1
I’ve laid down ten statistical commandments in this book. First, we should learn to stop and notice our emotional reaction to a claim, rather than accepting or rejecting it because of how it makes us feel.
Second, we should look for ways to combine the ‘bird’s eye’ statistical perspective with the ‘worm’s eye’ view from personal experience.
Third, we should look at the labels on the data we’re being given, and ask if we understand what’s really being described.
Fourth, we should look for comparisons and context, putting any claim into perspective.
Fifth, we should look behind the statistics at where they came from – and what other data might have vanished into obscurity.
Sixth, we should ask who is missing from the data we’re being shown, and whether our conclusions might differ if they were included.
Seventh, we should ask tough questions about algorithms and the big datasets that drive them, recognising that without intelligent openness they cannot be trusted.
Eighth, we should pay more attention to the bedrock of official statistics – and the sometimes heroic statisticians who protect it.
Ninth, we should look under the surface of any beautiful graph or chart.
And tenth, we should keep an open mind, asking how we might be mistaken, and whether the facts have changed.
I realise that having ten commandments is something of a cliché. And in truth, they’re not commandments so much as rules of thumb, or habits of mind that I’ve acquired the hard way as I’ve gone along. You might find them worth a try yourself, when you come across a statistical claim of particular interest to you. Of course, I don’t expect you to run personally through the checklist with every claim you see in the media – who has the time for that? Still, they can be useful in forming a preliminary assessment of your news source. Is the journalist making an effort to define terms, provide context, assess sources? The less these habits of mind are in evidence, the louder alarm bells should ring.
Ten rules of thumb is still a lot for anyone to remember, so perhaps I should try to make things simpler. I realise that these suggestions have a common thread – a golden rule, if you like.
Be curious.
Look deeper and ask questions. It’s a lot to ask, but I hope that it is not too much. At the start of this book I begged you not to abandon the idea that we can understand the world by looking at it with the help of statistics, in favour of the cynical distrust so temptingly offered by the likes of Darrell Huff. I believe we can – and should – be able to trust that numbers can give us answers to important questions. My colleagues and I at More or Less work hard to earn listeners’ trust that we’re coming to the same conclusions they would if they investigated the issue themselves. But of course we want listeners to be curious and to question us, too. Nullius in verba. We shouldn’t trust without also asking questions.
The philosopher Onora O’Neill once declared, ‘Well-placed trust grows out of active inquiry rather than blind acceptance.’2 That seems right. If we want to be able to trust the world around us, we need to show an interest and ask a few basic questions. I hope I’ve persuaded you that those questions aren’t obscure or overly technical; they are what any thoughtful, curious person would be happy to ask. And despite all the confusions of the modern world, it has never been easier to find answers to those questions.
Curiosity, it turns out, can be a remarkably powerful thing.
About a decade ago, a Yale University researcher, Dan Kahan, showed students some footage of a protest outside an unidentified building. Some of the students were told that it was a pro-life demonstration outside an abortion clinic. Others were informed that it was a gay rights demonstration outside an army recruitment office. The students were asked some factual questions. Was it a peaceful protest? Did the protesters try to intimidate people passing by? Did they scream or shout? Did they block the entrance to the building?
The answers people gave depended on the political identities they embraced. Conservative students who believed they were looking at a demonstration against abortion saw no problems with the protest: no abuse, no violence, no obstruction. Students on the left who thought they were looking at a gay rights protest reached the same conclusion: the protesters had conducted themselves with dignity and restraint.
But right-wing students who thought they were looking at a gay rights demonstration reached a very different conclusion; as did left-wing students who believed they were watching an anti-abortion protest. Both these groups concluded that the protesters had been aggressive, intimidating and obstructive.3
Kahan was studying a problem we met in the first chapter: the way our political and cultural identity – our desire to belong to a community of like-minded, right-thinking people – can, on certain hot-button issues, lead us to reach the conclusions we wish to reach. Depressingly, not only do we reach politically comfortable conclusions when parsing complex statistical claims on issues such as climate change, we reach politically comfortable conclusions regardless of the evidence of our own eyes.*
And, as we saw earlier, expertise is no guarantee against this kind of motivated reasoning: Republicans and Democrats with high levels of scientific literacy are further apart on climate change than those with little scientific education. The same disheartening pattern holds from nuclear power to gun control to fracking: the more scientifically literate opponents are, the more they disagree. The same is true for numeracy. ‘The greater the proficiency, the more acute the polarization,’ notes Kahan.4
After a long and fruitless search for an antidote to tribalism, Kahan could be forgiven for becoming jaded.5 Yet a few years ago, to his surprise, Kahan and his colleagues stumbled upon a trait that some people have – and that other people can be encouraged to develop – which inoculates us against this toxic polarisation. On the most politically polluted, tribal questions, where intelligence and education fail, this trait does not.
And if you’re desperately, burningly curious to know what it is – congratulations. You may be inoculated already.
Curiosity breaks the relentless pattern. Specifically, Kahan identified ‘scientific curiosity’. That’s different from scientific literacy. The two qualities are correlated, of course, but there are curious people who know rather little about science (yet), and highly trained people with little appetite to learn more.
More scientifically curious Republicans aren’t further apart from Democrats on these polarised issues. If anything, they’re slightly closer together. It’s important not to exaggerate the effect. Curious Republicans and Democrats still disagree on issues such as climate change – but the more curious they are, the more they converge on what we might call an evidence-based view of the issues in question. Or to put it another way, the more curious we are, the less our tribalism seems to matter. (There is little correlation between scientific curiosity and political affiliation. Happily, there are plenty of curious people across the political spectrum.)
Although the discovery surprised Kahan, it makes sense. As we’ve seen, one of our stubborn defences against changing our minds is that we’re good at filtering out or dismissing unwelcome information. A curious person, however, enjoys being surprised and hungers for the unexpected. He or she will not be filtering out surprising news, because it’s far too intriguing.
The scientifically curious people Kahan’s team studied were originally identified with simple questions, buried in a marketing survey so that people weren’t conscious that their curiosity was being measured. One question, for example, was ‘How often do you read science books?’ Scientifically curious people are more interested in watching a documentary about space travel or penguins than a basketball game or a celebrity gossip show. And they didn’t just answer survey questions differently, they also made different choices in the psychology lab. In one experiment, participants were shown a range of headlines about climate change and invited to pick the ‘most interesting’ article to read. There were four headlines. Two suggested climate scepticism and two did not; two were framed as surprising and two were not:
1. ‘Scientists Find Still More Evidence that Global Warming Actually Slowed in Last Decade’ (sceptical, unsurprising)
2. ‘Scientists Report Surprising Evidence: Arctic Ice Melting Even Faster than Expected’ (surprising and not sceptical)
3. ‘Scientists Report Surprising Evidence: Ice Increasing in Antarctic, Not Currently Contributing to Sea Level Rise’ (sceptical and surprising)
4. ‘Scientists Find Still More Evidence Linking Global Warming to Extreme Weather’ (neither surprising nor sceptical)
Typically we’d expect people to reach for the article that pandered to their prejudices: the Democrats would tend to favour a headline that took global warming seriously while Republicans would prefer something with a sceptical tone. Scientifically curious people – Republicans or Democrats – were different. They were happy to grab an article which ran counter to their preconceptions, as long as it seemed surprising and fresh. And once you’re actually reading the article, there’s always a chance that it might teach you something.
A surprising statistical claim is a challenge to our existing world-view. It may provoke an emotional response – even a fearful one. Neuroscientific studies suggest that the brain responds in much the same anxious way to facts which threaten our preconceptions as it does to wild animals which threaten our lives.6 Yet for someone in a curious frame of mind, in contrast, a surprising claim need not provoke anxiety. It can be an engaging mystery, or a puzzle to solve.
A curious person might, at this point, have some questions. When I met Dan Kahan, the question that was most urgent in my mind was – can we cultivate curiosity? Can we become more curious, and can we inspire curiosity in others?
There are reasons to believe that the answers are ‘yes’. One reason, says Kahan, is that his measure of curiosity suggests that incremental change is possible. When he measures scientific curiosity, he doesn’t find a lump of stubbornly incurious people at one end of the spectrum and a lump of voraciously curious people at the other, with a yawning gap in the middle. Instead, curiosity follows a continuous bell curve: most people are either moderately incurious or moderately curious. This doesn’t prove that curiosity can be cultivated; perhaps that bell curve is cast in iron. Yet it does at least hold out some hope that people can be nudged a little further towards the curious end of that curve, because no radical leap is required.
A second reason is that curiosity is often situational. In the right place, at the right time, curiosity will smoulder in any of us.* Indeed, Kahan’s discovery that an individual’s scientific curiosity persisted over time was a surprise to some psychologists. They had believed, with some reason, that there was no such thing as a curious person, just a situation that inspired curiosity. In fact it does now seem that people can tend to be curious or incurious. That does not alter the fact that curiosity can be fuelled or dampened by context. We all have it in us to be curious, or not, about different things at different times.
One thing that provokes curiosity is the sense of a gap in our knowledge to be filled. George Loewenstein, a behavioural economist, framed this idea in what has become known as the ‘information gap’ theory of curiosity. As Loewenstein puts it, curiosity starts to glow when there’s a gap ‘between what we know and what we want to know’. There’s a sweet spot for curiosity: if we know nothing, we ask no questions; if we know everything, we ask no questions either. Curiosity is fuelled once we know enough to know that we do not know.7
Alas, all too often we don’t even think about what we don’t know. There’s a beautiful little experiment about our incuriosity, conducted by the psychologists Leonid Rozenblit and Frank Keil. They gave their experimental subjects a simple task: to look through a list of everyday objects such as a flush lavatory, a zip fastener and a bicycle, and to rate their understanding of each object on a scale of one to seven.8
After people had written down their ratings, the researchers would gently launch a devastating ambush. They asked the subjects to elaborate. Here’s a pen and paper, they would say; please write out your explanation of a flush lavatory in as much detail as you can. By all means include diagrams.
It turns out that this task wasn’t as easy as people had thought. People stumbled, struggling to explain the details of everyday mechanisms. They had assumed that those details would readily spring to mind, and they did not. And to their credit, most experimental subjects realised that they’d been lying to themselves. They had felt they understood zip fasteners and lavatories, but when invited to elaborate, they realised they didn’t understand at all. When people were asked to reconsider their previous one-to-seven rating, they marked themselves down, acknowledging that their knowledge had been shallower than they’d realised.
Rozenblit and Keil called this ‘the illusion of explanatory depth’. The illusion of explanatory depth is a curiosity-killer and a trap. If we think we already understand, why go deeper? Why ask questions? It is striking that it was so easy to get people to pull back from their earlier confidence: all it took was to get them to reflect on the gaps in their knowledge. And as Loewenstein argued, gaps in knowledge fuel curiosity.
There is more at stake here than zip fasteners. Another team of researchers, led by Philip Fernbach and Steven Sloman, authors of The Knowledge Illusion, adapted the flush lavatory question to ask about policies such as a cap-and-trade system for carbon emissions, a flat tax, or a proposal to impose unilateral sanctions on Iran. The researchers, importantly, didn’t ask people whether or why they were in favour of or against these policies – there’s plenty of prior evidence that such questions would lead people to dig in. Instead, Fernbach and his colleagues just asked them the same simple question: please rate your understanding on a scale of one to seven. Then, the same devastating follow-up: please elaborate; tell us exactly what unilateral sanctions are and how a flat tax works. And the same thing happened. People said, yes, they basically understood these policies fairly well. Then when prompted to explain, the illusion was dispelled. They realised that perhaps they didn’t really understand at all.9
More striking was that when the illusion faded, political polarisation also started to fade. People who would have instinctively described their political opponents as wicked, and who would have gone to the barricades to defend their own ideas, tended to be less strident when forced to admit to themselves that they didn’t fully understand what it was they were so passionate about in the first place. The experiment influenced actions as well as words: researchers found that people became less likely to give money to lobby groups or other organisations which supported the positions they had once favoured.10
It’s a rather beautiful discovery: in a world where so many people seem to hold extreme views with strident certainty, you can deflate somebody’s overconfidence and moderate their politics simply by asking them to explain the details. Next time you’re in a politically heated argument, try asking your interlocutor not to justify herself, but simply to explain the policy in question. She wants to introduce a universal basic income, or a flat tax, or a points-based immigration system, or ‘Medicare for all’. OK: that’s interesting. So what exactly does she mean by that? She may learn something as she tries to explain. So may you. And you may both find that you understand a little less, and agree a little more, than you had assumed.
Figuring out the workings of a flush lavatory, or understanding what a cap-and-trade scheme really is, can require some effort. One way to encourage that effort is to embarrass someone by innocently inviting an overconfident answer on a scale of one to seven; but another, kinder, way is to engage their interest. As Orson Welles said, once people are interested they can understand anything in the world.
How to engage people’s interest is neither a new problem nor an intractable one. Novelists, screenwriters and comedians have been figuring out this craft for as long as they have existed. They know that we love mysteries, are drawn in by sympathetic characters, enjoy the arc of a good story, and will stick around for anything that makes us laugh. And scientific evidence suggests that Orson Welles was absolutely right: for example, studies in which people were asked to read narratives and non-narrative texts found that they zipped through the narrative at twice the speed, and recalled twice as much information later.11
As for humour, consider the case of the comedian Stephen Colbert’s ‘civics lesson’. Before his current role as the host of The Late Show, Colbert presented The Colbert Report in character as a blowhard right-wing commentator.* In March 2011, Colbert began a long-running joke in which he explored the role of money in US politics. He decided that he needed to set up a Political Action Committee – a PAC – to raise funds in case he decided to run for President. ‘I clearly need a PAC but I have no idea what PACs do,’ he explained to a friendly expert on air.
Over the course of the next few weeks, Colbert had PACs – and Super PACs, and 501(c)(4)s – explained to him: from where they could accept donations, up to what limits, with what transparency requirements, and to spend on what. He was to discover that the right combination of fundraising structures could be used to raise almost any amount of money for almost any purpose, with almost no disclosure. ‘Clearly (c)(4)s have created an unprecedented, unaccountable, untraceable cash tsunami that will infect every corner of the next election,’ he mused. ‘And I feel like an idiot for not having one.’
Colbert later learned how to dissolve his fundraising structures and keep the money – without notifying the taxman. By repeatedly returning to the topic and – in character – demanding advice as to how to abuse the electoral rules, Colbert explored campaign finance in far more depth than any news report could have dreamed of doing.
Did all of this actually improve viewers’ knowledge of the issue? It seems so. A team including Kathleen Hall Jamieson, who also worked with Dan Kahan on the scientific curiosity research, used the Colbert storyline to investigate how much people learned amid the laughter. They found that watching The Colbert Report was correlated with increased knowledge about Super PACs and 501(c)(4) groups – how they worked, what they could legally do. Reading a newspaper or listening to talk radio also helped, but the effect of The Colbert Report was much bigger. One day a week of watching Colbert taught people as much about campaign finance as four days a week reading a newspaper, for example – or five extra years of schooling.
Of course this is a measure of correlation, not causation. It’s possible that the people who were already interested in Super PACs tuned in to Colbert to hear him wisecrack about them. Or perhaps politics junkies know about Super PACs and also love watching Colbert. But I suspect the show did cause the growing understanding, because Colbert really did go deep into the details. And large audiences stuck with him – because he was funny.12
You don’t have to be one of America’s best-loved comedians to pull off this trick. The NPR podcast Planet Money once shed light on the details of the global economy by designing, manufacturing and importing several thousand T-shirts. This allowed a long-running storyline investigating cotton farming; the role of automation in textiles; how African communities make new fashions out of donated American T-shirts; the logistics of the shipping industry; and strange details such as the fact that the men’s shirts, which were made in Bangladesh, attract a tariff of 16.5 per cent, whereas the women’s shirts, made in Colombia, are duty-free.13
These examples should be models for communication, precisely because they inspire curiosity. ‘How does money influence politics?’ is not an especially engaging question, but ‘If I were running for President, how would I raise lots of money with few conditions and no scrutiny?’ is much more intriguing.
Those of us in the business of communicating ideas need to go beyond the fact-check and the statistical smackdown. Facts are valuable things, and so is fact-checking. But if we really want people to understand complex issues, we need to engage their curiosity. If people are curious, they will learn.*
I’ve found this in my own work with the team who make More or Less for the BBC. The programme is often regarded affectionately as a myth-buster, but I feel that our best work is when we use statistics to illuminate the truth rather than to debunk a stream of falsehoods. We try to bring people along with us as we explore the world around us with the help of reliable numbers. What’s false is interesting – but not as interesting as what’s true.
After the referendum of 2016, in which my fellow British voters decided to leave the European Union, the economics profession engaged in some soul-searching. Most technical experts thought that leaving the EU was a bad idea – costly, complex, and unlikely to deliver many of the promised benefits or to solve the country’s most pressing problems. Yet, as one infamous soundbite put it, ‘the people in this country have had enough of experts’.* Few people seemed to care what economists had to say on the subject, and – to our credit, I think – professional economists wanted to understand what we had done wrong and whether we might do better in future.
Later, at a conference about ‘the profession and the public’, the great and the good of the British economics community pondered the problem and discussed solutions.14 We needed to be more chatty and approachable on Twitter, suggested one analysis. We needed to express ourselves clearly and without jargon, offered many speakers – not unreasonably.
My own perspective was slightly different. I argued that we were operating in a politically polarised environment, in which almost any opinion we might offer would be fiercely contested by partisans. Economists deal with controversial issues such as inequality, taxation, public spending, climate change, trade, immigration and, of course, Brexit. In such a febrile environment, speaking slowly and clearly will only get you so far. To communicate complex ideas, we needed to spark people’s curiosity – even inspire a sense of wonder. The great science communicators, after all – people such as Stephen Hawking and David Attenborough – do not win over people simply by using small words, crisply spoken. They stoke the flames of our curiosity, making us burn with desire to learn more. If we economists want people to understand economics, we must first engage their interest.
What is true of economists is equally true for scientists, social scientists, historians, statisticians or anyone else with complex ideas to convey. Whether the topic is the evolution of black holes or the emergence of Black Lives Matter, the possibility of precognition or the necessity of preregistration, the details matter – and presented in the right way, they should always have the capacity to fascinate us.
Awaken our sense of wonder, I say to my fellow nerd-communicators. Ignite the spark of curiosity and give it some fuel, using the time-honoured methods of storytelling, character, suspense and humour. But let’s not rely on the journalists and the scientists and the other communicators of complex ideas. We have to be responsible for our own sense of curiosity. As the saying goes, ‘only boring people get bored’. The world is so much more interesting if we take an active interest in it.
‘The cure for boredom is curiosity,’ goes an old saying. ‘There is no cure for curiosity.’15 Just so: once we start to peer beneath the surface of things, become aware of the gaps in our knowledge, and treat each question as the path to a better question, we find that curiosity is habit-forming.
Sometimes we need to think like Darrell Huff; there is a place in life for the mean-minded, hard-nosed scepticism that asks, Where’s the trick? Why is this lying bastard lying to me?16 But while ‘I don’t believe it’ is sometimes the right starting point when confronted with a surprising statistical claim, it is a lazy and depressing place to finish.
And I hope you won’t finish there. I hope that I have persuaded you that we should make more room both for the novelty-seeking curiosity that says ‘tell me more’, and the dogged curiosity that drove Austin Bradford Hill and Richard Doll to ask why so many people were dying of lung cancer, and whether cigarettes might be to blame.
If we want to make the world add up, we need to ask questions – open-minded, genuine questions. And once we start asking them, we may find it is delightfully difficult to stop.
___________
* The study is titled ‘They Saw a Protest’, echoing a classic psychology paper from 1954, ‘They Saw a Game’, which found similarly biased perceptions when rival fans watched footage of a bad-tempered game of football.
* Trolls, populists, manufacturers of outrage and other professional controversialists will, of course, try to frame debates in ways that crush curiosity and reinforce preconceptions. But curious and open-minded folk can also frame debates, and we would be wise to take the opposite tack.
* I was once a guest on The Colbert Report. Stephen was a gracious host. In the green room, as himself, he explained the basic idea of the show to me: ‘I’ll be in character, and my character is an idiot.’ Then later, after getting into character, ‘I’m going to tear you apart, Harford!’
* And if people are not curious, they will not learn until their curiosity can somehow be sparked into life. Remember the TV producers who were making a programme about why inequality had risen, but apparently weren’t curious enough to check whether it had?
* When pro-Brexit campaigner Michael Gove said this, he was referring specifically to experts based at international organisations such as the International Monetary Fund. The statement, however, took on a life of its own.