RULE EIGHT
Don’t Take Statistical Bedrock for Granted
“What do you base your facts on?”
“Statistics from the International Monetary Fund and the United Nations, nothing controversial. These facts are not up for discussion. I am right, and you are wrong.”
• Interview with Hans Rosling1
Monday, October 9, 1974. The place: Washington, DC, near the picturesque Tidal Basin—a quiet, leafy sanctuary not far from the White House. The time: two o’clock in the morning. A car is weaving around in the darkness, fast, with its headlights off. The police pull the car over, at which point a flamboyantly dressed woman with two black eyes jumps out of the passenger’s side, runs down the road yelling alternately in English and Spanish, and leaps into the water. The police pull her out and she tries to jump in again, at which point they handcuff her. At the wheel is an elderly fellow with broken glasses and minor cuts to his face. He is steaming drunk.2
Just another night in DC, perhaps. Except the woman, Annabelle Battistella, was better known as Fanne Foxe, the Argentine Firecracker, an erotic dancer at the Silver Slipper nightclub. And the man was one of the most powerful men in the United States: Wilbur Mills, an Arkansas congressman since the 1930s, who as the long-serving chair of the House Ways and Means Committee effectively had veto power over most legislation. These were, however, deferential times. The police offered to drive Mills home to his wife in his own car, and he was reelected by voters just a few weeks later.
But hard on the heels of that electoral triumph, Mills—drunk again—appeared onstage with Foxe in the middle of her act, received a peck on the cheek, and made his exit, stage left. To be caught cavorting with a stripper once might be considered a misfortune. To do it twice suggested carelessness. His colleagues had a quiet word. Wilbur Mills stepped down from the Ways and Means Committee and joined Alcoholics Anonymous. Fanne Foxe rebranded herself “The Tidal Basin Bombshell,” wrote a tell-all memoir, and eventually retired into obscurity.3
To most people, this story might be dimly remembered as America’s third most spectacular sex scandal. But in my home country of Nerdland, it has another significance. At the time, Congress was deadlocked over a putative new agency, the Congressional Budget Office, which would provide advice to Congress about the budgetary costs of different policy proposals. One congressional dinosaur in particular objected to plans to appoint a woman as its director. But Wilbur Mills’s resignation triggered the usual game of musical chairs, the indirect consequence of which was that the deadlock was broken. The Congressional Budget Office was duly established, and with the dinosaur ambling off to graze elsewhere on Capitol Hill, there was no obstacle to its first director being the woman every sensible person would have wanted: Alice Rivlin. Forty years later, she reflected, “I owed my job to Fanne Foxe.”4
After this strange beginning, Alice Rivlin went on to lead the Congressional Budget Office to glory.* The CBO had been established by Congress to serve as a counterweight to what was seen as Richard Nixon’s overreaching and overmighty presidency. Congress saw the value of having better statistics and more analysis of policy issues. But Rivlin interpreted this role in a particular way: rather than churning out talking points for the majority party or running statistical errands for the powerful chairs of congressional committees, she would supply impartial, high-quality information and analysis to Congress as a whole. In the judgment of one academic, the CBO duly became “one of the most influential and well-regarded institutions in Washington . . . the authoritative source of information on the budget and the economy.”5
Alice Rivlin’s deputy and later one of her successors, Robert Reischauer, described the CBO as
basically a manhole in which Congress would have a bill or something, and it would lift up the manhole cover and put the bill down it, and you would hear grinding noises, and twenty minutes later a piece of paper would be handed up, with the cost estimate, the answer, on it. No visibility, [just] some kind of mechanism below the ground level doing this . . . noncontroversial, the way the sewer system is.6
The analogy is apt, and not just because sewers are invisible and uncontroversial. Independent statistical agencies, like sewers, are an essential part of modern life. As with sewers, we tend to take them for granted until something goes wrong. And like sewers, they can suffer badly from neglect—or because someone tries to force something unsuitable through them for their own selfish or foolish reasons.
The official statistics and analyses produced by organizations such as the CBO are more important than we might think, and more useful in the everyday lives of ordinary citizens. They are also under threat—and we should defend them. They should not have to depend on twists of fate involving drunk congressmen and strippers.
The CBO was established, remember, with Richard Nixon in mind. But Nixon had resigned before the CBO began operations, and the first president to object to what the CBO was doing was not a Republican like Nixon, but a Democrat: Jimmy Carter. With oil prices spiking in the late 1970s, President Carter had ambitious goals to improve America’s energy efficiency. Alice Rivlin’s CBO team evaluated the proposals and judged that they wouldn’t work as well as Carter hoped.
“It made the Carter Administration unhappy,” Rivlin later recalled. The House Speaker, Thomas P. “Tip” O’Neill, also a Democrat, wasn’t happy either. “He was fighting for the legislation and the CBO wasn’t helping.”7
No. It wasn’t helping. This was exactly the point: Alice Rivlin knew that the value of the CBO would lie in being impartial rather than in serving up propaganda for the party in power. It wasn’t long before the party in power was the Republicans again, and it was their turn for their grand claims to smack into the unyielding reality of the CBO’s independent opinion. In 1981, the CBO argued that the budget deficit was likely to be far higher than the Reagan White House projected. President Reagan called the CBO numbers “phony.”
In 1983, Alice Rivlin left the CBO after eight years in charge. Successive administrations continued to put pressure on it—in the 1990s, for example, leading Democrats wanted the CBO to produce a more flattering analysis of President Clinton’s health care reforms—and it continued to assert its independence.8 The CBO certainly isn’t perfect: much of its task is to make projections of the future gap between spending and tax revenue, and—as we’ll discuss in chapter ten—such economic forecasts are hard to make; official agencies often get them wrong. The important point, though, is that they don’t make politically expedient errors, systematically warping their forecasts to fit a political agenda. Evaluations of the CBO9 have tended to find that it produces forecasts that are as accurate as we might reasonably hope, and—crucially—unbiased.*
In the UK, the Office for Budget Responsibility (OBR) performs a similar role to the CBO. It was established as an independent agency only in 2010. Forecasts of spending, tax receipts, and other economic variables had previously been made by the Treasury, where officials are more directly answerable to politicians. That enables us to make an interesting comparison: Are the OBR forecasts better? It turns out they are, substantially so.10 That’s encouraging for the OBR’s reputation and future work, but it also suggests there was previously a problem—that before 2010, the Treasury’s economists had been routinely shaping their forecasts to please their political overlords.
The CBO and OBR are far from the only kinds of statistical agencies that need to assert their political independence. While they project the future impacts of proposed tax or spending changes, many other agencies describe current realities. There are censuses, which try to estimate how many people live in different parts of the country, along with some basic information about those people. There are economic statistics—measuring inflation, unemployment, economic growth, trade, and inequality. There are social statistics—measuring crime, education, access to housing, migration, and well-being. There are studies of particular industries, or of issues such as environmental pollution.
Every country has its own arrangements for putting together these official statistics. In the UK, many are produced and published by one organization, the Office for National Statistics. In the United States, the task is spread across a range of agencies including the Bureau of Economic Analysis, the Bureau of Labor Statistics, the Census Bureau, the Federal Reserve, the Department of Agriculture, and the Energy Information Administration.
How useful is all this counting and measurement? Very useful indeed; it’s hard to overstate how useful. The numbers produced by such agencies are a nation’s statistical bedrock. When journalists, think tanks, academics, and fact-checkers want to know what is going on, their analysis usually rests, either directly or ultimately, on this bedrock. I’ll have more to say about the costs and varied benefits of producing professional and impartial official statistics later in the chapter. But perhaps the most vivid argument for their value is to look at attempts to distort, discredit, or suppress them.
As a presidential candidate in 2016, Donald Trump faced a problem. His campaign wanted to claim that the American economy was broken, but official statistics showed that the unemployment rate was very low—below 5 percent and falling. There could have been a thoughtful response to that—for example, that the unemployment rate doesn’t measure the quality, security, or earning power of jobs. But Trump took the simpler path of repeatedly dismissing unemployment figures as “phony” and “total fiction” and claiming that the true unemployment rate was 35 percent.
Simply inventing your own numbers is a tactic more often used by totalitarian dictators than by candidates for democratic election, but Trump evidently figured it was a tactic that would be effective. And perhaps he was right. His supporters believed him: just 13 percent of them trusted the economic data produced by the federal government, versus 86 percent of those who voted for Hillary Clinton.11
As president, Trump changed his mind. According to the official data, unemployment crept even lower after he had assumed office. Now, however, Trump wished to get credit for this rather than to dismiss it. His spokesman Sean Spicer declared, with a straight face, “I talked to the president prior to this, and he said to quote him very clearly. They may have been phony in the past, but it’s very real now.” Amusing as this kind of shamelessness might be, it also carries a real risk—that Trump’s opponents will start to distrust official statistics just as much as his supporters do.12
If you grow tired of undermining trust in your own statistical agency when it isn’t producing politically convenient figures, you could always attack the statistical agency of someone else. For example, after Germany’s leader Angela Merkel took the politically risky step of welcoming almost a million refugees into the country in 2015, Donald Trump wanted to use Germany as a cautionary tale. “Crime in Germany is way up,” he tweeted in June 2018. Look at all the crimes those refugees were causing!
Unfortunately for President Trump, one group of people stood in the way to spoil his story: German statisticians. Their latest figures, a month before Trump’s tweet, showed that not only was crime in Germany not “way up,” it was at its lowest level since 1992.13 Unabashed, Trump had an answer. A few hours later, he tweeted: “Crime in Germany is up 10% plus (officials do not want to report these crimes).”14
The allegation is implausible. In part that’s because the ministry in Germany responsible for putting together the police crime statistics was run by Horst Seehofer, an immigration hawk who in the same year threatened to resign if Germany’s immigration policy wasn’t tightened up: Seehofer would hardly have wanted to pressure officials to hide uncomfortable truths about migration. It’s also implausible because Germany has not acquired a reputation for political interference in statistics.
Sadly, that’s not true for every country. Around the world, pressure to fiddle the figures is real and widespread—and the consequences for statisticians can be far more serious than grumbling from senior politicians.
In 2010, the economist Andreas Georgiou left a two-decade career at the International Monetary Fund, bringing his baby daughter with him from Washington, DC, to his home country, Greece. His mission was to run ELSTAT, Greece’s new statistical agency.
At the time, Greece’s statistics were in bad shape. They had never been well funded or well respected. When, in 2002, the economist Paola Subacchi visited the Greek statistical office, she found it tucked away in a residential suburb of Athens, “in a square of ordinary shops, and I had to hunt for a doorway in a 1950s apartment block that took me up some stairs to a dusty room with a handful of people. I can’t remember seeing any computers. It was extraordinary, not a professional operation at all.”15
But when Georgiou arrived, there was more to worry about than dust and outdated technology. The entire world had reached the conclusion that you should trust Greek official statistics about as much as you should trust their giant wooden horses. Eurostat, the statistical office of the European Union, had repeatedly complained about the credibility and quality of the official Greek economic data. The European Commission issued a blistering report about them.16
The basic problem was that Greece was supposed to keep its government budget deficit at a modest level. The budget deficit is the amount the government borrows each year to cover any gap between what it spends and what it receives in taxes. One of the obligations that come with membership in the eurozone is for a country to keep its deficit below 3 percent of gross domestic product, with some exemptions for various exceptional circumstances. (Economically speaking, it’s not a very sensible rule—but that’s another story for another book.) That target was onerous, so why not tweak the figures until all seemed well? One year the Greek accounts left out several billion euros of borrowing to pay for hospitals. Another year, they omitted a big chunk of the cost of the military. They also did a deal with the investment bank Goldman Sachs that effectively made borrowing look like a different kind of transaction, and thus not counting toward the deficit.17
In 2009, the shock of the global financial crisis was followed by the realization that Greece had been underplaying its borrowing for years. Nobody believed its debts could be repaid. The EU and IMF moved in with the customary mix of a bailout and some brutal austerity, and the Greek economy collapsed. Into this situation stepped Andreas Georgiou. He might not be able to rescue Greece’s prosperity, but there was some hope that he would save the reputation of Greek official statistics.
Georgiou’s first priority was to look at the deficit figures for 2009, the most recent available. The initial forecast, from the Greek Ministry of Finance, had been 3.7 percent of gross domestic product—not too far outside the EU’s target, but unfortunately quite implausible. Even before Georgiou’s arrival the Greek authorities had revised that to a shocking 13.6 percent. Eurostat was still unconvinced. Within a few months, Georgiou published his conclusion: the deficit had actually been 15.4 percent, a grimly large number. But it was, at least, believable—and Eurostat believed it.
It was then that Georgiou’s troubles began. First, there was an almighty row within ELSTAT. The police eventually realized that Georgiou’s email account had been hacked by his own deputy, ELSTAT’s vice president. Then the Greek Prosecutor of Economic Crimes began legal action against Georgiou, accusing him of deliberately exaggerating Greece’s deficit and causing immense damage to the Greek economy. Various other charges were added, including failing to allow ELSTAT’s board to vote on what the deficit should be. (The idea that the size of Greece’s budget deficit should be put to a vote seems more Eurovision than Eurostat.) The potential sentence for Georgiou’s “crimes” was life imprisonment. The judicial system threw out the charges six times, but they were repeatedly reinstated by the Greek supreme court. Indeed, his convictions, acquittals, and reconvictions have been so frequent that it is hard to have any confidence that any verdict will stick.18 This is harassment worthy of a Kafka novel.
Of course it is possible that Georgiou really is a traitor. But it does not seem likely. Eighty former chief statisticians from around the world signed a letter protesting against his treatment, Eurostat repeatedly signed off on the quality of his work, and in 2018 he received a special commendation from a group of respected professional bodies including the International Statistical Association, the American Statistical Association, and the Royal Statistical Society “for his competency and strength in the face of adversity, his commitment to the production of quality and trustworthiness of official statistics and his advocacy for the improvement, integrity and independence of official statistics.”19
Andreas Georgiou is not the only statistician who has shown courage in adversity, as Graciela Bevacqua, a long-serving Argentine statistician, could attest. Argentina has long suffered from high inflation. The Argentine government, under a husband-and-wife pair of populist presidents, Néstor Kirchner (president 2003–07) and Cristina Fernández de Kirchner (president 2007–15), decided to solve the problem not by reducing inflation but by changing the inflation statistics. Bevacqua found herself receiving some discomfiting demands.
For example, she was instructed to round down all decimals in the monthly inflation figures—as though Argentine computers had run out of decimal points. That makes more difference than you might think, because each distortion compounds the earlier ones: compounding inflation of 1 percent a month gives 12.7 percent a year, while 1.9 percent a month is 25.3 percent a year. Funnily enough, official estimates of annual inflation in Argentina have tended to be close to the first number, and independent unofficial estimates have been closer to the second.
When Graciela Bevacqua produced a monthly figure of 2.1 percent inflation at the beginning of 2007, her supervisors weren’t happy. Hadn’t they told her to produce a number below 1.5 percent? They told her to take a vacation, then sacked her when she returned, transferring her from the statistical agency to a library and slashing her pay by two-thirds. She resigned soon afterward.20
With Bevacqua out of the way—and having been made an example of—Argentina’s official inflation numbers showed inflation of below 10 percent. That’s high by the standards of a developed country but still implausibly low. Most independent experts reckoned it was close to 25 percent, and a group of those experts produced their own unofficial price index, advised by none other than Graciela Bevacqua—who was promptly fined $250,000 for false advertising.
As with Georgiou, international observers stand behind Bevacqua and her methods, and with a new government in Argentina it looks like she’ll be OK. As for Georgiou, he stuck it out for five years at ELSTAT and then returned to the United States, leaving behind him an organization with a credibility it never had before he arrived. He is most unlikely to go to prison, but other Greek statisticians will have noticed the way he was persecuted for trying to tell the truth about the statistics that were his responsibility. “It will not be lost on them that their well-being—not only professional but personal—is at risk if they do the right thing and follow the law,” he told Significance magazine. He added that the Greek government was only damaging itself in the long run, by “undermining the statistics which they themselves use. They’re undermining the credibility of the country itself.” Meanwhile, the people who repeatedly understated Greece’s deficit before the crisis seem to have escaped censure.21
Heroic as Andreas Georgiou and Graciela Bevacqua have shown themselves to be, we would be naive to assume that every statistician has their determination, or that every attempt to exert pressure comes to public attention. One respected statistician, Professor Denise Lievesley, told me that a fellow statistician from Africa had been told that if he didn’t produce the numbers that his nation’s president required, his children would be murdered. For understandable reasons, she didn’t wish to identify him.22 It would be equally understandable if he had decided to comply.
There are subtler ways to undermine the independence of official statisticians. In Tanzania, in late 2018, the government passed a law that made criticism of official statistics a criminal offense, punishable with fines or a minimum of three years in prison. Candidates for the presidency there will think twice before following Trump’s example of calling the jobless figures “phony.” But imprisoning anyone who finds fault with government statistics is not only an outrage against free speech, it will ensure that faults go uncorrected. Tanzania’s move—which has been criticized by the World Bank—would be the perfect prelude to distorting its own statistics for political reasons.23
In India, Prime Minister Narendra Modi’s government quietly stopped publishing data on unemployment in 2019. Modi had made big promises about creating jobs, but in the run-up to that year’s election (which he won comfortably) it began to look as though reality was going to prove embarrassing. The answer was simply to find an excuse to stop publishing, pending the arrival of “improvements” in the data. One Indian expert explained to the Financial Times exactly what was going on: “It’s very clear that for a long time, the objective of the government has been to keep the picture fuzzy.”24
Even in countries with the most solid of reputations in Nerdland, serious conflicts can arise between the politicians and the statisticians. The Canadian statistical agency, Statistics Canada, has long been admired by statistical agencies around the world for its competence and independence—but the same qualities are not always appreciated closer to home. First the government under Prime Minister Stephen Harper (2006–15) tried to abolish the traditional census, replacing it with a voluntary survey—something that would have been cheaper and more convenient but massively less robust. The chief statistician, Munir Sheikh, made his objections very public and resigned.25 The Harper government also wanted to move IT infrastructure to an organization called Shared Services Canada; when the administration of the next prime minister, Justin Trudeau, pressed ahead with that plan, the next chief statistician, Wayne Smith, also resigned. He argued that if his data and computing power were being moved into another organization, he could not guarantee the confidentiality of the statistics he was collecting. Nor could he be sure that Canadian statisticians would remain independent, since they could be squeezed or pressured by any government official with power over Shared Services Canada.
It’s fair to say that Statistics Canada’s reputation for robust independence has only been enhanced by these episodes. But there is a risk that if one side of the political spectrum is seen as hostile to the statisticians while the other side leaps to their defense, statistics itself becomes a partisan political issue. With that in mind, perhaps we should be reassured that when the last two chief statisticians of Canada resigned in protest, they did so under two different governments.26
In Puerto Rico, the government’s response to troublesome statisticians was more radical: they attempted to disband entirely the statistical agency, PRIS, soon after the disastrous hurricane of September 2017. The ostensible reason was that PRIS was too expensive: its million-dollar budget could be better spent elsewhere.
That may not have been the real reason. You may recall that shortly after that hurricane, President Trump expressed gratitude that the death toll had been so small—sixteen or seventeen people, not a “real tragedy” like the hurricane that had flooded New Orleans twelve years earlier. That was glib, but in line with the official death toll at the time—which later rose, but only to just over fifty. It seemed suspiciously low. Numerous independent researchers attempted to figure out their own estimates, to include not just the people who had been killed outright by the storm but those who had later died because of overstretched medical services, or because they were cut off from assistance by blocked roads and downed power lines. Alexis Santos was one of these researchers. He is a demographer at Penn State, and his Puerto Rican mother was on the island when the hurricane struck. Professor Santos put out an estimate that around a thousand people had died, directly or indirectly, as a result of the hurricane. It was big news in Puerto Rico. Even graver estimates were published later.
All of these estimates were built on demographic data from PRIS. PRIS itself, meanwhile, was suing the Puerto Rican health ministry in an effort to get accurate, timely information about the dead.27 Given the embarrassment it was causing the administration, perhaps the threat to disband PRIS was not entirely surprising.
Still, let’s take the given reason at face value: Is PRIS really worth its million-dollar budget? The question of how much value official statistics create is a valid one, and there are fewer attempts to quantify this than one might hope.
One cost-benefit exercise was conducted in the UK in the run-up to the 2011 census; it produced a long list of benefits from the census, everything from informing the debate over pension policy, to ensuring that schools and hospitals were located in the right areas, to enabling all sorts of other statistics to be calculated. After all, you can’t produce any “per capita” statistics—from crime to teen pregnancy to income to the unemployment rate—unless you know the population.
The analysts observed that “statistics in themselves don’t deliver benefits. It’s the use of statistics that delivers benefits through better, quicker decisions by governments, companies, charities and individuals.”28 That sounds plausible, and there are some surprising examples. London’s Metropolitan Police, for example, used the census to identify streets with large numbers of elderly residents, and focused efforts on preventing fraudsters and burglars from preying on vulnerable people. Everything from public health campaigns to nuclear disaster contingency plans depends on figuring out where everyone lives.
Disappointingly, the cost-benefit analysts shrugged their shoulders and declared themselves unable to put a value on all this, except to declare that it was obviously jolly useful. Still, they did find some benefits they judged to be quantifiable, and they pegged those as being worth £500 million a year—a bit less than £10 per UK resident. Since the census itself cost less than £500 million and lasts ten years, that suggests a tenfold return is a rather conservative estimate of the benefits.
Another attempt to tot up the value of official statistics was made in New Zealand, where the census, which cost NZ$200 million to conduct (about $120 million), was reckoned to have produced a benefit of at least a billion New Zealand dollars—a fivefold return. The study reckoned that refreshing the basic knowledge provided by the census—who lives where—produced a more accurate allocation of public spending on facilities such as hospitals and roads, and better-informed policy more generally.29 Back in Puerto Rico, researchers pointed out ways in which PRIS had paid for itself, such as enabling the introduction of new systems to prevent fraud in collecting Medicare payments.30
But perhaps the strongest evidence that statistics are worthwhile is how cheap they are to collect, relative to the value of the decisions they inform. Consider the CBO: it advises Congress on $4 trillion worth of annual spending, on a budget of just $50 million a year. To put it another way, for every $80,000 the US government spends, one dollar funds the CBO to shed light on the other $79,999.31 To justify its existence, the CBO would need to improve the effectiveness of government spending decisions by a mere 0.00125 percent. It’s hard to imagine how the CBO could fail to clear that bar.
Likewise, the million-dollar budget of PRIS sounds a lot more modest when you put it in the context of the Puerto Rican government’s overall spending, which at nearly $10 billion is about 10,000 times larger. The UK’s Office for National Statistics costs about £250 million a year—less than one pound for every £3,000 the UK government spends. Between them, the thirteen principal statistical agencies in the United States cost one dollar for every $2,000 the federal government spends.32 If serious, independently gathered data improve government decision-making even by a tiny fraction, then these agencies are well worth the small sliver of public spending that is devoted to them.
Without statistics, then, governments would fumble in ignorance. But there is an intriguing counterargument, which is that governments are so reliably incompetent that giving them more information is risky; it will only encourage them.
One prominent advocate of this view was Sir John Cowperthwaite. Sir John was the financial secretary of Hong Kong throughout the 1960s, at a time when it was still under the control of the British—and when it was experiencing scorchingly rapid economic growth. Exactly how rapid was hard to say, because Sir John refused to collect basic information about Hong Kong’s economy. The economist Milton Friedman, later to win the Nobel Memorial Prize in Economics, met Sir John at the time and asked him why. “Cowperthwaite explained that he had resisted requests from civil servants to provide such data because he was convinced that once the data was published there would be pressure to use them for government intervention in the economy.”33
There was a logic to this. Hong Kong’s rapid growth was partly thanks to an influx of immigrants from famine-struck communist China, but Cowperthwaite and Friedman also believed—with some reason—that it was flourishing thanks to a laissez-faire approach to policy. Cowperthwaite’s government levied low taxes and provided very little in the way of public services. The private sector, he argued, would tend to solve people’s problems more quickly and efficiently than the state. Why, then, collect data that would only encourage meddling from the authorities back in London? Cowperthwaite figured that the less London’s politicians did, the better—and the less they knew, the less they would try to do.
Similarly, in his magisterial book Seeing Like a State, James C. Scott argues that the statistical information that states gather is flawed, missing the local details that matter. Imagine, say, a rural community in Southeast Asia with complex customs regarding a piece of local land. Every household has some rights to farm the land, in rough proportion to its number of able-bodied members; then after each harvest, it becomes common land for grazing. Everyone can gather firewood, too, but the village baker and blacksmith are allowed to gather more. A surveyor from the new national land registry turns up, asking, “Who owns this land?” Well—it’s not so simple.
Now, it’s one thing to be wrong, or to have a view of the world that misses out on something important. But, argues Scott, because the state is powerful, its misperceptions of the world often take physical form, producing well-meaning but clumsy and oppressive modernist schemes that ignore local knowledge and stifle local autonomy.34 Perhaps our frustrated land registry surveyor decides to write on her clipboard that the local government owns the land; then a few years later the villagers are surprised to find the land being cleared for a palm oil plantation.
One can take the argument even further: that governments can be utterly malevolent, and the worst cases are so catastrophic that they should inform our thinking about how much data any government should have. Wouldn’t it have been better if Hitler, Mao, and Stalin had understood less about their own societies? Might they have done less harm? And is it reasonable to worry that the more governments know about us, the more they will be tempted to exert control over us?
This argument seems plausible, but I’m not convinced. From communist East Germany to modern-day China, governments interested in mass surveillance and population control have tended to use very different methods from those deployed by independent statistical offices in modern democracies, and to collect very different kinds of data. And history suggests that dictators often have either little interest in the collection of solid statistics or little ability to collect them.
Consider the disastrous government-induced famine of the late 1950s caused by the Great Leap Forward in communist China, in which people were reduced to eating tree bark, bird droppings, and rats. Between 20 and 40 million people died. The catastrophe was made worse by a lack of accurate data about agricultural production. When official statistics began to make the death toll apparent, they were destroyed.35
Stalin, similarly, suppressed the publication of the 1937 census of the Soviet Union when it showed that the population was lower than he’d previously announced. This contradiction was an affront in its own right, but it also highlighted the millions of deaths as a result, directly and indirectly, of Stalin’s brutality. The penalty for accurately counting the Soviet population? Olimpiy Kvitkin, the statistician in charge, was arrested and shot. Several of his colleagues met the same fate.36 This is not the act of a totalitarian leader who finds accurate statistical information to be an indispensable tool of oppression.37
In Nazi Germany, there was no lack of ambition to use data to support the apparatus of the state. The Reich tried to use punch-card machines, the latest technology, in an effort to track the entire population. But as Adam Tooze argues in Statistics and the German State, statistical standards actually fell apart under the Nazis: “No workable system was ever devised.”38 The traditions of official statistics—privacy, confidentiality, and independence—were so alien to the Nazi project that the system all but collapsed under the political pressure and factional infighting.
All that said, I have a great deal of sympathy with James C. Scott’s argument (I discuss Scott’s ideas in more detail in my book Messy) and some sympathy with Sir John Cowperthwaite’s. States should be humble. Bureaucrats must recognize the limits of their knowledge. There is always a risk that the bird’s-eye view is so grand and sweeping as to induce delusions of omnipotence.
Sir John’s strategy to deny information to the British government seems to have worked for Hong Kong fifty years ago, but Hong Kong was in a very particular situation—a colonial possession of a fading imperial power in which big government was fashionable, and any government intervention would have taken place at a distance of 6,000 miles. Those are unusual circumstances.
But the tactic of simply refusing to collect basic statistics could only make sense for a libertarian, laissez-faire regime. And the truth is that very few people seem attracted by that prospect. For better or worse, we want our governments to take action, and if they are to take action they need information. Statistics collected by the state make for better-informed policies—on crime, education, infrastructure, and much else.
In poor countries, where official statistical agencies tend to be less well resourced, there is especially wide scope to improve decision making through better statistics. One example may illustrate the problem. How effective is education in improving literacy? That seems like the kind of question that might usefully help to inform education policy and spending. So researchers at the World Bank looked into statistics collated by UNESCO (the UN Educational, Scientific and Cultural Organization) and found there was an amazingly high correlation between education and literacy: without fail, countries that provided more years of formal education to more people had higher literacy rates. Clearly, education worked! They excitedly published their findings.39
Unfortunately, they hadn’t read the small print. UNESCO simply hadn’t had the resources to collect all the data they wanted to: they had just seventy staff covering 220 countries trying to pull together data in all kinds of areas—adult literacy was just one. (What does literacy even mean in a place such as Papua New Guinea? It has four hundred languages, some of which have no written form.) Inevitably, there would be shortcuts. UNESCO couldn’t send teams of people to assess rates of adult literacy themselves, so they looked for a proxy indicator—a best guess in a difficult situation. And they decided that if someone had fewer than five years’ formal education, they would be assumed to be illiterate. No wonder the World Bank researchers found such a close correlation between education and literacy.
If organizations like UNESCO had more resources to collect statistics, they would have less need to rely on proxies, and researchers would have greater ability to answer questions such as how well education promotes literacy. Statistical bedrock is so patchy in poor countries that already one dollar in every three hundred that is spent on international aid goes toward funding statistics. There is a case that doubling that might well produce much more value from the remaining $298.40
Sir John’s comment to Milton Friedman contains an implicit assumption: that government statistics are not just collected by government, but they are collected for government. He was unusual in believing that government would do a better job without those statistics, but otherwise that perspective is common. Congress seemed to have the same idea in mind when creating the Congressional Budget Office: the CBO was designed to provide information to Congress. The clue is in the name. And the idea goes back a long way. As the future president James Madison put it in 1790, politicians should be willing to commission accurate statistics, “in order that they might rest their arguments on facts, instead of assertions and conjectures.”41
There is nothing wrong with the idea that government should collect statistics to inform itself. But there is a risk that this view slips into a proprietorial sense of ownership, when politicians believe not only that they should be using statistics to run the country, but that those statistics are none of anyone else’s business, and that external scrutiny is a distraction. The facts are no longer the facts—they become the tools of the powerful.
Sir Derek Rayner was a proud proponent of the view that statistics should be managerial tools.42 He had already been a highly successful manager at Marks & Spencer, national treasure of the British high street, before advising the UK government on how to become more efficient. In 1980, Prime Minister Margaret Thatcher asked him to review the way official statistics were collected and published in the UK. Sir Derek was happy to oblige: he saw these numbers as basically a management information system. Those that helped the government run the country could be retained; those that did not could be discarded. And there was no need to make a big fuss about publishing the numbers so that anyone could learn from them, or challenge them.
Sir Derek’s view was a mistake. Good statistics don’t just serve government planners; they are valuable to a far wider group of people. In the commercial sector, businesses rely on government-collected data to plan their production targets; the location of factories, offices, and stores; and other business activities. Data gathered by the Bureau of Labor Statistics, the Census Bureau, the Energy Information Administration, and the Bureau of Economic Analysis allow banks, real estate agents, insurance companies, auto manufacturers, construction firms, retailers, and many other businesses to make plans and to assess their own data against a broader backdrop. The multibillion-dollar turnover of data-intensive private sector companies such as Bloomberg, Reuters, Zillow, Nielsen, and IHS Markit suggests that businesses are willing to pay handsomely for useful statistics; what is less well understood is that these businesses build their statistical edifices on the foundations of government data.43
This isn’t just about making money; it’s about making sure that citizens have access to accurate information about the world in which they live. Government statistical agencies typically make their work available to all, free of charge. Some of that data might be impossible for a private agency to collect, at any price: governments can legally require a response that a private agency cannot, as with the case of the census. Other data could be collected but they would be offered only on an expensive subscription basis—private providers can charge tens of thousands of dollars a year for people who want data at their fingertips. Of course some data might be gathered by private firms and given away without charge, but such statistics are often just ads in the guise of information.
Publicly available statistics can be used to understand and illuminate pressing social issues. To pick just one example, W. E. B. Du Bois—historian, sociologist, and civil rights campaigner—led a remarkable data visualization effort at the end of the nineteenth century as part of the Paris Exposition of 1900.44 His team produced beautiful, modernist graphs showing the situation of African Americans in the United States at the time, with data on demographics, wealth, inequality, and more besides. Some of them used data that Du Bois and his team had gathered at Atlanta University, but some of the most striking graphics relied on official statistical sources such as the US Census. It’s just one example of the way in which those who want to understand the world or campaign for change, or both, can turn to official statistics to help them.
With reliable statistics, citizens can hold their governments to account and those governments can make better decisions. If the government decides instead that the statistics belong to politicians, not to citizens, the quality of government decisions will not improve as a result. Neither will the esteem in which government is held.
Sir Derek Rayner’s ideas appalled many statisticians. The problem was partly the corrosive message to the British public: “These numbers aren’t for you—they’re only for important people.” But even if, like Sir Derek, one believes that statistics really are just for the important people, there’s still a good reason to make them publicly available: doing so keeps them honest. As we saw in the previous chapter, public scrutiny is vital. It’s what distinguishes science from alchemy. If statistics are published and designed to be accessible to all, they can be analyzed and examined by academics, policy wonks, and indeed anybody with a bit of time and access to a computer. Errors can be identified and corrected.
As it was, Sir Derek’s proposed reforms led to a situation where the definition of unemployment was tweaked more than thirty times in a decade, generally in a way as to lower the headline unemployment rate.45 That is what happens when statistics are no longer regarded as a public good. And unsurprisingly, people became extremely cynical about the quality of these statistics. “Phony,” as Donald Trump might have said. Of course when official data keeps being tweaked for propaganda reasons, trust will rightly evaporate.
The UK’s statistical system, now reformed, has spent a quarter of a century trying to recover its reputation. That has taken time and hard work, because trust is easy to throw away and hard to regain. Still, the UK’s Office for National Statistics is more trusted than comparable organizations such as the Bank of England, the courts, the police, and the civil service—and vastly more trusted than politicians or the media.46
Sir Derek’s view—that government-collected statistics exist mainly for the convenience of government administrators, and that citizens have no particular right to see them—has thankfully fallen out of fashion around most established democracies. But one vestige clearly remains, and this was unwittingly highlighted by President Trump on Friday, June 1, 2018—the day on which the monthly jobs report was to be published.
“Looking forward to seeing the employment numbers at 8:30 this morning,” Trump tweeted, at 7:21 a.m., in an uncanny demonstration of how to wink on Twitter. Markets leaped in expectation of good news. Sixty-nine minutes later the jobs report was released, and—surprise, surprise—the news was indeed good.
Was Trump clairvoyant? No. He had simply been given advanced sight of the job numbers and decided to tell the world to expect good news.
Official statistics are often both politically and financially sensitive—for example, if the latest numbers on unemployment show that lots of jobs have been created, financial markets will respond in a different way than if the report looks grim. The numbers sometimes shape political arguments, too. For this reason, official statistics are kept confidential as they are being calculated and checked; they are then released at a particular moment, on the dot.
But in some countries, including the United States and the UK, certain people get to see certain official statistics in advance. This is called “pre-release access” and it’s a controversial practice. The justification for it is to allow ministers to prepare a response, to answer questions from journalists, and so on. For this reason various political advisers, press officers, and the like are often on the list of people given this privileged access. A self-congratulatory review of the practice by the Cabinet Office in the UK noted that press officers thought that ending the practice of pre-release “would be a disaster . . . The media would simply have their stories without any proper, official comment.” Boo hoo.47
It’s clear why politicians in power might find it convenient to get advance notice of statistics so they can plan to crow about them if they’re good—or if they’re bad, to get their story straight or create a distraction. But it’s far from clear that this is in the public interest. Why shouldn’t everyone, on all sides of the debate, get access to the numbers at the same time, once they’re ready?
(There is a compromise position: ministers could receive the statistics thirty minutes in advance and sit alone, without access to a cell phone, to compose a response. Quite apart from being pleasingly like sending powerful people back to sit exams, this is how journalists are sometimes given sensitive official releases. We cope. I was told a story about a Canadian statistician explaining this approach at an international gathering of colleagues. Her Russian counterpart chimed in with a question: “How does that approach work if the minister wishes to change the statistics?” Exactly.)
There’s more at stake here than a sense of fair play. In the UK, where a number of officials and advisers have routinely had pre-release access to the unemployment statistics, market watchers noticed something strange: key financial market prices such as foreign exchange rates and the price of government bonds would sometimes move sharply not long before the numbers were published. Most of the time this happened, the data would be surprising—either much better or much worse than the market had expected—and the trading would be in the direction that took advantage of the surprise.
Just to check that the market wasn’t somehow figuring out the same thing that the statisticians had, forty-five minutes in advance of publication, economist Alexander Kurov made a systematic comparison of the situation in the UK and the situation in Sweden—which is economically quite similar to the UK but which bans pre-release access to official statistics. Swedish politicians and their press officers learn about the numbers at the same time as everyone else—and traders of the Swedish krona, it seems, do not have the same weird powers of clairvoyance as traders of the British pound.48
It’s impossible to prove, but it seems highly likely that someone with pre-release access was giving the nod to his or her trader friends, allowing insider trading on official data. Who? Well, there were 118 people with pre-release access to the unemployment statistics, which doesn’t make it easy to identify a specific culprit. (If you are wondering why it took 118 people to prepare “proper, official comment” for the media, so am I.)
Trump’s tweet probably didn’t do much harm in itself: after all, everyone had access to the tweet at the same time. Indeed the president may unwittingly have done some good, by turning the hidden scandal of pre-release access—and the way it is an invitation to corruption, at least when the data go to subtler operators than Trump—into a widely discussed blunder.
Such privileged access facilitates insider trading—but perhaps more important is that it is corrosive of trust in official statistics. The UK press officers, keen to retain the insider perk, protested that if ministers weren’t able instantly to offer some polished patter about the data, trust in statistics would be damaged. But the truth is that the countries that are most scrupulous about forbidding pre-release access are also the countries with the strongest public confidence in official data. Those press officers might be surprised at that. I’m not.
Thankfully, the data detectives were at hand to lead the charge. In the UK, the Royal Statistical Society campaigned hard against the practice of letting ministers and other insiders sneak a peek at valuable data before the rest of us. The idea that the government needed to see the numbers so as to compose a press release, said the RSS, “is pernicious. It skews debate over the figures and perpetuates the impression that ministers control the data.” That seems right to me. In the UK, our levels of trust in official statistics are not as high as in some countries, and not as high as they should be—but they are still far higher than our levels of trust in politicians. I can see why politicians would like to get themselves wrapped up in the release of trusted statistics; it’s far from clear why any of the rest of us should want that.
So I’m delighted to report that as of July 1, 2019, the UK decided to emulate the Swedes and end pre-release access to official statistics. Under the new system the only people who will know these numbers before they’re published will be the statisticians working on them. Something tells me that trust in official statistics will survive the shock of ministers knowing the facts at the same time as everyone else.
This chapter has been a wholehearted defense of my fellow nerds, the ones who do an essential job in government, sometimes facing indifference from voters, interference from the powerful, and skepticism from all sides.
I wouldn’t want to suggest that any nation’s official statistical machinery is by definition unimpeachable. We’ve seen that the official statistics emerging from Argentina and Greece turned out to be deceptive, that the unemployment data emerging from the UK were tweaked every few months throughout the 1980s, and that in Canada statisticians have been forced to resign in protest at the decisions politicians have made. Some statisticians have had to endure death threats made against their families; others openly acknowledge that ministers can change data if they wish. It would be naive to assume that such problems are always exposed and that the truth always triumphs.
Even when official statistics are produced as skillfully and independently as we’d hope, they will never be perfect. Some things we care about are simply hard to measure, such as domestic violence, tax evasion, and rough-sleeping. There is, no doubt, plenty of scope for official statisticians to make the data they collect more representative, more relevant, easier to reconcile with everyday experience, and fully transparent. The more they are able to do this, the more they will deserve our trust.
Yet for all their problems and weaknesses, official statistics are still the closest we have to data bedrock. When a country picks and defends a team of skilled, professional, and independent statisticians, the facts have a way of making themselves known. When a country’s national statistics fall short, an international community of statisticians will complain. When an independent statistician is attacked or threatened by politicians, that same community will rally to his or her defense. Statisticians are capable of greater courage than most of us appreciate. Their independence is not something to take for granted, or to casually undermine.
As citizens, we need to look for that statistical bedrock. If we want to understand the situation a country is in—whether to inform our own decisions, or to hold our government to account—then we will usually start with the statistics and the analysis produced by organizations such as the Office for National Statistics, Eurostat, Statistics Canada, the Bureau of Labor Statistics, and the Congressional Budget Office.
Tough, independent-minded statistical agencies make us all smarter. So: be grateful to Andreas Georgiou, Graciela Bevacqua, and for that matter the late Alice Rivlin. And, if you like, raise a glass to Fanne Foxe.