This week we might bust an entire nation for handing over dodgy economic statistics. But first: why would they bother? Well, it turns out that whole countries have an interest in distorting their accounts, just like companies and individuals. If you’re a Euro member like Greece, for example, you have to comply with various economic criteria, and there’s the risk of sanctions if you miss them.
Government figures are subjected to various forms of audit already, of course, but alongside checking that things marry up with each other, forensic statisticians also have a few interesting tricks to try to spot suspicious patterns in the raw numbers, and so estimate the chances that figures from a set of accounts have been tampered with. One of the cleverest tools is something called Benford’s law.
Imagine you have the data on, say, the population of every country in the world. Now, take only the ‘leading digit’ from each number: the first number in the number, if you like. For the UK population, which was 61,838,154 in 2009, that leading digit would be six. Andorra’s was 85,168, so that’s eight. And so on.
If you take all those leading digits, from all the countries, then overall you might naïvely expect to see the same number of ones, fours, nines and so on. But in fact, for naturally occurring data, you get more ones than twos, more twos than threes, and so on, all the way down to nine. This is Benford’s law: the distribution of leading digits follows a logarithmic distribution, so you get a ‘one’ most commonly, appearing as first digit around 30 per cent of the time, and a nine as first digit only 5 per cent of the time.
The next time you’re waiting for a bus, you can think about why this happens (bear in mind what leading digits do when quantities repeatedly double, perhaps). Reality agrees with this theory pretty neatly, and if you go to the website testingbenfordslaw.com you’ll see the proportions of each leading digit from lots of real-world datasets, graphed alongside what Benford’s law predicts they should be, with data ranging from Twitter users’ follower counts to the number of books in different libraries across the US.
Benford’s law doesn’t work perfectly: it only works when you’re examining groups of numbers that span several orders of magnitude. So, for example, for the age in years of the graduate working population, which goes from around twenty to seventy, it wouldn’t be much good; but for personal savings, from nothing to millions, it should work fine. And of course Benford’s law works in other counting systems, so if three-fingered sloths ever developed numeracy, and counted in base 6, or maybe base 12, the law would still hold.
This property of naturally occurring data has been used to check for dubious behaviour in figures for four decades now: it was first used on socioeconomic data submitted to support planning applications, and then on company accounts; it’s even admissible in US courts. In 2009 an economist from the Bundesbank suggested using Benford’s law on countries’ economic data, and last month the results were published (hat-tip to Tim Harford for the paper).
Researchers took macroeconomic data on all twenty-seven EU nations, looking specifically at the accounting data that countries have to hand over for monitoring, which is all posted for free at the online repository Eurostat: things like government deficit, debt, revenue, expenditure, and so on. Then they took just the first digits from all the numbers, and checked to see if they deviated from what you would predict using Benford’s law.
The results were fun. Greece – whose economy has tanked – showed the largest and most suspicious deviation from Benford’s law of any country in the Euro.
This isn’t a massive surprise: the EU has run several investigations into Greece’s numbers already, and the ones from 2005 to 2008 were repeatedly revised upwards after the fact. But it’s neat, and if you wanted to while away a very nerdy afternoon, could even download the data, for free from Eurostat, and repeat the analysis for yourself. Joy!