Good charts are useful because they untangle the complexity of numbers, making them more concrete and tangible. However, charts can also lead us to spot patterns and trends that are dubious, spurious, or misleading, particularly when we pair them with the human brain’s tendency to read too much into what we see and to always try to confirm what we already believe.
The eminent statistician John W. Tukey once wrote that “the greatest value of a picture is when it forces us to notice what we never expected to see.”1 Good charts reveal realities that may otherwise go unnoticed.
However, charts can also trick us into perceiving features that are meaningless or misleading. Did you know, for example, that the more cigarettes people consume, the more they live? This runs contrary to decades of evidence of the dangers of tobacco consumption—particularly cigarettes—but below you have a chart based on publicly available data from the World Health Organization and the United Nations.2
If I were a cigarette smoker, I’d likely feel reassured by this chart. Tobacco doesn’t reduce life expectancy! As unlikely as it sounds, the opposite might be true! My interpretation of this chart, though, is an example of several common challenges in chart reading: the relationship between correlation and causation, the role of amalgamation paradoxes, and the ecological fallacy. Let’s examine them one by one.3
There’s nothing wrong with the data plotted here, but my description of the chart (“The more cigarettes we consume, the more we live”) is wrong. Describing the content of a chart correctly is critical. All this chart shows is that, at least at a country-by-country level, there’s a positive association between cigarette consumption and life expectancy, and vice versa. But that doesn’t mean that cigarettes increase life expectancy. Based on this example and others we’ll soon see, we can enunciate a core rule of chart reading:
A chart shows only what it shows, and nothing else.
As I explained in chapter 1, “Correlation doesn’t equal causation” is an old phrase repeated in elementary statistics courses. Correlation is usually one of the first clues to later finding causal links between phenomena, but the classic statistics saying has undeniable wisdom. The saying applies to this case, as there could be other factors that I’m not contemplating and that may influence both cigarette consumption and life expectancy. For instance, take wealth: People in wealthier countries tend to live longer because they usually have access to better diets and health care and are less likely to be the victims of violence and wars. Plus, they can afford to buy more cigarettes. Wealth could be a confounding factor in my chart.
The second and third challenges I mentioned before—amalgamation paradoxes and the ecological fallacy—are related. The ecological fallacy consists of trying to learn something about individuals based on characteristics of the groups they belong to. We saw it already when I mentioned that, despite being born in Spain, I’m far from being your average, stereotypical Spanish male.
The fact that people in one country smoke a lot and also live long doesn’t mean that you or I can smoke a lot and also live long. Different levels of analysis—the individual versus the group—may require different data sets. If my data is created and summarized to study a group—countries, in this case—its utility will be very limited if what we need to study is either smaller groups, regions or cities of those countries, or individuals living in those places.
This is where amalgamation paradoxes come in. They are based on the fact that certain patterns or trends often disappear or even reverse depending on how we aggregate or subset the data.3
Considering that wealth may be a confounding factor in my first chart, let’s draw it again with colors for different high-, middle-, and low-income groups:
The chart looks quite messy, as countries of different income groups overlap too much, so let’s split up the countries by income:
The strong positive association between cigarette consumption and life expectancy doesn’t look that strong anymore, does it? Poor nations have a high variability in life expectancies (vertical axis) but, on average, they don’t smoke that much. Middle-income countries have a large variation in both life expectancy and cigarette consumption, and the relationship between those variables is weak. High-income nations tend to have high life expectancies overall (they’re high on the vertical axis), but cigarette consumption (horizontal axis) is all over the place: cigarette consumption is high in some countries and low in others.
The picture becomes even more muddled if we subdivide the data further, by geographic region. Now the formerly strong positive association between smoking and life expectancy seems very weak, if not outright nonexistent:
The association would vanish even further if we could split each one of those countries into its constituent parts—regions, provinces, cities, and neighborhoods—down to the individual. With each further split, the relationship between cigarette consumption and life expectancy will decrease to the point that it becomes negative: when we observe individuals, we notice that cigarette consumption has a negative effect. The following chart, based on different sources,4 compares survival rates of people older than 40. Notice that around 50% of people who have never smoked or who stopped smoking several years ago were still alive by 80, but just a bit more than 25% of cigarette smokers survived to that point. According to several studies, cigarette smoking shortens life by around seven years (this kind of chart of survival time is called a Kaplan-Meier plot):
Paradoxes due to the different levels at which data can be aggregated are abundant, and they can push us into faulty reasoning. Several blog posts in the website Why Evolution Is True, by biology professor Jerry Coyne, author of an excellent book with the same title, discuss the inverse relationship that exists between religiosity on one hand and happiness and other indicators of well-being on the other.5
Here are two maps and a scatter plot that summarize the association between the percentage of people in each country who say that religion is important in their lives (according to a 2009 Gallup poll) and those countries’ scores in the Happiness Index (a measure calculated by the United Nations for its World Happiness Report):
The association between the two variables is relatively weak and negative: in general, the more religious a country is, the less happy its people tend to be. The association is noticeable, even if there are plenty of exceptions. For instance, Ukraine isn’t very religious, but its happiness score is low, and Costa Rica is very happy, while being very religious.
Happiness scores are positively related to equality and well-being. Countries with higher equality, as well as with well-fed and healthy inhabitants, tend to be happier. Equality and happiness are positively correlated, while equality and happiness are both inversely correlated to religiosity: the more inequality, the less happy a country tends to be and the higher the percentage of people who say that religion is important in their lives.
The inverse relationship between religiosity and indicators of happiness and well-being remains even if we disaggregate the data a bit to look at the regional level. Data from Gallup allows us to compare the percentage of people in the United States who say they are very religious with the overall well-being and life-satisfaction score of their state, a metric based on weighing factors such as access to affordable health insurance, quality of diet, amount of exercise, sense of community, and civic engagement.6 (See the chart on the next page.) As is usual in any scatter plot, there are exceptions: West Virginia is very low in well-being and in the middle of the distribution in terms of religiosity, and Utah is high in both measures.
An enthusiastic atheist may be too quick to extract lessons from these charts. Do they mean that religiosity leads to more misery or is the opposite true? Moreover, do the charts suggest that I as an individual will be happier if I abandon my religion or even become an atheist? Of course not. Let’s emphasize another rule of good chart reading:
Don’t read too much into a chart—particularly if you’re reading what you’d like to read.
First of all, these charts tell you that higher levels of religious commitment are inversely related to happiness and well-being, but they don’t say that increasing religiosity will lead to more misery. In reality, this might be a case of reverse causality. It might be that less suffering makes countries less religious.
A study by University of Iowa professor Frederick Solt showed that year-by-year changes in inequality in different countries led to variations in religiosity, independently from how wealthy each person was in those countries. Both poor and rich people became more religious when inequality increased.7 The rich and powerful become more religious, according to Solt, because religion can be used to justify societal hierarchies; to the poor, on the other hand, religion provides comfort and a sense of belonging.
This helps explain why the relationship between religiosity and happiness or perceived well-being reverses and becomes positive when we disaggregate the data even further to the individual level. This is particularly true in unstable and unequal societies, where religious people have a higher sense of well-being.8
Let’s think of an extreme case: If you live in a poor country ravaged by war and institutional collapse, organized religion may be a strong source of meaning, consolation, community, and stability that you can latch on to. You can’t compare yourself to the average Norwegian or Finnish person—who may be very happy but not very religious—and say that forfeiting religion will make you happier. Your living conditions are too different. For individuals living in rich, egalitarian, and safe places, being religious or not may not have a noticeable effect on happiness, as their societies provide health care, good education, safety, and a sense of belonging. But religion may make a huge difference for people like you. On average, being poor and religious in an unstable place may be better than being poor and nonreligious.9
Allow me, then, to reiterate another rule of chart reading based on the examples I’ve discussed so far in this chapter:
Different levels of thinking may require different levels of data aggregation.
In other words, if our goal is to learn something about the relationship between religiosity and happiness in different countries or regions, the chart should compare aggregated data of those countries or regions. If the goal is to learn about individuals, country-level or regional-level charts are inappropriate; instead, charts should compare people to each other.
Jumping to conclusions too quickly after we see a chart that corroborates what we already think is a universal malady that can infect anyone. After every presidential election, friends of mine who are on the left end of the political spectrum usually wonder why it often is that in poorer regions that rely more on the safety net, people tend to vote more for candidates who promise to undermine that safety net.
We could call this the “What’s the Matter with Kansas?” paradox, after the title of a popular 2004 book by journalist and historian Thomas Frank. The main thesis of the book is that some voters support candidates who go against their interests, because they agree with those candidates on cultural values such as religion, abortion, gay rights, political correctness, and others. My friends are dumbfounded by charts like these:
These charts seem to confirm Frank’s thesis: the poorer a county is (the higher its red dot is on the chart), the more its Democratic vote decreased in 2016 in comparison to 2012 (the farther to the left its red dot is).
The pattern is real, but does it really tell us that poor people in states such as West Virginia or Tennessee are “voting against their interest”? Maybe not. To begin with, the accusation is simplistic. When we vote, we don’t base our decision only on our own economic interests. I’ve repeatedly voted for candidates who proposed to increase taxes for families like mine. Also, voters care about candidates’ values. I’d never vote for anyone who hints at anti-immigrant animus or xenophobia, no matter how much I agree with her or him on what to do with the economy.
But let’s stick to the charts and imagine that economic self-interest is the only factor that voters should take into account. The charts don’t become better, because what they reveal is not that poor people moved away from the Democratic party. They show that poorer counties did, which is different. Turnout in U.S. elections is usually low, and it becomes lower when you descend the economic ladder. As Alec MacGillis, a ProPublica reporter specializing in government, has written:
The people who most rely on the safety-net programs secured by Democrats are, by and large, not voting against their own interests by electing Republicans. Rather, they are not voting, period. . . . The people in these communities who are voting Republican in larger proportions are those who are a notch or two up the economic ladder—the sheriff’s deputy, the teacher, the highway worker, the motel clerk, the gas station owner and the coal miner. And their growing allegiance to the Republicans is, in part, a reaction against what they perceive, among those below them on the economic ladder, as a growing dependency on the safety net, the most visible manifestation of downward mobility in their declining towns.10
Thinking about differences between aggregated data and individual data is crucial to understanding how charts may bias our perceptions. Take a look at the pattern revealed by the following chart, from the website Our World in Data, which is a treasure trove if you enjoy visuals:11
This is a connected scatter plot. We learned how to read it in chapter 2, but here’s a refresher: lines correspond to countries, and you should imagine that they are like snail trails moving from left to right and from the bottom up. Focus on the United States’ line. The position of the initial point, on the left, corresponds to life expectancy (vertical axis) and health expenditure per person in adjusted dollars (horizontal axis) in 1970. The end point of the U.S. line, on the right, corresponds to the same variables in 2015. This point is higher and farther to the right than the initial one, meaning that both life expectancy and health expenditures were higher in 2015 than in 1970.
What the chart reveals is that in most countries, life expectancy and health expenditures between 1970 and 2015 increased at similar rates. The exception is the United States, where life expectancy didn’t increase much but average health expenditure per person soared. I can use this chart to propose another rule of good chart reading:
Any chart is a simplification of reality, and it reveals as much as it hides.
Therefore, it’s always worth asking ourselves: What other patterns or trends may be hidden behind the data displayed on the chart? We could think of the variation around those national trends. Health expenditure in the United States varies enormously depending on your wealth and where you live, as does life expectancy. A 2017 study by researchers from the University of Washington found that “while residents of certain affluent counties in central Colorado had the highest life expectancy at 87 years [much more than the average Swiss or German], people in several counties of North and South Dakota, typically those with Native American reservations, could expect to die far younger, at only 66.” That’s a difference of more than 20 years.12 My guess is that the variability of health expenditures and life expectancy isn’t nearly as wide in wealthy nations with universal health care systems.
On March 23, 2010, President Barack Obama signed the Affordable Care Act (ACA, also known as Obamacare) into law. The ACA has been the topic of heated debates since it was proposed and up to the time of my writing in the summer of 2018. Questions include: Is it good for the economy? Is it truly affordable? Can it survive administrations that try to undermine it? Does it boost employment or does it inhibit employers from hiring?
The answers are still being debated, but some pundits have used charts like the following to argue that, contrary to what Republicans have always claimed, the ACA is really good for the job market. Notice that the number of jobs declined during the economic crisis but started recovering around 2010. Then look at what happened close to the turning point of the chart:
When anyone intends to persuade us with a chart, it’s worth asking:
Are the patterns or trends on this chart, on their own, sufficient to support the claims the author makes?
I don’t think that’s the case here. The first reason is that, as we have just learned, a chart shows only what it shows, and nothing else. All this chart shows is that there are two events that happened at about the same point in time: the ACA being signed into law and the turning point of the jobs curve. But the chart doesn’t say that one event caused or even influenced the other. It’s your brain that makes that inference.
The second reason is that we could think of other events that also happened around those months and that could have influenced the recovery of the job market. Obama’s stimulus package—the American Recovery and Reinvestment Act—was signed into law in February 2009, in response to the financial crisis of 2007–2008. It could well be that the billions of dollars injected into the economy kicked in months later, pushing companies to start hiring again.
We could also think of counterfactuals. Imagine that the Affordable Care Act had died in Congress. How would the private jobs curve change under that hypothetical? Would it be the same? Would the recovery be slower (because the ACA makes creating jobs easier) or faster (because the ACA hinders hiring, by making companies wary of health care costs)?
We just don’t know. The original chart tells us nothing about whether the ACA had any influence on the job market. Alone, the chart is useless for either defending or attacking the program.
I’ve seen similar charts being mishandled by people on the right. In his first years in office, Donald Trump liked to claim that the job market was a “disaster” before he was sworn in but recovered right after, and he used charts that cropped the horizontal axis in a convenient place:
But if we go back in time and mark the point when Trump became president, we’ll see that there’s no remarkable change in the trajectory and slope of the line. Jobs began recovering in 2010. What Trump could have taken credit for instead was continuing an existing trend:
In October 2017, Trump boasted about the Dow Jones in a tweet, writing simply “WOW!” as a comment to this image revealing that the stock market was flat before Election Day in November 2016 and picked up right after:
It’s easy to guess what the error is: The Dow Jones follows a pattern similar to employment. It has increased quite steadily since 2009. There have been some plateaus and bumps, including the “Trump Bump” after the 2016 inauguration,13 but the line hasn’t changed direction:
The more we cherish an idea, the more we’ll love any chart that corroborates it, no matter how simplistic that chart is. The first chart below has become quite popular in creationist circles, as it reveals that there was an abrupt increase in the diversity of animal genera around a period popularly known as the Cambrian explosion. (Genera are groups of species; for instance, the genus Canis contains wolves, jackals, dogs, and others.) The chart is usually paired for comparison with an idealized Darwinian “tree of life,” which is how evolution is supposed to work, branching out new genera steadily, little by little:
The pattern displayed on the first chart is one of sudden appearance of new kinds of animals during the Cambrian period. The Cambrian “explosion” was a mystery to biologists for more than a century—Darwin himself expressed his bafflement about it in his On the Origin of Species—as the incompleteness of the fossil record, particularly from Precambrian times, reinforced the idea of a rapid diversification of life. Creationists have claimed that “in a moment of geological time complex animals first appeared on earth fully formed, without evidence of any evolutionary ancestors. This remarkable explosion of life . . . is best explained by special creation by a Designer.”14
However, the term “explosion” and the chart creationists tout are misleading. Modern scientists, benefiting from a much more complete fossil record in comparison to Darwin’s time, favor the term “Cambrian diversification”: Many new genera indeed appeared during the Cambrian period, but their arising was far from abrupt. The Cambrian lasted for more than 50 million years, from 545 to 490 million years ago. That’s an enormous amount of time for an explosion.
Being aware of this inconvenient reality, some creationist authors such as Stephen C. Meyer stick to their chart, but they narrow the “explosion” to the third stage of the Cambrian period, the Atdabanian; occurring between 521 and 514 million years ago, this stage was when a larger diversification of genera happened. Meyer has said that “new information can only come from intelligence, and so the burst of genetic information during the Cambrian era provides compelling evidence that animal life is the product of intelligent design rather than a blind undirected process like natural selection.”15
Seven million years is still a lot of time for a “burst”—our own species has been around for just 300,000 years—but this isn’t the only problem. Donald R. Prothero, a paleontologist from Occidental College and author of the book Evolution: What the Fossils Say and Why It Matters, favors a much more detailed chart of the Precambrian and Cambrian periods (below) and explains that
the entire diversification of life is now known to have gone through a number of distinct steps, from the first fossils of simple bacterial life 3.5 billion years old, to the first multicellular animals 700 m.y. ago (the Ediacara fauna), to the first evidence of skeletonized fossils (tiny fragments of small shells, nicknamed the “little shellies”) at the beginning of the Cambrian, 545 m.y. ago (the Nemakit-Daldynian and Tommotian stages of the Cambrian), to the third stage of the Cambrian (Atdabanian, 520 m.y. ago), when you find the first fossils of the larger animals with hard shells, such as trilobites.16
Take a look at the chart below: the bars on the right represent the diversity of genera. The bars increase gradually, not suddenly. And that pattern of continuous multiplication—which ended in a mass extinction event at the end of the Botomian—began long before the Cambrian period, refuting the claim that “complex animals first appeared fully formed, without any evolutionary ancestors.” You are free to believe in an “intelligent designer” if you want, but you shouldn’t ignore reality.
At this point, it should be clear that we can make a chart say whatever we wish—up to a point. We can do this by controlling how it is built, how much detail it contains, and, more importantly, how we interpret the patterns it displays. Read these two charts from the hilarious website Spurious Correlations, by author Tyler Vigen, who wrote a book with the same title:17
When I first visited Vigen’s website, I thought that a better title—although arguably a less catchy one—would be spurious causations. The reason is that the number of people who drowned by falling into a pool does covary with the number of movies Nicolas Cage appeared in. The data is the data, and its graphical representation is fine—although the dual-axis chart can be risky sometimes; as we saw in chapter 2, we can play with the axes to make the lines adopt any slope we wish.
What is truly spurious isn’t the correlation but the possible interpretation we may extract from the covariation of those variables: Does the fact that Nicolas Cage appears in more movies provoke more accidents? Or could it be that watching Nicolas Cage movies makes people more prone to swimming in their pools, exposing themselves to a higher risk of drowning? I’ll let you try to come up with spurious causal links between U.S. spending in science and the number of suicides by hanging. Have fun, if you enjoy gallows humor.