STAT WARS
Conflicts over Social Statistics
In the early 1980s, missing children became a prominent social problem; their faces appeared on milk cartons, and their stories were featured on television specials. Advocates coupled frightening examples of murdered or vanished children with disturbing statistics: strangers, they claimed, kidnapped 50,000 children each year. In 1985, reporters at the Denver Post won a Pulitzer Prize for pointing out that the movement’s statistics were exaggerated: they identified a “numbers gap” between the 50,000 estimate and the roughly 70 child kidnappings investigated annually by the FBI. In response, one activist testified before Congress: “I don’t think anything has surprised me more than this preoccupation with numbers, and the … ‘only 67 or 68 or only 69.’ … What is it with the ‘only,’ sir?” A movement that had promoted big numbers now argued that smaller, more accurate numbers were irrelevant. 1
All too often, bad statistics endure because no one questions them and points out their flaws. Any number—even the most implausible figure (for example, 50,000 stranger abductions)—can survive if it goes unchallenged. We have identified the key issues in evaluating statistics: How did they arrive at that number? Is the number being used properly? Are the appropriate comparisons being made? While these issues aren’t all that difficult to understand, many of us seem intimidated when faced with almost any statistic. Rather than ask the obvious questions, we tend to remain silent. As a result, bad statistics can take on a life of their own; they survive and even thrive.
But not always. Some statistics—such as the number of children abducted by strangers—lead to public controversies, open debates over numbers and their interpretation. We are told that guns kept in the home are rarely—no, frequently—used in self-defense, that they often—no, only rarely—kill family members. Environmentalist scientists estimate that water contaminated by nuclear power plants will cause many cancer deaths; scientists employed by the power industry insist that such deaths will be very rare. Such stat wars also intimidate us. If it is hard to evaluate one number, how can we be expected to assess the relative merits of different, competing statistics?
Stat wars indicate that someone cares enough to dispute a statistic. Usually these debates reflect the opponents’ competing interests. In chapter 1, we noted that statistics can become weapons in political and social debates. People who make claims about social problems have goals: they hope to convince others that they have identified a particular problem, that the problem is serious and deserves attention, that they understand the problem’s causes, and that they know how to solve it. Those whose claims succeed in convincing others stand to benefit. These benefits may include influencing social policy—perhaps by getting a new law passed, or a new program funded—but they go beyond that. People who make successful claims are likely to gain influence, power, status, even money; they become more important. Therefore, advocates usually have a vested interest in the success of their claims—whatever sincere commitment they have to their cause, they also stand to gain personally if their claims convince others.
In addition to advancing their personal beliefs and interests, people present claims that promote the interests of their groups. For decades, the tobacco industry insisted that there was no convincing evidence that cigarette smoking caused disease. Obviously, those claims reflected the industry’s bedrock economic interests: if tobacco were harmless, the tobacco industry should be able to continue selling it and profiting from its sale; if tobacco were harmless, the industry was justified in resisting every effort to restrict its business and therefore its profits; if tobacco were harmless, the industry should not be held liable for any harms suffered by smokers. While this example is particularly blatant, claims often reflect underlying interests of the individuals and groups who make them.
Often advocates find themselves struggling against others with competing interests: Republicans and Democrats oppose one another in political races; liberals and conservatives clash over the direction of public policy; corporations compete with one another to control markets; and on and on. Even activists debate the tactics and priorities of their social movements. Outsiders struggle to become insiders, while insiders try to keep them out; the have-nots want to get theirs, while the haves want more. These competing interests foster competing claims, and competing claims often lead to competing statistics.
We should expect people to promote statistics that support their interests. This need not be dishonest or cynical. Most people believe that their interests are legitimate, that their cause is just. They seize upon whatever evidence supports their position, and they point to it with pride and conviction. Often, this evidence is statistical. Remember that contemporary society tends to treat numbers with respect; statistics seem to offer hard, factual, indisputable evidence. When people find numbers that seem to support what they believe—and their interests—they tend to accept them. Recall chapter 2’s discussion of the ease with which activists produce, believe in, and justify big estimates for the size of social problems. It is especially difficult for most of us to think critically about statistics that seem to confirm what we already believe to be true. We may question the definitions, the measurements, or the samples that lie behind statistics offered by our opponents, but we are less likely to ask the same sorts of questions about figures that seem to support our own positions. Our numbers are undoubtedly good numbers, while our opponents’ figures are questionable at best.
Precisely because our society treats statistics as a powerful form of evidence, competing interests often lead to conflicts over statistics—to stat wars. If one group offers statistics (“These are facts!”) to support its position, its competitors are likely to feel pressured to find other numbers that support their interests. At times, debates over whose numbers are more accurate attract media attention. This is risky. If the media treat one side’s numbers as correct, then that side’s cause—and its interests—will seem to be right. Because the stakes can be great, stat wars can be very serious, as advocates denounce their opponents’ figures while seeking to promote their own numbers.
This chapter examines some cases of recent debates over statistics. While these debates appear to be about which numbers are better, the debaters often see themselves as defending not just a statistic, but important underlying interests or principles. Stat wars can focus narrowly on the accuracy of particular numbers or on the best methods for collecting statistical data, but often they are only a small part of a long-standing dispute over some broader social issue. The cases discussed in this chapter illustrate some different types of stat wars. Reviewing these cases can help us understand both the nature of these conflicts and the ways we should respond when confronted with these dueling figures.
DEBATING A PARTICULAR NUMBER: DID ONE MILLION MEN MARCH?
In the summer of 1995, the Nation of Islam’s leader, Louis Farrakan, called for African American men to join in a “Million Man March” on October 17, on the Capitol Mall in Washington, D.C. The Mall has long been a site for mobilizing huge political protests; Martin Luther King Jr. gave his “I Have a Dream” speech there in the 1963 March on Washington, and other large crowds have gathered in protest demonstrations over the war in Vietnam, abortion, women’s rights, and gay and lesbian rights. Many of these gatherings inspired disputes, not only about political issues, but also about the numbers of protesters. Predictably, demonstration organizers offered big estimates for the number of people gathered; after all, a big crowd suggests both that the cause has widespread support and that the demonstration was well organized and successful. In contrast, the U.S. Park Police, who police the Mall and had been charged by Congress with making official estimates of the crowds gathered there, usually gave markedly lower estimates.2 For example, organizers of an April 25, 1993, gay rights march on Washington estimated that more than a million demonstrators participated, but the Park Police estimated that only 300,000 attended.
It is easy to see how demonstration organizers and the Park Police could disagree. Not only do demonstration organizers have an interest in claiming that many people attended but, confronted by a huge mass of people, few organizers have any method of calculating a crowd’s size. In contrast, over the years, the Park Police devised relatively sophisticated methods for estimating demonstration sizes. They used aerial photographs to reveal the portion of the Mall covered by the crowd and, since the Mall’s dimensions are known, they could then calculate the area covered by people. The Park Police then multiplied the area covered by a multiplier—the estimated average number of people per square yard; the result was an estimate of the crowd’s size.
The Million Man March highlighted the political nature of crowd estimates. The very name of the demonstration set a standard for its success; as the date of the march approached, its organizers insisted that the crowd would reach one million, while critics predicted that it would not. (According to the Park Police, there had been only two occasions when million-person crowds gathered on the Mall: the celebrations marking Lyndon Johnson’s 1965 inauguration and the 1976 Bicentennial.) On the day of the demonstration, the organizers insisted that they had reached their goal; when Farrakan spoke, he claimed there were between a million and a half and two million people present. In contrast, the Park Police estimated the crowd’s size at 400,000 (a record for civil rights demonstrations).
While the discrepancy between organizers’ and the Park Police estimates fit the familiar pattern, Farrakan responded with outrage, charging that “racism, white supremacy, and the hatred of Louis Farrakan disallows them to give us credit,” and threatening to sue the Park Police.3 A team of Boston University specialists in the analysis of aerial photographs (of natural features, not crowds) examined the Park Police photographs and produced their own estimate—870,000 with a 25 percent margin of error (that is, they acknowledged that the crowd might have reached one million). The Park Police countered with more information, including additional photographs and public transportation records that showed only modestly higher traffic into central Washington on the day of the march. This led the BU researchers to revise their estimate slightly, down to 837,000.4
Very simply, the different multipliers used to calculate the crowd’s size caused the difference between the two estimates. The BU researchers assumed that the crowd was densely packed, containing six people per square meter—this is equivalent to 1.8 square feet per person, about the level of crowding in a packed elevator. It seems unlikely that a huge crowd of men would stand that close together for the hours that the rally lasted. In contrast, the Park Police assumed the crowd had about half that density, averaging 3.6 square feet per person. This is still quite densely packed for a listening audience; most crowds listening to speakers spread out and average 5.7–8.5 square feet per person.5
Clearly, the Million Man March attracted a very large crowd. Does it matter whether the crowd actually numbered one million, or whether it fell short of that number? In this case, the crowd’s size came to represent a number of symbolic issues, including Louis Farrakan’s popularity and influence among African Americans, and the degree to which the Park Police estimates reflected racism or other bias. Farrakan obviously felt committed to defending the figure of one million, and his critics delighted in insisting that the demonstration had fallen short of its goal and that Farrakan had been caught exaggerating. The BU researchers’ estimate called the Park Police estimation procedures into question, although the Park Police’s assumptions about crowd density seem more plausible. Still, providing crowd estimates that almost always undercut and angered demonstration organizers was a thankless task, and some of the press coverage of the BU team’s estimate implied that the Park Police might have been biased. Under new instructions from Congress, the Park Police announced that they would no longer provide estimates of Mall demonstration sizes.6
This debate focused on an apparently simple statistical question: how many people participated in the Million Man March? There were three different answers to this question. Louis Farrakan and the march’s other organizers insisted that the crowd numbered well over one million. Presumably, they derived their estimate in the same way many activists calculate the sizes of their demonstrations (and of social problems)—they guessed. The demonstration drew a huge crowd, and like other organizers of large demonstrations on the Mall, they guessed that there must have been a million people present. The second estimate, of course, came from the Park Police; they used already established methods—photographing the crowd, calculating the area it covered, and then using multiplier (one person for every 3.6 square feet) to estimate the crowd’s size. The third answer, by the team from Boston University, used the same method, but applied a different multiplier (one person for every 1.8 square feet). (Obviously, it would be possible to produce any number of estimates for the crowd’s size, simply by using different estimates for the crowd’s density—that is, by changing the multiplier.) The issue was really very simple: How close together were the people in the crowd standing? Remarkably, although the press was interested in the debate over crowd size, reporters generally covered the story by simply reporting the competing numbers; most reporters made no effort to understand how the numbers were derived, let alone to evaluate which assumptions were more likely to produce the most accurate estimate. Instead, the press attention focused on the motivations of those making the estimates: were the Park Police biased?
The Million Man March offers a couple of important lessons. First, it is often possible to understand the basis—that is, the broad outline, if not the technical details—of statistical debates. Second, the media often do not do much to advance such understanding. Press reports often fail to explain, let alone evaluate, how different groups arrive at different numbers. Instead, media coverage is often limited to reporting that one group gives the Number X, while the other group counters with the Number Y. This gives readers and viewers little help in interpreting these different estimates.
The stakes in the debate over the Million Man March were largely symbolic. Because Farrakan had promised to bring a million men together (and because he insisted that he had done so), some commentators turned the headcount for the march into a measure of his influence or credibility. Yet, regardless of whether a million men actually showed up, it was an impressive demonstration, and the number of marchers soon became a forgotten issue. Other, more complex, debates over statistics have longer histories.
DEBATING DATA COLLECTION: HOW SHOULD THE CENSUS COUNT PEOPLE?
The efforts to measure the Million Man March had to be completed on the day the crowd gathered—a relatively quick, inexpensive, limited operation. In contrast, consider the fantastically elaborate arrangements needed to gather the U.S. census. The U.S. Bureau of the Census has a huge budget—used to hire professional statisticians and social scientists, as well as thousands of people to collect and process the census forms, and to pay for all the equipment (not just forms, but computers, sophisticated maps, etc.) needed to conduct the census. Every ten years since 1790, the census has sought to locate and record basic information about every individual in the United States. The sheer size and complexity of this task—the U.S. population was approaching 300 million as the 2000 census approached—is astonishing. Still, the extraordinary effort that goes into preparing the census does not prevent the results from becoming controversial.
In spite of the bureau’s huge budget and the professional determination of the people responsible for the census, errors are inevitable.7 After all, hundreds of millions of people won’t stand still to be counted, and confusion cannot be avoided. Some people are counted more than once; for example, college students living on campus are supposed to be counted there, but their parents sometimes also list them as living at home. But more often people go uncounted and, overall, the census under-counts the population.
In some cases, undercounting occurs because people aren’t reached, perhaps because the census takers don’t realize that anyone lives where they do. But, much more often, people deliberately avoid being listed in the census because they don’t want to cooperate with or come to the attention of the government. Perhaps their political principles lead them to refuse to cooperate. Perhaps they are fugitives from arrest warrants or court orders. Perhaps they are undocumented aliens (illegal immigrants) who fear deportation. Perhaps they are violating welfare regulations by secretly living with (and helping to support) welfare recipients. Although other government officials (e.g., police, immigration, and welfare officials) are forbidden access to census records, many people suspect that responding to the census might get them into trouble, and they prefer to go uncounted. The resulting undercounting is not random. In general, those who go uncounted tend to be poor, urban males; this also means they are more likely to be nonwhite than the general population.
Census undercounting matters because census figures are put to use. Increasingly during the second half of the twentieth century, the federal government began using census figures to allocate major resources. Perhaps most important, the Supreme Court ruled that legislative districts needed to represent approximately equal populations. Population figures, of course, come from the census, so that a large city where a significant number of citizens went unrecorded by the census could wind up under-represented in Congress or the state legislature. In addition, federal funds for all manner of “set-aside” programs—funds for highway construction, many social services, and so on—are allocated to the states according to their populations. Imagine a program that delivers one dollar per person to each state; for every person not counted, the state receives a dollar less than it should receive. Other people care about undercounting as well. A civil rights activist seeking to measure discrimination in employment might argue that the proportion of workers in some industry who belong to a minority group is lower than that minority’s proportion in the population. But, if minority group members are undercounted in the census, the activist’s calculation of the gap between the minority’s share of population and their share of the jobs will be less than the actual gap.
Note that, while the census probably undercounts all groups, some groups have larger undercounts. These patterns benefit some and disadvantage others. Groups that have relatively low undercounts benefit; they receive more legislative representation, more federal funds, and so on. This gives these groups an interest in maintaining existing census procedures. In contrast, groups that have relatively more undercounting lose, and they have an interest in trying to correct the undercount.
How many people go uncounted? Obviously, no one knows exactly what this dark figure is (if we could count all the people who were not counted by the census, they could simply be added to the census total, and the problem would be solved). However, the best estimates are that the 1990 census undercounted the population by about 2 percent. At first glance, this might seem to be a relatively small percentage, but remember that even a small percentage of a large population is a fairly large number. The 1990 population was 248 million; that means that the undercount was roughly 5 million. Moreover, 2 percent is a net undercount (that is, after estimated overcounting is subtracted from the estimated undercount). Studies suggest that the census manages to count the vast majority of people accurately; the best estimates are that about 90 percent are properly counted, but there remain millions of errors in both overcounting and undercounting.
Moreover, undercounting is not distributed randomly throughout the population. The best estimates are that the net undercount for nonblacks was about 1.5 percent, while the net undercount for blacks was about 5 percent (there are also fairly substantial undercounts of other ethnic minorities). This means, for example, that the census significantly underestimates the size of cities and states with large minority populations.
Critics argue that the census results ought to be adjusted to reflect estimated undercounting. They recommend basing adjustments on postenumeration surveys (PES), in which specially trained census takers conduct interviews with a large sample of households. PES interviews produce more accurate data than the census forms that people are supposed to fill out and return by mail. Calculating the differences between the data gathered through the PES interviews and the census forms completed by the same households establishes a basis for estimating the undercount. (In fact, the Bureau of the Census already does PES interviewing; that is how it estimates the size of the under-count. PES analysis is the major source for the estimate that the undercount was 2 percent in 1990. Still, PES results have not been used to adjust the final census figures.)
There is fairly general agreement that the census is imperfect, that it undercounts the population, and that relatively large numbers of African Americans and other ethnic minorities are uncounted. The question is how the government ought to respond. Increasingly, demographers (including many of the professionals in the census bureau) argue that the final census results should be adjusted (that is, changed to reflect the best estimates that can be derived through PES analyses) to produce more accurate final totals. It is important to appreciate that these adjustments would change not just the total population, but the population totals for individual states, cities, and so on. Adjusting the census would probably mean, for example, that cities with large minority populations would be credited with markedly more people than the census managed to count, while mostly white, middle-class suburbs would gain relatively little over the census count. These adjusted figures would, the demographers insist, be more accurate than the inevitably flawed results of the regular census enumeration.
If the census were simply the government’s best effort to collect accurate data about the population, the argument in favor of adjustment might gain wide acceptance. (Survey researchers often recalculate the results of public opinion polls in order to produce more accurate estimates by giving more weight to respondents thought to represent undersampled groups within the population.) But the census is a powerful political symbol. Every ten years, the federal government attempts to tally all Americans, and everyone is supposed to cooperate with that effort. The census is supposed to be a great compilation of individuals’ responses; we do not think of it as just an estimate or a calculated guess. At least in the popular imagination, census figures represent an actual count; the very real inaccuracies in the census are not widely understood. Thus, one objection to adjusting the census findings is that many people think of census results as factual, and adjusting census figures smacks of tampering with the truth.
Remember, too, that changing the census results would be consequential. For instance, if the adjusted results for the 1990 census had been used to apportion congressional districts, two congressional seats would have shifted (Pennsylvania and Wisconsin would have each lost one, while Arizona and California would have gained one apiece). In addition, federal funding for all manner of programs would have changed; some states would have gained, but others would have lost. Choosing the basis for calculating the census is not just an abstract problem; it has real political consequences (at least so long as census figures are used as the basis for such policies as apportioning legislative districts and distributing federal funds to the states).
In recent decades, calls for adjusting the census totals have come from the mayors of big cities and other political leaders who believe they represent populations that are undercounted. Because poor nonwhites are most likely to be undercounted, and because poor nonwhites tend to vote Democratic, many of the politicians favoring adjusting the census totals have been Democrats. These politicians tend to argue that undercounting has serious consequences; they claim, for instance, that their cities and districts have lost millions of dollars in federal funding. In contrast, Republicans have tended to favor retaining the totals from the census enumeration as the official figures (and to argue that the consequences of undercounting aren’t that serious). Both sides have sued in various courts, seeking to compel the use of whichever set of figures favors their interests.
The debate over the methods adopted by the census bureau is necessarily technical, and most citizens have no real grasp of how the bureau does what it does, what it might do differently, or the relative limitations of the different approaches. Those favoring adjusting the census argue that the resulting figures would be more accurate; most—although not all—social scientists favor adjustment because they are familiar with the need to weight results in survey research. Those opposed to adjusting the census warn that adjustment can never be perfect, and they question whether it makes sense to add an imperfect adjustment to an already imperfect enumeration. The courts have produced different rulings on the question but, in 1999, the U.S. Supreme Court ruled that the 2000 census could not be adjusted for purpose of congressional reapportionment. The debate will undoubtedly be renewed as planning starts for the 2010 census. Given the entrenched interests and high stakes in the outcome, the stat war over how the census counts the population is sure to continue.
STATISTICS AND CONTENTIOUS ISSUES
Both the short-lived argument over the turnout for the Million Man March and the long-standing, ongoing debate over whether the census should be adjusted are narrowly focused disputes. At issue are relatively straightforward questions—about a single number in one case and the method of measurement in the other. In contrast, many stat wars involve multifaceted debates that continue without resolution over years, even decades. Such struggles feature numerous skirmishes over different statistics related to some core social issue.
Advocates often hope to create societal consensus about some social problem, to bring a previously neglected condition to public attention, arouse concern, and promote new policies to deal with the problem. But disputes over some social issues can never reach consensus because there is real disagreement about what the problem is and what ought to be done about it. Is abortion a social problem because it involves the murder of unborn children, or is the problem that difficulty gaining access to abortion is one more way society restricts women’s opportunities? Should the troubles associated with illicit drug use be resolved by decriminalizing drugs, or are even tougher laws and more vigorous enforcement the answer? Should we protect or restrict the right to bear arms?
In contrast to the debate over the turnout for the Million Man March, where the statistic became the central issue, these more complicated disputes over social issues are fundamentally disagreements over values. The abortion debate, for example, is often framed in terms of rights—the right of the fetus to societal protection vs. the right of a pregnant woman to choose abortion. Any individual’s judgment that one of these rights outweighs the other derives from some interpretation of societal values. Americans value all sort of things, such as freedom and equality. While we like to imagine that our values are complementary and perfectly consistent with one another, this is a fiction: a perfectly free society is not likely to be especially egalitarian, nor is a perfectly egalitarian society likely to be especially free. Contentious social issues are contentious precisely because people make different judgments about which values are more important in the particular case: Is protecting a fetus more important than a woman’s freedom to choose abortion? Does the individual’s right to bear arms outweigh society’s need to control violence? And so on.
Contentious issues pit competing advocates against one another. Each side is likely to have its own activists affiliated with social movement organizations (e.g., the National Right to Life Committee vs. the National Abortion and Reproductive Rights Action League). Each side can usually muster its own authorities—medical experts, religious leaders, legal scholars, social scientists, and media commentators. Depending on the issue, each side may be affiliated with a particular ideological slant (liberals vs. conservatives) or political party (Republicans vs. Democrats). Although we commonly speak of these debates as having two conflicting sides, reality is often more complex, with advocates staking out multiple positions based upon differences in their ideologies or interests. Unlike cases where there is widespread consensus about some social problem, contentious issues involve groups with competing interests making conflicting claims.
Statistics usually play a supporting role in these conflicts. The debate over abortion revolves around a clash of values, and no statistic can resolve that issue. When statistics do enter the abortion debate, advocates typically use them to show that there is broad support for their position (“Polls show that most Americans share our values or agree with us.”). (Recall from chapter 2 how selective wording of questions allows advocates on both sides of the abortion and gun-control debates to point to public opinion surveys that seem to support their positions.) Moreover, long-running debates over broad issues can lead to many different struggles over particular statistics that somehow bear on the larger topic.
Consider issues of equality. Americans readily endorse equality as a value, yet inequalities of race, gender, and class have been—and continue to be—visible and important. American history features lengthy, ongoing campaigns by the civil rights movement, the women’s movement, the labor movement, and other advocates claiming that particular kinds of people are blocked from full equality. Typically, these (usually liberal) advocates insist that, whatever progress may have been made, serious inequities remain, and these warrant making further significant changes in social policy to foster equality (examples might include raising the minimum wage, establishing tougher antidiscrimination policies, and so on). In contrast, their (usually conservative) opponents argue that considerable social progress toward equality already has been achieved (implying that additional drastic changes in social policies are not only not needed, but may cause harm by unnecessarily restricting people’s freedom).
Arguments about recent changes in the distribution of income are just one small aspect of this larger debate over equality.8 Many Americans like to imagine that their economy fosters growing prosperity. This vision finds statistical support in measures of per capita personal income (basically the nation’s total income divided by its population). Per capita income rose fairly steadily in the second half of the twentieth century (see the first column in Table 6). Those with a stake in defending the status quo (not just conservatives, but political incumbents generally) can point to the growth in per capita income as proof that things have been getting better (“Prosperity is increasing! There’s more money per person. There’s no need to change.”).
Table 6. Gross Domestic Product Per Capita and Average Hourly Earnings, 1959–1999
| Year | Per Capita Incomea | Hourly Earningsb |
1959 |
$12,985 |
$6.69 |
| 1964 | 14,707 | 7.33 |
| 1969 | 17,477 | 7.98 |
| 1974 | 18,989 | 8.28 |
| 1979 | 21,635 | 8.17 |
| 1984 | 23,171 | 7.80 |
| 1989 | 26,552 | 7.64 |
| 1994 | 28,156 | 7.40 |
| 1999 | 32,439 | 7.86 |
a Gross domestic product per capita in 1996 dollars.
b Average hourly wages in private nonagricultural industries in 1982 dollars.
SOURCE: U.S. Department of Commerce, Economic Report of the President 2000 (Washington, D.C., 2000), pp. 341, 360.
In response, critics of the status quo (often liberals, but also the incumbents’ challengers) cite statistics showing that the economy is not getting better—and may actually be getting worse. For example, average hourly earnings peaked in the early 1970s and generally fell during the century’s last three decades (see the second column in Table 6). Of course, this seems troubling (“Things are getting worse! People are earning less per hour of work. We need to do something.”).
How is it possible for the average income per person to rise at the same time the average hourly wage fell? Changes in the work-force help account for this apparent discrepancy. Most important, the proportion of the population in the workforce grew, in particular, the proportion of employed women rose. (As a growing percentage of families featured two wage-earners, average family incomes rose. If, on average, the husband’s income declined a bit, this was offset by increases in the wife’s income as she entered the workforce or increased her hours.) If a growing proportion of the population is employed, per capita income can grow, even if hourly wages decline somewhat. (Increases in the number of hours individuals work can have the same effect.)
Defenders of the status quo argue that hourly earnings are a poor measure of economic prosperity. They note, for example, that real hourly compensation (that is, wages plus the value of benefits) generally rose in recent decades. Moreover, year-to-year comparisons of income must be converted to constant dollars to adjust for inflation, and the status quo’s defenders claim that the Consumer Price Index (CPI) used to make the adjustments for inflation exaggerates the amount of inflation (and thereby underestimates the growth in the value of wages). By correcting the CPI and taking the value of benefits into account, these advocates can make a case that hourly compensation in fact rose in recent decades.
Their critics’ response to this rosy view focuses on income inequality. The old aphorism “The rich get richer and the poor get poorer” expresses this critique. Contemporary critics warn about the “shrinking middle class.” There is a great deal of evidence that income inequality has grown in recent decades. Whether we consider family incomes or individual incomes (and whether we look at the incomes of males or females, of black or white workers), the same pattern emerges: incomes among those already earning more have been growing faster than the incomes of those earning less. Typically, these measures reveal that the incomes of those earning the least (say, the lowest-earning fifth of the population) actually declined during the same period that the incomes of those earning the most (the highest-earning fifth of the population) showed substantial increases. In effect, the rich have been getting richer, and the poor poorer.
It is important to appreciate that these measures of growing inequality usually do not track particular individuals through time. That is, when we compare, say, the poorest fifth of the population in Year 1 with the poorest fifth ten years later, we are not necessarily talking about the same people; some poor people experience upward mobility. Even if incomes in the lowest-earning fifth of the population fell between Year 1 and Year 10, an individual who was in the lowest fifth in Year 1 may, in Year 10, be in some higher-earning category (for an extreme example, imagine someone working at a mininum-wage job in Year 1 while going to law school; in Year 10, the same individual might be a highly paid lawyer who falls into the highest-earning fifth). Such upwardly mobile people create vacancies in the lowest-earning fifth, and some of those slots will be filled by newcomers just entering the workforce. On the other hand, while upward mobility does occur, it is far from universal; many people—particularly those with limited education and job skills—remain trapped in the lowest-earning fifth and experience declining incomes. While some individuals may not notice the growing inequality in the larger society because they personally experience upward mobility, others find themselves falling further behind.
The debate over income distribution is complex. The government produces vast amounts of economic data, and economists disagree about the best measures (the dispute over the most reasonable way to calculate the CPI is only one example). By choosing carefully among the available statistics, advocates can find support for very different positions (“Rising per capita income shows growing prosperity!” “No, falling hourly income shows diminishing prosperity!”). In the resulting barrage of statistics, proponents of different positions argue that their numbers are significant, while their opponents’ figures are poor measures of whatever is at issue.
Making sense of this confusion is challenging. Clearly, it is better to evaluate competing statistical claims, rather than listening to only one side, and it helps to understand as much as possible about what the different numbers mean. It also helps to realize how social changes affect statistics and their significance: more women are working and people are having fewer children (two changes that lead to higher family and per capita incomes); a growing proportion of jobs are highly skilled (which contributes to income inequality); and so on. There is, alas, no single, authoritative measure of prosperity and, as society changes, various statistics may become better or worse indicators of economic and social conditions.
Debates over topics as broad as equality have countless facets; although this discussion has focused on income inequality, we might have chosen to focus on inequalities of race, gender, and so on. The broader issue is grounded in philosophical disputes over the nature of not just equality, but also liberty, justice, and other values, and over how government and other institutions should devise social policies to foster and protect those values. With a broad, mutifaceted issue, virtually every facet—income inequality, public opinion, criminal justice, quality of health care, access to employment and higher education, and on and on—can be contested, and in each of these contests advocates can cite statistics to support different points of view.9
Whenever there is disagreement about the statistical evidence, it is possible to look more closely, to discover how different measurement choices, different definitions, or other factors can explain the disparities. But, of course, this can be a lot of work; few people will make the effort to examine original sources in order to reconcile a stat war. And, even when it is possible to clarify a specific statistical disagreement, that clarification will not resolve the larger debate about the broader social issue. Again, debates over broad social issues have their roots in competing interests and different values. While advocates for different positions tend to invoke statistics as evidence to bolster their arguments, statistics in and of themselves cannot resolve these debates.
CLAIMING STATISTICAL AUTHORITY
Still, our society makes it easy to create and spread statistics about social problems. This is important because we often equate numbers with “facts.” Treating a number as a fact implies that it is indisputable. It should be no suprise, then, when people interested in some social problem collect relevant statistics and present them as facts. This is a way for them to claim authority, to argue that the facts (“It’s true!”) support their position.
One interesting way of claiming authority in recent years has been to publish collections of social problems statistics in small, specialized reference books. The titles of these volumes often emphasize the factual nature of the contents, even though the books frequently promote a particular ideology or the interests of a specific group. Compare two books: William J. Bennett, the conservative politician, published The Index of Leading Cultural Indicators: American Society at the End of the Twentieth Century, while social scientists Marc Miringoff and Marque-Luisa Miringoff wrote The Social Health of the Nation: How America Is Really Doing.10 Both books present multiple social statistics intended to document trends over the past two or three decades. Both books insist that these trends are troubling. According to Bennett: “In two generations, America has undergone dramatic and traumatic social change—the kind that one would normally associate with cataclysmic natural [sic] events like famine, revolution, or war. Civilizations stand on precious few pillars, and during the last three and a half decades, many of ours have fractured.”11 Similarly, Miringoff and Miringoff argue: “On the whole, long-term trends in social performance may be viewed as less than encouraging. While some indicators show improvement …, many have worsened significantly over time…. These are warning signs which require attention.”12
Attempts to track social indicators over time confront many of the problems discussed in earlier chapters. These include the basic difficulties in defining and measuring social conditions. However, there is a larger problem: the government does not collect and publish many series of statistics for social indicators. This is in sharp contrast to the government’s treatment of economic statistics; anyone who follows the news cannot help but hear regular (usually monthly) statistical updates about the balance of trade, consumer confidence, the Consumer Price Index, housing starts, unemployment, and so on. Many of these measures have been collected and published for decades. In contrast, there are few comparable indexes of social trends, they tend to be published less frequently (often annually), and the lag time between data collection and publication tends to be longer.13 In addition, with the exception of population statistics and crime rates, relatively few data have been collected at regular intervals, using standard measures, for enough years to establish clear trends.
Moreover, social statistics pose problems of interpretation. Even when data are available, they do not speak for themselves. For example, both Bennett and Miringoff and Miringoff note the increase in child poverty. From 1970 to 1996, the percentage of children (under age 18) living in poverty rose, from 14.9 percent in 1970, to 19.8 percent in 1996. (In contrast, the percentage of Americans aged 65 and above living in poverty fell markedly during the same period, from 24.6 percent in 1970, to 10.8 percent in 1996. That is, in 1970 children were much less likely to be poor than were older Americans, but in 1996 the positions had been reversed.) What should we make of this shift? Miringoff and Miringoff note that the U.S. rate of child poverty is far higher than those found in other industrialized nations, although they point out that other countries “have relatively high child poverty rates before the application of tax and transfer programs designed to improve the status of children.”14 In contrast, Bennett’s interpretation of the data locates the cause of child poverty in family structure: “Poverty afflicts nearly one of every two mother-only families (45 percent in 1992) and fewer than one in ten married-couple families (8 percent in 1992)”; and “Almost 60 percent of children under 6 living in families with only a mother had an income below the poverty level, more than five times as many as children under 6 in married-couple families (10.6 percent).”15
How should we interpret rising child poverty? For Bennett, a political and social conservative, child poverty is a “cultural indicator,” a product of a deteriorating culture that tolerates out-of-wedlock births, divorce, and children raised in single-parent families. That is, cultural changes have led to more children in single-parent families, which in turn means more children live in poverty. Presumably, Bennet would argue that the ultimate solution for child poverty would be a return to traditional values or virtues that would ensure that children are raised in intact families. In contrast, Miringoff and Miringoff do not specify the causes of rising child poverty, but they clearly favor social policies to provide additional support for poor children. A reader might infer that they locate the causes of child poverty in the sort of structural inequalities emphasized in liberal analyses, rather than in the cultural failures decried by conservatives.
This example reveals that facts do not speak for themselves. The previous section used a stat war about income inequality to illustrate how debaters choose to measure social problems in different ways. But the competing interpretations of the growing percentage of poor children involve a single method of measurement, a relatively clear-cut social indicator (children living in households below the poverty level), and high-quality data collected by a federal agency; yet advocates interpret the same numbers very differently. Where Bennett finds evidence of moral collapse, Miringoff and Miringoff see inadequate social policies to protect children’s welfare. We may think of statistics as facts, but people make facts meaningful, and analysts’ ideologies shape the meanings they assign to social statistics.
Worse, these reference books’ efforts to claim statistical authority do not always involve high-quality data. Earlier chapters have noted the relative ease with which statistics circulate, even numbers based on guesses, peculiar definitions, deceptive measures, and weak samples, to say nothing of numbers that have been mangled to produce mutant statistics. Bad numbers may originate in particular—even glaring—errors, but they can live on indefinitely in media reports. Reference books compile statistics—good and bad—and reprint them without subjecting them to much critical analysis. For example, the Women’s Action Coalition (WAC) published WAC Stats: The Facts about Women. This book repeats some flawed statistics discussed in earlier chapters (e.g., “150,000 American women die of anorexia a year”; “Gay and lesbian teenagers are up to 3 times more likely to commit suicide than their heterosexual peers.”) among the hundreds of statistical claims in its pages.16 The source for each claim is given in a footnote, and the range of sources is extraordinary: some numbers come from government documents, but others come from flyers, handouts, or fact sheets distributed by activists; and most seem to have been taken from newspaper or newsmagazine stories. That is, this reference book depends upon what historians would call secondary, rather than primary, sources. For example, if a sociologist conducts a study and publishes the results in a scholarly journal, that is a primary source; but a newspaper story referring to the study’s findings is at least an additional step removed from the research—it is a secondary source. In most cases, the newspaper story gives little or no information about the key decisions that shape research results (how concepts were defined and measured, what the sample was, and so on). Worse, the press often fails to differentiate between statistics produced through carefully designed research and far less reliable numbers. Newspaper readers have enough trouble evaluating the statistics they find in newspaper stories. When a reference book uncritically copies hundreds of such numbers from newspaper stories and other, even less reliable sources, and presents them as straightforward facts, it makes an unjustified claim to statistical authority. And, because many of these reference books are intended to present data that support particular ideologies or interests, they make no effort to include competing statistical claims that might lead their readers to be cautious. Just because someone claims authority does not mean we ought to grant it.
INTERPRETING STAT WARS
Advocates use statistics to support their claims about social problems. Rarely will they invoke numbers that seem to call their claims into question. So long as they can mobilize consensus about a social problem, so long as their claims encounter little opposition, advocates’ numbers go unquestioned. In contrast, debates over social issues feature competing, contradictory claims that often include arguments over statistics—what I have called stat wars.
Stat wars create confusion. Because we tend to think of numbers as facts, most of us have difficulty reconciling conflicting figures. Certainly stat wars often distress the press. Ideally, press coverage presents the facts, and reporters and editors like to repeat statistics because numbers seem factual.17 When the press is confronted with what are clearly contradictory numbers (as in the case of the conflicting estimates for the size of the Million Man March), it has trouble doing more than simply acknowledging the disagreement. Even advocates find stat wars troubling. Advocates often believe their own statistics, and they respond to challenges to their numbers with outrage: at best, their opponents are misinformed; at worst, the competing figures are outright lies.
While some social problems statistics are deliberate deceptions, many—probably the great majority—of bad statistics are the result of confusion, incompetence, innumeracy, or selective, self-righteous efforts to produce numbers that reaffirm principles and interests that their advocates consider just and right. The best response to stat wars is not to try and guess who’s lying or, worse, simply to assume that the people we disagree with are the ones telling lies. Rather, we need to watch for the standard causes of bad statistics—guessing, questionable definitions or methods, mutant numbers, and inappropriate comparisons. In some cases, we may conclude that one number is right and another is deeply flawed; in others, we may discover that the different figures reflect people choosing different methods to answer different questions. Whatever we conclude, we should come away with a better understanding of all the statistics.