CHAPTER TWO
SCIENCE & LIBERALISM
The spirit of liberty is the spirit which is not too sure that it is right.
—JUDGE LEARNED HAND, 1944
The method of freedom is the method of science.
—SOCIOLOGIST LYMAN BRYSON, 1947
Liberalism shares many of the qualities of science, but since both are widely misunderstood it may be useful to examine their relationship. Liberalism nourishes science by fostering a free and flexible milieu in which scientific creativity can flourish, which in turn increases the knowledge, power, and wealth of liberal societies. In doing so, science helps demonstrate that liberal governance works; and so the cycle continues.
Liberalism (from the Latin for “freedom”) is based on what John Stuart Mill in 1859 called
one very simple principle…. that the sole end for which mankind are warranted, individually or collectively, in interfering with the liberty of action of any of their number, is self-protection. That the only purpose for which power can be rightfully exercised over any member of a civilized community, against his will, is to prevent harm to others.
This thesis is admirably easy to test: Just increase personal freedoms and see what happens. Originally such experiments took the form of putting limitations on the powers of monarchs, as with the English Bill of Rights of 1689. Today, every day, liberalism is tested through the functioning of the world’s liberal democracies. Its accomplishments to date include the abolition of slavery (seven million slaves freed in a single century, an unprecedented accomplishment), the extension of legal rights to women and minorities, the maintenance of free speech and a free press, and an unprecedented global growth in knowledge, prosperity, and power.
Liberalism arose when, with the advent of science, a relatively static Western civilization began to become more creative, dynamic, and free. The word liberal entered the English language through education, where it signaled a shift from the narrow preparation of students for the vocations to which their social status had predestined them (usually law or the priesthood) to a “liberal sciences” (later called liberal arts) approach aimed at arming students with the tools needed to thrive in an unpredictable world. Andrew Abbott of the University of Chicago used to tell incoming freshmen, “There are no aims of education. The aim is education.” Liberally educated men and women will have learned how to keep learning throughout their lives, so that they can participate in a dynamic, changing society. Liberalism’s acceptance of unpredictable change distinguishes it from conservatism, which concentrates on the lessons of the past, and from progressivism, which plans for a future about which it knows rather less than it sometimes claims.
Liberalism is inherently nonpartisan: It means freedom for all, or it means nothing at all. It maintains that everyone benefits from everyone’s freedom, and that all are diminished whenever one individual or group is not free. This precept can contort liberals into the uncomfortable posture known as tolerance. Some think that tolerance means treating all opinions as equally deserving of respect, but the point of liberalism is not that all views are equally valid. It is that society has no reliable way to evaluate opinions other than to let everybody freely express and criticize them—and, if they can garner sufficient support, to try them out.
It was difficult even for the founders of liberalism to fully embrace tolerance. John Locke would have denied equal rights to atheists: “Those are not at all to be tolerated who deny the being of a God,” he declared, in his Letter Concerning Toleration, since the “promises, covenants, and oaths, which are the bonds of human society, can have no hold upon the atheist.” Many otherwise liberal thinkers today recoil from the prospect of granting homosexual couples the same legal benefits that heterosexual couples enjoy, or affording legal rights of due process to those accused of terrorism. Such concerns—essentially the nagging worry that something terrible will happen if too much freedom is extended to people who do not closely resemble ourselves—have so far prevented societies from becoming entirely liberal. But each step taken to extend equal rights to those previously denied them has in retrospect been seen to benefit not only the group in question but the society as a whole.
As an empirical, experimental philosophy that accommodates error and uncertainty, liberalism rejects all absolutist political claims, including absolute faith in religion at one extreme and in rationalism at the other. Liberalism does not oppose religion—it is a staunch defender of religious freedom—but it demands that the state grant special status to no religion; as Machiavelli observed, religion plus politics equals extremism. Rationalists are apt to imagine that they can reason their way to a political scheme so self-evidently superior that its implementation justifies at least a temporary suppression of opposing views; liberals will make no such concessions, because they appreciate that nobody is prescient enough to justifiably sacrifice present liberties for imagined future gains. That is the sense of Judge Learned Hand’s suggestion that Oliver Cromwell’s injunction, “I beseech ye, in the bowels of Christ, think that ye may be mistaken,” be inscribed over the door of every church, school, and courthouse in the nation.
The ideal of liberalism is universal peace and mutual aid. “The starting point of liberal thought is the recognition of the value and importance of human cooperation,” wrote the economist Ludwig von Mises, “and the whole policy and program of liberalism is designed to serve the purpose of maintaining the existing state of mutual cooperation among the members of the human race and of extending it still further…. Liberal thinking always has the whole of humanity in view and not just parts.” Liberals are opposed to war not only for the usual reasons but also because wars tend to aggrandize governments, ballooning their budgets and emboldening them to draft conscripts. Similarly, liberalism opposes imperialism, colonialism, racism, and every other form of oppression.
There is, however, one inherent problem with liberalism. Since absolute liberty would be anarchy, liberalism must sanction some form of coercion to prevent the strong from abridging the freedoms of the weak. The troubled history of American race relations is replete with examples of such illiberalism on the march: The white-robed Klansmen who bombed churches and burned crosses might be said to have been exercising their freedom of speech and association, as were the white American shipyard workers who harassed their black coworkers during World War II. To prevent such injustices, liberals concede to government a monopoly on coercive force. The police, the national guard, and the military are entitled to employ force, while corporations, vigilante groups, and self-appointed militias are not.
This governmental monopoly on coercive force calls forth two further liberal mandates. The first is equal protection under the law. (Locke: “Where there is no law there is no freedom.”) Liberty is abridged whenever the government places itself above the law. The other mandate is that government be kept small, lest the citizenry be snared in growing tendrils of laws and regulations backed by ever-increasing powers of enforcement and intimidation. Each such measure may be well intended—as when voters seek to save the jobs of steelworkers or stockbrokers, curb hate speech, or comb millions of e-mail messages in search of threats to national security—but their proliferation increases the power of government to confiscate your wages and property, put you in jail, and send you off to war. This battle liberalism has been losing. In every liberal democracy today the government’s share of the national wealth is increasing with each passing decade. In the United States, a more laissez-faire nation than most, the government currently claims a third of the wealth, and is growing rapidly; some of the socialistic democracies of Scandinavia collect more than half. Elected officials find the trend difficult to reverse. President Ronald Reagan and Prime Minister Margaret Thatcher attempted to shrink the size of government, yet Thatcher never managed to do more than to slow the rate at which the British government grew, and Reagan presided over major increases in federal spending that swelled the national debt from 23 percent to 69 percent of GDP. The only recent presidents to have significantly shrunken federal spending as a percentage of GDP were Harry Truman (–8.6 percent, in the aftermath of World War II) and Bill Clinton (–1.8 percent). This is not a new story; liberalism seems always to be creating wealth and fighting against its confiscation. “Opinion tends to encroach more on liberty,” complained J. S. Mill in 1855, “and almost all the projects of social reformers in these days are really liberticide.”
To the extent that liberalism is scientific, it is obliged to judge its success in terms of measurable quantities. Conservatives may argue that people in the old days were healthier in spiritual terms—that for instance they were happier when everyone knew his place—but liberals prefer to measure what can be measured, in a quantitative way. (As Lord Kelvin said, in words now chiseled on the University of Chicago’s Social Science Research Building, “When you cannot measure, your knowledge is meager and unsatisfactory.”) One quantitative measure of national status is productivity—the creation of wealth—and there the verdict is favorable: People have most prospered where they have been most free. Nor has this just been the prosperity of a few. Even though the United States grew more economically vertical from 1950 to the early twenty-first century, all five quintiles of the population—from the bottom fifth to the top—increased substantially in wealth during that same period.
The emphasis placed by liberals on economic productivity has prompted critics to dismiss them as materialistic. Liberals plead guilty to this charge. The quantitative measurement of material results may ignore spiritual considerations, but even when people are asked ethereal questions about how they feel, those living in liberal states say they prefer it that way, while those living elsewhere—if they have any reasonable access to uncensored information—say that they, too, would prefer liberal governance. It is on such bases that von Mises could call liberalism “the application of the teachings of science to the social life of man.”
What this book calls liberalism is the philosophy more often known in the United States as “classical” liberalism—the original liberalism, dating from Thomas Hobbes and John Locke and the American founders. That is not, however, what liberalism has come to mean for most Americans. In the United States today the word liberal is more apt to be applied to leftists or progressives—those who value equality over liberty, and are willing to put the force of government behind efforts to create greater political and economic equality even if personal freedoms are abridged in the process.
Consider the term liberal democracy. If you’re a classical liberal, what matters most to you is the liberal side of the concept. You may be enthusiastically democratic, doubting that any form of government other than a democracy can be liberal, but you know that democracies can behave in illiberal ways—as when millions of Germans voted for the National Socialist Party, one of whose leaders declared, “We are socialists, we are enemies of today’s capitalistic economic system for the exploitation of the economically weak, with its unfair salaries [and] evaluation of a human being according to wealth and property instead of responsibility and performance.” Alert to such dangers, you feel that liberalism is more likely to rescue a wayward democracy than the other way around. In this your position resembles that of Locke and other Enlightenment philosophers, and of the many Whigs and other liberals who, well into the nineteenth century, regarded liberalism as a given but democracy as an ongoing experiment.
If, on the other hand, you are an American “liberal” of leftist inclinations, what matters most to you about liberal democracy is that it’s democratic. Democracy is mainly about equality—one man, one vote—and when you look around and see people living in steeply unequal conditions, you’re apt to feel that the system isn’t working. If, say, African-Americans are substantially poorer and less well educated than are white Americans, you are not likely to be satisfied with (classically) liberal talk about universal freedom and equal protection under the law. You may be willing to vote for higher taxes and a bigger, more intrusive government if the result will be greater equality of outcome, even at the expense of reducing equalities of opportunity. Such views are best described not as liberal but as, say, progressive.
The tension between liberals and progressives—between those who stress liberty and those who stress equality—has long persisted; as Reinhold Niebuhr observed in 1943, “Whether democracy should be defined primarily in terms of liberty or of equality is a source of unending debate.” The use of one word to describe them both needlessly confounds American political discourse. Only 20 percent of Americans describe themselves as liberals, even though a majority hold both liberal views (supporting equal rights for all and specifically for homosexuals) and progressive views (60 percent favoring universal health care, and 70 percent agreeing that the government has an obligation “to take care of people who can’t take care of themselves”). As a result many classical liberals are reluctant to be called liberals, since the term is so often conflated with big-government progressivism.
Mathematicians are familiar with the distortions that result when a complex structure is collapsed into too few dimensions. A snowflake, smashed into two dimensions, becomes a botched asterisk; squeezed into one dimension it is a line. That is precisely the problem with employing a one-dimensional political spectrum that puts conservatives at the right and so-called liberals at the left, and pretends that all other political philosophies reside between. This crude paradigm, which originated in the French National Assembly circa 1789, when the Jacobin faction sat to the left of the president’s chair and the Girondins to his right, has long caused confusion. Better to add another dimension, forming a triangle:
In one corner of the triangle stand the conservatives—upholders of tradition, captains of a sea anchor that steadies the ship of state, comfortingly reliable if sometimes awkward in the face of change. There will always be conservatives, because there will always be traditions worth conserving. If you’re a young person who detests conservatism and manages to enact radical reforms which in later years you are obliged to defend, in that respect you shall have become a conservative.
Occupying the second corner are the liberals—defenders of liberty, embracers of change, and friends of science.
In the third corner are those who emphasize equality of outcome over freedom of choice—the progressives, socialists, or social democrats. Their standing is made clearer in nations that have three major parties rather than America’s two—progressives in Finland and Denmark tend to be Social Democrats, while in Canada they are known as New Democrats—but even there the labels can be confusing. By any name, progressives can boast of many honorable accomplishments, some of their own instigation (universal health care and state pension plans such as Social Security) and others instigated by liberals (women’s suffrage and the thwarting of racial and sexual discrimination).
As the triangle illustrates, the distance from conservatism to liberalism is no greater than from conservatism to progressivism. This helps explain why a conservative can be liberal in certain respects, such as by upholding free markets or opposing the jailing of drug abusers, and why a classical liberal may be otherwise conservative (“neoconservative”) or progressive (“neoliberal”). It also clarifies the previously puzzling fact that many hard left progressives have borrowed doctrines and terminology from the hard right. As many demagogs have demonstrated, it is possible to oscillate between the left and the right without ever approaching liberalism.
The political triangle is useful as well for visualizing the process of political consensus building. Consider the question of how to find sufficient votes in Congress to pass a bill aimed at reducing carbon emissions to combat global warming. One option is to take a command-and-control approach, empowering the Environmental Protection Agency to regulate limits on carbon emissions, and fining or shutting down power plants that exceed the mandated limits. That appeals to progressives, since it uses the government’s power to get rapid results, but is unpalatable for conservatives, since it would be bad for business, and for liberals, since it reduces personal freedom and enhances the power of government. More likely to gain support is a cap-and-trade approach that sets limits while establishing a carbon market in which the greener factories earn credits which they can sell to firms whose dirtier operations put them over the cap. Such a scheme has a reduced impact on business (which appeals to conservatives), and a smaller government role (appealing to liberals). Hence the triangular model predicts that the cap-and-trade bill is more likely to be enacted.
Liberals are deaf to the popular opinion that everything would be fine if everybody would just do what one individual thinks they should do. They oppose that stance on principle and also on the basis of historical fact. The modern world has extensive experience with all-powerful leaders—among them Hitler, Stalin, Mao, and the Kim dynasty in North Korea—who got their way, promising wonderful results in return for the sacrifice of their subjects’ liberties. These promises went unfulfilled. The divisions imposed on Korea and China more than a half century ago created the nearest thing to scientific experiments to be found in human affairs on such a large scale. In North Korea, decades of strongman control—he gets his way, we do not—produced a per capita GDP circa 2008 of $1,700, while the figure for South Korea was $26,000. Communist China’s per capita GDP that year was a sixth that of neighboring Taiwan. The communist regimes were unable even to consistently feed their own citizens, with millions dying of famine while North Korean boys at age seven stood, on average, five inches shorter than their South Korean counterparts.
It can be objected that the results of social experiments are contaminated, inasmuch as no nation has ever had a purely liberal or a purely progressive government. That certainly is the case: Even the most stringently socialist states have found some room for free enterprise, if only in the form of flourishing black markets operating under the radar of the controlled economy, while the United States—a wild-and-wooly bastion of free enterprise, where the word Liberty appears on every coin—has Social Security and a vast federal bureaucracy. But it is not necessary to have unsullied examples of conservative, socialist, and liberal societies in order to make rough yet reasonable determinations of what does and doesn’t work. The modest proposal of those who favor liberalism and science is not to subject people to involuntary experimentation of any sort, but simply to assess the empirical successes and failures of the innovations wrought by free peoples.
In practice, democratic nations constantly slosh around within the triangle, tending toward conservatism, liberalism, or progressivism at various times and for various reasons. Their short-term motivation may amount to little more than disenchantment with the status quo and a desire to “throw the bums out,” but cultural differences play a major long-term role. High taxes and socialistic programs are more popular in nations that have ethnically homogeneous populations, such as Sweden, than in ethnically diverse nations such as the United States. (It may be easier to think of your compatriots as one big happy family if they all look like members of the same family.) Within a given nation, sociologists find that city dwellers tend to be more socialistic than those in the countryside—and that it’s not a matter of voting one’s pocketbook. In the United States, the rural red counties that vote Republican are net consumers of federal dollars, while the blue cities that vote Democratic contribute more federal taxes than they consume; if the so-called urban elites were to change course, voting to cut taxes and slash federal government spending, the people worst impacted would be the rugged individualists in the red counties.
Liberal democracy normalizes such differences by holding elections, but when political parties are swept from office the programs they created frequently survive. The resulting proliferation of costly programs that have outlived their usefulness (or never had any to begin with) can be reduced by building evaluative feedback loops into the legislation that creates them—as the U.S. Congress and other legislatures have been trying to do, with the Department of Education, for example, spending $100 million a year to see which learning programs work best. It remains to be seen whether improved social-scientific evaluations can prevail against political pork, but governments do at least now have better tools for determining whether programs are doing what they were chartered to do. Statecraft used to be like driving at night with the headlights off; today legislators have headlights that, though flickering and dim, are better than nothing.
When liberalism is evaluated quantitatively, via the social science of economics, it comes off rather well. By fostering individual creativity and free markets, liberalism has created enormous prosperity—but it seems that a dash of government involvement helps, too. Party politics may be a crude metric, but the United States in the past half century has experienced faster GDP growth, lower unemployment, and higher corporate profits during Democratic than during Republican administrations. The stock markets performed better, too, with annualized returns on investment averaging almost 9 percent when Democrats were in office against less than 1 percent for the Republicans. It is not yet possible to identify exactly which conditions best bolster a free-market economy’s production of wealth—given the internal differences in the geography, history, ethnicity, and other conditions of various nations, there may be no one best way to set the dials—but it is clear that almost everyone is economically better off in the liberal-democratic, free-market states. Eighteen of the world’s twenty largest economies circa 2007 were liberal democracies—the exceptions being China, a communist state which boosted its economy into double-digit growth rates by admitting free-market practices, and Russia, which was foundering between democracy and oligarchy on an ocean of petroleum profits. That same year, all forty of the world’s cities said to offer the “highest quality of life” (measured in terms of thirty-nine factors ranging from recreational opportunities to political stability) were in liberal-democratic nations. It also appears that liberalism really does move humanity toward universal peace: Seldom do liberal democracies make war on one another, and as the world has become more liberal it has also become more peaceful.
But how, exactly, does liberalism facilitate scientific inquiry, and science benefit liberalism? Primarily through change and creativity.
Liberalism stands out among political philosophies in its readiness to embrace change. Liberals do not pretend to know what the future will bring, and so are skeptical about planning. They stress the importance of individual creativity, noting that humans are profoundly ignorant and so must be free to keep learning. As the liberal economist Friedrich Hayek declared, “All institutions of freedom are adaptations to this fundamental fact of ignorance, adapted to deal with chances and probabilities, not certainty. Certainty we cannot achieve in human affairs.”
Prior to the rise of science and liberal democracy, people had few choices other than to put their faith in a regnant belief system (which was girded against dissent by the power of church and state) or reason their way to an idiosyncratic system of their own (which unless done discretely could land them in jail). Science opened up a third option. Scientific research, whether conducted by an eccentric loner or a club of propertied gentlemen, could adduce facts of universal validity—facts objectively verifiable from every point of view. Such phenomena are called invariant, meaning that they are the same regardless of the perspective from which they are examined. All scientific laws are statements of invariance, and as such transcend both official dogmas and individual subjectivities. Liberalism’s claim that all humans have equal rights is therefore mirrored in the universal validity of scientific facts. “Far from being relative truths, scientific results tend to make everyone’s truth property the same across cultures,” noted the American philosopher Robert Nozick. “In this sense, science unifies humanity.”
Some thinkers accept that science finds facts but imagine that science can say nothing about values. Nozick summarizes their argument in terms of a syllogism: (1) science is objective, (2) values are not objective, so (3) values have no place in science. But is it true that values are nonobjective? The fact that the majority of cultures, while differing in many particulars, take quite similar positions on fundamental moral and ethical values like prohibiting murder and incest, suggests that there may be a universal, evolutionary basis for human ethics and values. As Nozick notes, “Science could contain values, and even make value assertions, and still be objective if values themselves are objective.”
The rapid changes in knowledge, power, and preference produced by the rise of science and technology have spotlighted the value of creativity. Back when the pace of change was glacially slow, the average person had scant opportunity to imagine, much less live, a life greatly different from that of his parents and grandparents: If you were the child of a peasant or a serf you almost certainly were going to remain in a similar station. Science and liberalism changed this dynamic. They opened up economic opportunities—if you could invent an improved clock or steam engine, or discover the universal laws of gravitation and inertia, you could become rich and famous regardless of whether your father was a carpenter, as was James Watt’s, or a farmer, as was Isaac Newton’s—and created a dynamic of progress that benefited not just the inventors but the general public. “In an advancing society,” noted the mathematician H. B. Phillips, “any restriction on liberty reduces the number of things tried and so reduces the rate of progress. In such a society freedom of action is granted to the individual, not because it gives him greater satisfaction but because if allowed to go his own way he will on the average serve the rest of us better than under any orders we know how to give.” This wasn’t clear in the old days, when few individuals enjoyed many benefits from creativity beyond hearing a stirring sermon from the pulpit or a new song produced by a bard from a faraway court. Today, the value of individual creativity is much more evident: Starting with the rise of literacy and libraries, and now expanding in a world of mobile phones and the Internet, people can appreciate that their well-being is enhanced by the creativity of others, and that the world’s total expertise far exceeds the personal understanding of any one individual. Hence the benefits of everyone’s being free to come up with new ideas and inventions become increasingly clear, even if most of their work is often too specialized for the rest of us to comprehend. Liberalism fosters science, which expands the intellectual and material universe, and liberalism can best cope with the changes that it and science have engendered.
Although thinkers long assumed otherwise, freedom is efficient. Perhaps it was the machine age, with its picture of society as resembling a factory run by a boss from a windowed aerie high above it all, that misled so many on this point, but by now it has become clear that the world is far too complex to be run by individuals or by committees of experts. No leader can assimilate enough information to accurately price tomatoes, much less chart the course of scientific or social advance. So the liberal ideal of peoples being free to decide matters for themselves turns out to have practical value—and the more complicated society becomes, the more apparent become the benefits of liberty.
Democracy has always been less popular than liberalism—if liberalism is a gift, democracy is the rattletrap truck that delivers it—and the two have distinctly different origins. Liberalism arose as a matter of principle, while democracy arose more for sociological than for intellectual reasons, and was always more about power than ideals. As Mill observed in 1859, the movements of societies toward democracy “are not the work of philosophers, but of the interests and instincts of large portions of society recently grown into strength.” Prominent among them were what the English called the gentry, a class whose wealth and influence came largely from scientific and technological innovations and free-market economies. They were trades and crafts people who gained power by gaining wealth, clawing their way to political prominence on the votes of previously disenfranchised multitudes who saw them as opening up fresh opportunities for all—which may explain why the poor, though impatient with snobs, demonstrate little animus toward the self-made rich. The process was not pretty, but it worked. “It could be said of democracy,” writes the historian Roland Stromberg, “that all theory was against it and all experience for it.”
Experience with democracy having been in short supply for most of human history, the great mass of humankind was long thought too ignorant, stupid, or distracted to make sound judgments. Intellectuals holding antidemocratic views took their cues from Plato, who regarded the people as a “beast,” and were inclined to agree with Sir Thomas Browne that “the multitude” is the “great enemy of reason.” Such opinions were thick on the ground for many centuries. “The multitude is always in the wrong,” sniffed Wentworth Dillon, the Earl of Roscommon, in 1684. “The public has very crude ideas,” opined Stendhal. Locke feared that the people were mired in “passion and superstition,” and apt, as Voltaire wrote, to “speak without thinking.” Jean-Jacques Rousseau asked rhetorically how “a blind multitude” could “carry out for itself so great and difficult an enterprise as a system of legislation?” The susceptibility of mobs to passionate irrationality, detailed in Charles Mackay’s 1841 bestseller, Extraordinary Popular Delusions and the Madness of Crowds, was thought to demand the steely leadership celebrated by the poet Alfred Tennyson as “civic manhood firm against the crowd.” Education wouldn’t be much help if, as Henry David Thoreau assured his readers, “The mass never comes up to the standard of its best member, but on the contrary degrades itself to a level with the lowest.” The historian Thomas Carlyle spoke for the majority of thinkers in declaring, “I do not believe in the collective wisdom of individual ignorance.”
They had a point. The public can be crude, and sullen in the face of change. (Creative artists frequently encounter this problem; as Bob Dylan remarked, “You’re nobody if you don’t get booed sometimes.”) But there were at least three serious flaws in the traditional derogation of the multitude. First, the intellectuals were educated and the masses were not. It is misleading to compare the ignorant to the learned unless the ignorant have a choice in the matter—which is why liberalism stresses the importance of universal education. Second, to object to the liberal precept that all persons are created equal, on the grounds that everybody is not equally intelligent, strong, or creative, is to miss the point. Liberalism does not maintain that everyone is equal in any particular regard. What is meant is that everyone has—or ought to have—equal standing as citizens, because the strength of the society resides in the very diversity of their abilities. If a nation required only a few talents, then it might perhaps be justified in conferring a superior standing to those who were the best accountants, carpenters, brewers, or whatever else was needed. But no such nation exists, and even if it did it could not be sure that the talents it valued today would be the most useful tomorrow; in diversity there is strength, both at present and in the unknowable future. Third, the empirical evidence acquired with the spread of democracy has failed to support the dire views of democracy’s critics. Democracies have, on the whole, compiled better performance records, by just about every agreed-upon quantitative measure of such matters, than any other system of governance.
But why is this so? Why does democracy work so well?
Social scientists are beginning to glimpse part of an answer, in what is known as the wisdom of crowds.
That story begins with an event in 1906 that may assume a place in the annals of scientific folklore, although only one among the many persons present saw anything remarkable in it. The scene was a weight-guessing competition held at the annual West of England Fat Stock and Poultry Exhibition, in Plymouth. Wandering through the crowd was a snowy-haired polymath named Francis Galton. This not-quite-eminent Victorian had done a number of things, several of which showed sparks of originality although none was likely to earn him a place in scientific history. A prodigious child—he was reading and doing arithmetic by age two and memorizing Shakespeare by age six—Galton never quite settled down. He studied a little medicine and took up mathematics at Trinity College, Cambridge, but abandoned these studies in 1844 when his father died and left him financially independent. A decade of gentlemanly exploration followed, the young Galton traveling in Egypt and filling his notebooks with advice on how to pitch a tent, steer by the stars, fire a rifle from horseback, and sleep safely when being pursued (tie your horse’s reins to your wrist, so that if the horse stirs you will be awakened). Thereafter he settled in South Kensington to study heredity, meteorology, psychology, and criminology. (His was the first proper study of fingerprints.) A repressed, eccentric, and rather solitary man, Galton belonged to various scientific organizations but never held any professional post. He admired beautiful women from afar—literally so, using a sextant to calculate their measurements. (“I surveyed them in every way…and tabulated the results at my leisure.”) His marriage—to Louisa Butler, the daughter of Erasmus Darwin, who was Charles Darwin’s grandfather—produced no children.
Galton was fascinated (if also from afar) by breeding. Poring over his in-law Charles Darwin’s Origin of Species, he concentrated on its many accounts of animal husbandry. Most readers were unexcited by these parts, which contain discursions on “peculiarities in the silkworm,” “the many admirable varieties of the strawberry,” and “the former and present state of carrier and tumbler pigeons in Britain, India, and Persia,” but Galton saw in them a blueprint for social engineering. He feared that democracy might not work, given that “the stupidity and wrong-headedness of many men and women [is] so great as to be scarcely credible.” His solution was to isolate what he called “degenerates”—of whom, in his view, there were a great many—to prevent their having children. “I think that stern compulsion ought to be exerted to prevent the free propagation of the stock of those who are seriously afflicted by lunacy, feeble-mindedness, habitual criminality, and pauperism,” he wrote, proposing that this be done in “ways yet to be devised that are consistent with a humane and well-informed public opinion.” It was his conviction that “a democracy cannot endure unless it be composed of able citizens; therefore it must in self-defense withstand the free introduction of degenerate stock.” Galton coined a word for this proposed practice; he called it eugenics, from the Greek for “well born.” Stillborn as a science and self-evidently illiberal, eugenics rose to notorious practice in the twentieth century, when it was taken up by British socialists, German Nazis, and a strange combination of American progressives and racists. (The U.S. Supreme Court in 1927 upheld a Virginia law requiring the sterilization of those declared mentally defective, while southern laws prohibiting interracial marriage were not struck down until 1967, Chief Justice Earl Warren writing for the Supreme Court that “the freedom to marry is one of the ‘basic civil rights of man.’”) But if eugenics today is little more than a disreputable memory, for Galton it was a bright prospect.
Such was the background of the eighty-five-year-old amateur scientist who found himself at the West of England Fat Stock fair that fall day in 1906. There he watched as nearly eight hundred people paid sixpence each to guess the weight, slaughtered and dressed, of an ox on display. The winner came pretty close. “The sixpenny fee deterred practical joking,” Galton noted, “and the hope of a prize and the joy of competition prompted each competitor to do his best.” Intrigued, Galton obtained the entry tickets and subjected them to statistical analysis. Adept at quantitative thinking, he was open-minded enough—though his scientific career seemed to be over and his memoirs had already been published—to make a discovery that ran counter to his beliefs.
Galton found that while no one competitor had come up with the exact weight of the slaughtered and dressed ox, and while many were wildly off, the mean value of all the entries was astonishingly accurate. The ox’s weight was 1,198 pounds; the crowd as a whole had estimated it to be 1,197 pounds. “The vox populi”—the voice of the people—was “correct to within one percent,” Galton reported. “This result is, I think, more creditable to the trustworthiness of a democratic judgment than might have been expected.”
Many other examples of the wisdom of crowds have since been adduced. Ask fifty students to guess the number of jellybeans in a jar, and you’ll usually find that the class as a whole not only comes quite close to the right answer but is closer than any one of the students. On the television quiz show Who Wants to Be a Millionaire?, the purportedly expert consultants telephoned by contestants got the right answer 65 percent of the time, while the studio audience—“random crowds of people with nothing better to do on a weekday afternoon than sit in a TV studio,” as the journalist James Surowiecki writes in his book The Wisdom of Crowds—got the right answer 91 percent of the time.
Predictions markets—Web sites where thousands of people invest in predicting election results and other unambiguous future events—routinely outperform the experts. The Iowa Electronic Markets bested 596 opinion polls in predicting presidential elections from 1988 to 2004, 74 percent of the time. The Dublin-based Intrade.com accurately forecast not only the 2004 and 2008 American presidential elections but congressional results in all fifty states. Futures markets for orange juice concentrate predict Florida weather better than the National Weather Service does. Such markets have proved to be so useful that corporations literally capitalize on them. Arcelor Mittal, the world’s largest steel maker, uses internal markets to predict the price of steel. Best Buy invites its employees to bid on which video game consoles will sell best. Invested crowds perform as well or better than experts in predicting the gross ticket sales of feature films and in categorizing craters on Mars. Encouraged by such results, corporations put puzzling problems on the Web and offer prizes to anyone who can solve them. When Eli Lilly created a Web site called InnoCentive to “crowd-source” problems that had stumped its vast R&D teams, more than a third of the problems were solved; since spun off, the site has overseen the invention of a method for dealing with oil spills, identification of an obesity biotarget, and a low-cost way of processing spices in developing countries. Everybody knows how hard it is for bettors to beat the odds on professional football games, but that is because the odds are established by the bettors themselves: The Las Vegas bookies establish only the initial line, a task they generally out-source, and thereafter adjust it to keep equal numbers of bettors on either side of the predicted outcome. So the individual bettor is up against not the experts, but the combined predictive power of all the other bettors. As Galton’s discovery suggested, the public not only has the right to make decisions but very often is right.
A great deal remains to be learned about how crowds can demonstrate superior predictive power, but a few precepts are already emerging. First, the outcome must be unambiguous, as when predicting election results or guessing the number of jellybeans in a jar. Second, the group members should function independently of one another, making their own determinations rather than being influenced by others; mob rule doesn’t work. Third, it helps if the group members are invested in the process, as happens when people purchase calls on Internet prediction markets; this was the point Galton grasped, noting that the price of tickets at the ox-weighing competition “deterred practical joking.” Finally, and quite interestingly, the accuracy of group predictions improves with its diversity: The more socially, ethnically, sexually, and intellectually diverse the composition of the group, the better it performs.
The discovery of the wisdom of crowds tends to validate liberal faith in free democratic processes, providing clues as to why liberal-democratic systems of governance outperform the competition. Liberalism may have inspired stirring rhetoric (“Give me liberty or give me death!”), but in essence it was an experiment—one that many skeptics predicted would fail. Today, with a growing body of evidence suggesting that liberal democracy makes people content, and their nations prosperous and peaceful, it is no longer necessary to rely upon moral and ethical grounds to argue, say, that the United States Senate would do better if it contained more women and minorities. The scientific data suggest that a more diverse Senate would be a more intelligent and aware Senate.
To appreciate how differently things look from this new perspective, consider Francis Galton’s eugenics. While Galton was wandering around livestock festivals, a number of younger scientists and social reformers were pushing his eugenics. Their motive was understandable: Darwin had shown that heredity plays a significant role in human behavior, yet reformers out to ameliorate the plight of the poor tended to focus entirely on environmental concerns such as poor housing and inadequate medical care. Eugenicists like Karl Pearson, who took over Galton’s Eugenics Records Office and renamed it the Francis Galton Laboratory for National Eugenics, argued that environmental improvements were a waste of money if the problem was due to genetics. Why build public housing and fund public hospitals if the suffering of the poor, indeed the very existence of poverty, could be remedied by isolating, sterilizing, or euthanizing the “defectives” who were, through no fault of their own, genetically responsible for their own suffering? This campaign eventually foundered in the face of liberal distaste at the prospect of the poor and helpless being treated like cattle, but questions have persisted as to whether there might have been something to it.
It is now possible to understand why the answer is no. At the time, one clear objection to eugenics was that nobody could know which genetic characteristics ought to be suppressed. The example often offered was alcoholism: Given a prevalence of alcoholic actors, poets, and painters, who could be sure that breeding out alcoholism might not also reduce a society’s artistic creativity? To this hesitation may now be added a stronger argument, drawn from the importance of diversity in the wisdom of crowds. Even if scientists and politicians could agree on which putatively undesirable characteristics ought to be eradicated, the effect would be to reduce the diversity of the whole—and that would reduce the overall intelligence of a necessarily diverse multitude. Selective breeding is not appropriate to human beings because the strength of humanity resides in its diversity rather than in any specific characteristics that a breeder might select for. Liberalism was right about that after all.
The legacies of science and liberalism include our sense of what it is to be modern. The historian of science Gerald Holton defined a modern individual in terms of four criteria: “Being an informed participant citizen,” “having a marked sense of personal efficacy (being able to control one’s own destiny and events in the world),” “being highly independent and autonomous,” and “being open to new ideas and experiences.” Despite opposition from many philosophies and almost all ideologies, science and liberalism have made real progress in all these regards, as measured against prior experience rather than an imagined ideal: The Earth is a planet, not a paradise. Humankind is much closer to the beginning than to the end of this endeavor; in many quarters, ignorance and superstitions continue to hold sway over knowledge and freedom. A flame has, however, been struck, the fire is being fed, and the cold and the dark are receding.