Chapter 1

What Is Confidence?

Confident people change the world. Consider Elon Musk, the South African immigrant to the United States who, as a young man, helped create PayPal, transforming the way people pay online. When PayPal was acquired by eBay in 2002, Musk used the money to invest in Tesla, the electric car company that upended the automobile market. Tesla’s market capitalization now makes it worth almost as much as General Motors. At the same time, Musk created SpaceX and revolutionized the business of launching satellites into space. He has announced plans for SpaceX to send humans to Mars in 2024 and, from there, to colonize other planets. Musk does not lack for ambition.

Musk’s fascination with space exploration traces back to a youthful love of science fiction. His brother, Kimbal, remembers that, as a child, Elon would read for as many as ten hours a day. When he ran out of books to read in his school’s library, he just started reading the encyclopedia. His sharp concentration and relentless work ethic were evident years later at Zip2, the company that Musk founded with Kimbal. Musk routinely worked late into the night. He would often wind up falling asleep in the beanbag chair next to his desk. “Almost every day, I’d come in at seven thirty or eight a.m., and he’d be asleep right there,” recalled Jeff Heilman, one of Zip2’s first employees. Musk would wake up and get right back to work. “Maybe he showered on the weekends,” Heilman speculated.

As a child, Musk’s bookishness did not always endear him to other children. “But Mom, he’s not fun,” his siblings objected when their mother implored them to include Elon in their play. Musk’s social awkwardness continued into adulthood. Doris Downes, one of his colleagues at Zip2, recalls: “Someone complained about a technical change that we wanted being impossible. Elon turned and said, ‘I don’t really give a damn what you think,’ and walked out of the meeting. For Elon, the word no does not exist, and he expects that attitude from everyone around him.”

SpaceX has transformed the business of launching rockets by bringing the price down to a fraction of what it used to be. Musk achieved this, in part, through relentless innovation. He saw expensive waste built into the deals between the US government and the contractors who had been building its rockets for decades. Musk did better by pushing the company and his people to pursue ambitious goals for both production timetables and costs. He worked hard to accomplish these goals and had high expectations of those who worked with him. Kevin Brogan, employee number twenty-three at SpaceX, recounts, “He doesn’t say, ‘You have to do this.’ He says, ‘I need the impossible done by Friday at 2 p.m. Can you do it?’” And often Musk’s employees do.

Musk’s story is just one example of the close tie between confidence and success. More-confident entrepreneurs, like Musk, are often more successful. Confident applicants are more likely to get hired, and confident political candidates are more likely to get elected. All around us, we see confidence precede success, which makes it tempting to conclude that cultivating confidence increases the odds of success. But focusing on those who wound up as winners is problematic, because it ignores two problems.

The first problem is that it risks confusing cause and effect. Is confidence really the cause, or is it possible that it is merely a consequence of something deeper—actual talent, financial advantage, or strategic positioning? For instance, strong job applicants with great credentials and a proven record of success have good reasons to be confident. Elon Musk is an enormously intelligent and talented person with many impressive achievements to his credit. He has a lot of money, tremendous power, and good reason to be confident about his potential success. In many cases, both confidence and success may share the same underlying cause.

Sports is another domain in which confidence is closely associated with success. It is easy to think of confident and successful athletes, such as the talented LeBron James who, at the tender age of sixteen, had CHOSEN 1 tattooed on his back. Did the same confidence that led him to make such a bold claim at such a young age also contribute to his many subsequent successes? An answer to that question should consider other sixteen-year-olds and their own claims to greatness. In a sport like basketball, in which boasting has been elevated to an art form, plenty of big talkers have not experienced NBA success.

The second problem with focusing on confident winners is that it overlooks the instances in which confidence has preceded failure. Plenty of confident people fail. It is the same confident Elon Musk who has experienced great successes and great failures. In 1996, he was ousted as CEO of Zip2. Four years later, he was ousted from PayPal. The first rockets launched by SpaceX blew up disastrously. The first Falcon 1 launch failed only twenty-five seconds after takeoff. The second launch, a year later, flew for four minutes before breaking up. Tesla, too, has had more than its share of trouble. In May 2016, Musk announced plans to produce 200,000 Model 3 sedans by the end of 2017. In fact, the company produced only a tenth of that number. Employees at Tesla were working overtime to ramp up production, but it just wasn’t happening fast enough. “I’m back to sleeping at the factory,” Musk tweeted in April 2018. “Car biz is hell.”

CONFIDENCE INTERVALS

To help you calibrate your confidence, I would like to invite you to play a little game with me. The table that follows lists ten quantities of which you are uncertain. For each one, please identify your answer, with a 90 percent confidence interval. A confidence interval consists of two numbers, one below your best guess and one above your best guess. That range should be wide enough that you are 90 percent sure the right answer is somewhere between. Obviously, the surer you are, the narrower you can make the confidence interval. If the question was about the date of your own birth, you could make the confidence interval very precise indeed. The less certain you become, the wider you should make the confidence interval. Your challenge is to calibrate your intervals so that you are 90 percent sure the right answer is inside it.

One way to think about it is like this: Set the lower bound so low that there is only a 5 percent chance the truth is below it (and a 95 percent chance the truth is above it). Then set the upper bound so high that there is only a 5 percent chance the truth is above it (and a 95 percent chance the truth is below it). With only a 10 percent chance the truth is outside the range, you should have a 90 percent confidence interval.

Without consulting any reference materials or other people, please estimate 90 percent confidence intervals for the ten quantities in the table.

Lower Bound Upper Bound
1. World population, according to the US Census Bureau on July 17, 2019
2. Year in which Orville Wright took the world’s first powered heavier-than-air flight
3. Hourly wage that Steve Jobs paid designer Dean Hovey for creating Apple’s mouse
4. Maximum depth (below sea level) of the Mariana Trench in the Pacific Ocean
5. Total revenues of the Tesla corporation in 2018 (according to its annual report)
6. Year in which Daniel Kahneman won the Nobel Prize in Economics
7. Amount of money that Google paid to buy YouTube in 2007
8. LeBron James’s average points per game in his NBA career, as of July 2019
9. Year in which William James first taught a psychology class at Harvard University
10. Number of honorary degrees awarded to the author Maya Angelou

Did you really take the time to answer the questions? Please do. It gives you some skin in this game. It improves your ability to apply the insights offered here to yourself and to your own decisions.

For how many of these questions should the right answer have landed between your lower and upper bounds? Well, if you have calibrated your confidence correctly, then each one should have a 90 percent chance of hitting. Out of ten items, nine should be inside the confidence interval. Read on to discover the right answers. How many did you get?

If you are like most people, your hit rate was substantially below 90 percent. In fact, hit rates inside 90 percent confidence intervals are closer to 50 percent. By drawing your confidence intervals too narrowly, you are acting as if you are surer than you deserve to be that your knowledge is correct. Your judgments reveal overprecision. This phenomenon also occurs for other ways people specify their confidence. It is not isolated to obscure trivia questions and 90 percent confidence intervals. In fact, overprecision emerges in nearly all the tests of it psychologists have devised. People usually act as if they are surer than they should be.

Research on “flashbulb memories” illustrates the illusion of knowing. These are memories that feel as accurate and faithful as a photograph. Many people, for example, recall with vivid clarity the moment they first learned of the terrorist attacks of September 11, 2001. If your flashbulb memories are as accurate as you think they are, then the same incident should be remembered the same way by others who shared that moment. But they are not. When researchers cross-checked people’s recollections, they discovered that those who were present in one another’s flashbulb memories often had inconsistent recollections of what happened—but they were nevertheless absolutely convinced that their own memories were correct.

When the question is not “What is the world’s population?” but “How much steel reinforcement is needed to prevent my bridge from collapsing?” then calibrating confidence judgments may be a matter of life and death. An architect who is too confident in a bridge’s design could skimp on structural supports. You probably also don’t want the underconfident architect who doubles the cost of the building, adding expensive earthquake reinforcements to your office building in Minneapolis, where earthquakes are unheard of. What you want in an architect, what you want in an employee, what you want in a life partner, and what you want in yourself is good calibration: the wisdom to see the truth, and to be as confident as evidence justifies.

OVERCONFIDENCE

Overconfidence sits at the center of decision biases that lead to error and irrationality. Scott Plous wrote that “no problem in judgment and decision making is more prevalent and more potentially catastrophic than overconfidence.” Daniel Kahneman, who won a 2002 Nobel Prize in Economics for his research on cognitive biases, wrote that overconfidence is “the most significant of the cognitive biases.” These claims are built on a strong consensus among those who have studied overconfidence, underscoring the importance and pervasiveness of overconfidence in human judgment. It would not be an exaggeration to say that overconfidence is the mother of all psychological biases. I mean this in two ways.

First, overconfidence is one of the largest and most ubiquitous of the many biases to which human judgment is vulnerable. “Perhaps the most robust finding in the psychology of judgment is that people are overconfident,” write Werner De Bondt and Richard Thaler. Overconfidence has been blamed for the sinking of the Titanic, the nuclear accident at Chernobyl, the loss of the space shuttles Challenger and Columbia, the subprime mortgage crisis of 2008 and the Great Recession that followed, the Deepwater Horizon oil spill in the Gulf of Mexico, and more. Overconfidence may contribute to excessive stock market trading, high rates of entrepreneurial failure, legal disputes, political partisanship, and even war.

Overconfidence further earns its title as the mother of all biases by giving the other decision-making biases teeth. It is, if you will, a gateway bias. These other decision-making biases arise from the many simplifying heuristics that we use to navigate our complex physical, intellectual, social, and informational worlds. These biases are the focus of several other books on the psychology of decision making, including Daniel Kahneman’s Thinking, Fast and Slow, Dan Ariely’s Predictably Irrational, and my book with Max Bazerman, Judgment in Managerial Decision Making.

If you were appropriately humble about your judgment, you would be better able to protect yourself from the errors to which everyone is prone. Intuitive human judgment is vulnerable to many such biases and errors. The problem with intuition is that because it results from unconscious processes, you cannot audit it. It arrives, fully formed, in our conscious minds, and it just “feels right.” Some possibilities feel more likely, some people feel more charismatic, and some risks just give you a bad feeling. Your confidence in these intuitions means that you too often accept them and forget that intuitive judgment is not perfect.

I ask my students to rate themselves relative to others in their class using a percentile scale. This scale quantifies the percentage of others to whom they are superior. The very worst in the class should get a zero on the percentile scale. The very best in the class should get one hundred. The median person, right in the middle of the pack, gets a fifty. Half the class is better than that person; the other half is worse. If all the students knew exactly where they stood, agreed on how to measure it, and answered honestly, the average for the class would have to come out to fifty. Those who rate themselves as better than others when they’re not are demonstrating overplacement: the exaggerated belief that they place above others in the rankings. When I ask my students to rate their honesty, their answers average to around seventy-five.

I wouldn’t want to accuse my students of dishonesty for making this glowing assessment of their own honesty, but I might accuse them of being vulnerable to bias. To get their views on this issue, I also include the following item on the questionnaire:

Research has led psychologists to conclude that when people rate themselves in terms of socially desirable qualities or performance, they tend to see themselves as being better than average when they are really not. This tendency is often referred to as a “self-serving bias” in judgment. To what extent do you believe that you avoid this bias relative to other members of the class? Again, give yourself a percentile rank that rates your objectivity, relative to the other members of the class. A score of 100 would indicate that you are less vulnerable to self-serving biases than all the other members of the class. A score of 0 would indicate that you are more vulnerable to self-serving biases than all the other members of the class.

Average responses to this one are consistently above fifty.

When you, as an individual, make biased decisions, it can lead to costly mistakes. But when we, collectively, are biased, the consequences can be momentous. Many have noted the role of collective overconfidence in the lead-up to the 2008 financial crisis. The 2008 subprime mortgage collapse and the Great Recession that followed were caused by a unique set of circumstances, many of which are closely tied to overconfidence. I begin with the investors, banks, and sovereign wealth funds that were buying up mortgage-backed securities. Their willingness to buy these securities was based on their belief that they knew what they were worth. Their confidence appears, in retrospect, to have been excessive. Had it not been for these investors’ overprecision, they would have been less interested in investing in subprime loans.

As it was, the investing market was so eager for mortgage-backed securities that normal mortgages to worthy borrowers were not enough. Zealous brokers resolved to give the investors what they wanted and developed what have been euphemistically called “innovations” in the mortgage market: NINA loans, appropriate for people with no income and no assets. Then NINJA loans for people with no income and no job or assets. Next came “liar loans,” in which borrowers were invited to lie about jobs, income, and assets that could justify the loan. Mortgage brokers advertised that their loan approval process omitted the onerous step of verifying the borrower’s income. You’re an intermittently employed actor making half a million dollars a year? Right. We have just the mortgage for you.

How did intermittently employed actors afford payments on million-dollar mortgage loans? Many of these loans were structured so that the payments started out small but ballooned over time. That wouldn’t be a problem if (1) your income was going to go up dramatically in the future, or if (2) the property value kept going up and you could refinance with another liar loan in a year or two. But in 2007 it became apparent that borrowers had been overestimating the prospects of at least one of these. The number of people who were not even making the first payment on their brand-new mortgages climbed steadily that year. Only then did investors start to suspect that the risk models they had been relying on to estimate default rates might not be doing a perfect job of forecasting the risk of default in mortgage portfolios full of NINJA loans.

Investors all over the world had been buying up mortgage-backed securities as if they knew the risk of default by borrowers. Default is the great risk with any loan or bond. It is why the interest rates are so much higher on credit cards issued to people with questionable credit than they are on US government bonds. For the banks to make money on those credit cards, they need to charge enough interest to the people who pay that it can cover those who default.

Had it not been for the overconfident buyers of mortgage-backed securities, there would have been no incentive to create NINA, NINJA, and liar loans. There would have been no balloon payment plans. And all those mortgage brokers could have kept their more honest jobs as bartenders, construction workers, and exotic dancers. There would have been no boom in housing prices that drew otherwise sensible people into the business of “house flipping,” in which a house is purchased and then resold after being spruced up with a new coat of paint and fresh layer of sod that, with luck, will survive the escrow period until closing. There would have been no cottage industry of books, TV shows, and infomercials touting opportunities for those who wanted to make a quick buck selling real estate, flipping houses, or originating mortgages. Without the massive global market for mortgage-backed securities, there would have been no global housing bubble.

So what fed the demand for mortgage-backed securities? At the very bottom was overprecise beliefs in bad risk models. The banks acted as if they were certain that the rates of default on the mortgages in their portfolios would be below 5 percent, consistent with historical trends and data covering decades of mortgage lending and borrower default. And in their defense, I must enthusiastically agree with the absolutely fundamental importance of making such estimates based on good data. Investors placed enormous bets—totaling trillions of dollars—on banks’ risk models being right. But their estimates were overprecise because they failed to consider the important fact that the historical data on which so much rested did not include NINJA loans and their ilk. When some “low-risk” mortgage portfolios saw default rates exceeding 50 percent, it’s easy to see why the investments in these securities turned out to be worth less than everyone thought.

When the crucial characteristics of the underlying loans started to change, no one updated the risk models—until it was too late. So why didn’t the alarm bells sound sooner? There were certainly people on the ground who knew that the liar loans they were selling weren’t all going to get paid back. These mortgage brokers kept getting paid as long as the big investors kept buying the mortgages. Most of the brokers knew that at some point the music would stop and there wouldn’t be enough seats for everyone to sit down and get out of the maelstrom. “But as long as the music is playing, you’ve got to get up and dance,” said Citibank’s CEO Chuck Prince in July 2007. Many justified their continued involvement with the belief that they were smarter than the suckers who would get stuck holding the bag. “I’ll be gone, you’ll be gone” was what they told each other. Maybe some of them did manage to get out. For the rest, this belief represented overplacement: they thought they were cleverer than others, but they weren’t.

UNDERCONFIDENCE

Given the risks of overconfidence, you might think that it would be wise to reduce your confidence. But by how much? Too little confidence is a recipe for self-doubt, inaction, and (as the self-help books will tell you) missing out on an awesome life. Every day, you decline to strike up conversation with others, go rock climbing, or start a new company. Some of your failures to act are attributable to underconfidence, especially when taking a risk would actually pay off. These failures qualify as mistakes.

Underconfidence is rife, and in many cases it is the mirror image of overconfidence. The same students in my classes who claim to be more honest than their classmates and more immune to self-serving bias are perfectly willing to rate themselves as below average. On average, my students think they are worse at juggling than their classmates. They also underplace their knowledge of Latin, their ability to ride a unicycle, the number of companies they will found, and the number of lives they will save.

The psychologist Justin Kruger deserves the credit for identifying the fact that people tend to believe that they are worse than others at difficult tasks for which success is rare. In his dissertation research at Cornell University, Kruger showed how easy it is for people to underplace themselves. Ask them about some difficult task, a task on which most people perform poorly or fall short of some salient standard, and they will tell you that they are worse than average. Most of the students in my class cannot juggle very well. They think, “I know I’m no good at juggling, but maybe someone in here can juggle. If so, then I’m definitely worse than they are.”

This line of reasoning highlights precisely the situation in which people are most prone to underplace themselves: when they know about their own ineptitude but aren’t so sure about the abilities of others. Justin Kruger and his colleague Ken Savitsky noted that people display such underconfidence when considering their success not only at difficult tasks, but also rare behaviors. People think they use their shoehorns and waffle irons less than other people use theirs. If I hardly ever use my shoehorn, the reasoning goes, I am likely to be using it less than average. But when nobody uses a shoehorn very often, that can lead people to mistakenly infer they use theirs less than others.

The impostor syndrome leads competent people to fear that they are not good enough—that they are impostors. Thomas Jefferson, founding father of the United States, principal author of the Declaration of Independence, crafter of the US Constitution, the nation’s first secretary of state, ambassador to France, third president, prolific writer, speaker of many languages, and polymath, insisted that “more confidence is placed in me than my qualifications merit.” John Steinbeck, winner of both the Nobel and Pulitzer Prizes for literature, insisted, “I am not a writer. I’ve been fooling myself and other people.” Maya Angelou, celebrated author, winner of the National Medal of Arts, the Presidential Medal of Freedom, Grammy and Tony Awards, and twenty-two honorary degrees from prestigious institutions of higher education, confessed, “I have written eleven books, but each time I think, ‘Uh oh, they’re going to find out now. I’ve run a game on everybody, and they’re going to find me out.’” And the actress Jodie Foster worried about both her Academy Award and her admission to college: “I thought it was a big fluke. The same way when I walked on the campus at Yale, I thought everybody would find out, and then they’d take the Oscar back.”

The impostor syndrome was first named in a 1978 article that examined the degree to which the syndrome afflicts high-achieving women. The Confidence Code by Katty Kay and Claire Shipman reprised the theme, bemoaning female underconfidence and encouraging women to be more confident. The book features interviews with impressive women, such as Christine Lagarde (the former head of the International Monetary Fund), the US senator Kirsten Gillibrand, and the basketball star Monique Currie. When asked whether they ever experienced self-doubt, these capable and successful women confessed that they did. Did the men with whom they competed experience self-doubt? “For guys,” Currie offered, “all the way down to the last player on the bench, who doesn’t get to play a single minute, I feel like his confidence and his ego is just as big as the player who is the superstar of the team.”

Others’ inner doubts are invisible, making it easy to imagine that others are less afflicted by self-doubt than you are. Kay and Shipman did not ask men about their self-doubts for their book, and it is entirely possible that that last player on the bench is just putting a brave face on his feelings of inadequacy. In my research, I rarely find gender differences in confidence, nonverbal confidence expression, or interpretation of others’ confidence. I do find plenty of evidence that both women and men worry about their credibility, that they both experience self-doubt, and they are sometimes sure that they are worse than others, even when they are not. Research also establishes that those who feel the impostor syndrome most keenly are rarely those who ought to. Frauds, posers, and con men often get away with arrogant displays of enormous confidence, without possessing true competence. Instead, it is the most hardworking and conscientious in any organization who worry most about their ability to deliver what others expect of them.

You are most prone to feel like an impostor when others’ flaws and self-doubts are hidden. To pick one example: You have privileged information about how you look naked. It is tempting to believe that your naked body has more moles, stretch marks, and inconvenient sproutings of hair than do others’. The naked truth is that every body is imperfect, but you are simply more familiar with your own naked body than with others’. It doesn’t help that most published photos of naked bodies are those of young, beautiful, fit people with perfectly coiffed pubic hair and whose imperfections have been meticulously photoshopped out.

Interestingly, the very same psychological mechanisms that lead to underplacement can also lead to overplacement. My students think they are more honest than their peers because they know more about themselves than they do about others. They know they are honest and cannot know for certain how honest others are. Assessing honesty, after all, depends on confirming that a person says what he or she believes; for others, you can only speculate about what they truly believe. If my students reason, “If I know I’m honest ninety-seven percent of the time, there is more room for others to be less honest than I am, so I am probably more honest than average,” then they will exaggerate the degree to which they are more honest than others, and in turn overplace their honesty. Knowing that they brush their teeth regularly can likewise lead to the mistaken inference that they brush their teeth more often than others.

There exists another example with profounder consequences than underconfidence about one’s naked body or use of toothbrushes, and that is the enrollment of qualified students from low-income households into higher education. Only a third of the most talented high school graduates from the bottom quartile of the income distribution attend the United States’ top universities, compared with 78 percent from the top quartile of the income distribution. The single biggest reason for this difference is that the lower-income students just don’t apply. In part, they don’t know about the opportunities and financial aid available to them. But they are also overly pessimistic about their chances for admission.

Part of their pessimism could be caused by their not knowing others who have attended such institutions. They infer that it is unlikely to happen. But in truth, talented kids from poor families are very much in demand at elite universities. Most universities are eager to increase their student diversity in matters of race, gender, culture, geographic representation, and socioeconomic background. The most selective private universities help poor families afford their high price tags. That often means they will provide generous financial aid packages that make tuition free for families who cannot afford it. The eligible students who fail to apply could clearly benefit from a bit more confidence.

Another task at which many people feel inadequate is writing. Writing is devilishly difficult, and even accomplished writers frequently feel incompetent. Take, for example, Hugh Howey’s blog confession, “I suck at writing.” Howey, a bestselling author of eleven published books, wrote: “I assure you that my writing skills are well below par. Watching a rough draft emerge from my fingertips in real time would induce nausea. It’s a haphazard, drunken affair.” Writers are keenly aware of their own difficulties, but it is rare to obtain insight into others’ writing challenges. Instead, you see the completed text, bound handsomely and displayed in the bookstore. It all looks so beautiful and orderly and competent.

The brilliant writer David Rakoff knew better. He described sitting down at the computer to start writing just as toddlers were being dropped off at their day care next door. Despite earlier frustrations, the new day afforded optimism:

Today will be good, you think. Not like the previous day’s lack of industry, a shameful waste of phone calls, e-mail, snacking, and onanism. Yes, it is all about today. But first, the crossword. And what does Paul Krugman have to say? Oh, that Gail Collins. Love her. E-mail, has it been checked in the last forty seconds? And now a snack. Friend Patty calls . . . Midday already? The toddlers, now screaming, are picked up from next door. Sit down and write a sentence for God’s sake. One fucking sentence, it won’t kill you. It almost kills you.

Even when it doesn’t almost kill you, writing is hard for just about everyone. You probably deserve to be a little more confident about your writing abilities, compared with those of others, and a little less confident about your honesty. For confidence to be well calibrated, it must be matched by underlying facts, evidence, and competence. It means actually understanding how you place relative to others. It means separating your beliefs from assumptions, fantasy, or delusion. But delusion can be awfully inviting, as William James reminds us.

BELIEVING IN YOURSELF

William James, widely regarded as the father of modern psychology, was a brilliant intellect and a visionary scholar. James’s book The Principles of Psychology is a classic that garnered high praise from the most eminent psychologists of his day, including Carl Jung and Sigmund Freud. The book’s wide-ranging exploration of what psychology is and what it could be anticipated many of the most important topics that have engrossed the field to this day. His ideas and his theories have endured remarkably well, especially when compared with those of other, more famous psychologists (including Jung and Freud).

But James had a difficult start in life. As a boy, he suffered ailments of the body and the mind, from back problems to depression severe enough that it brought him close to suicide more than once. He wanted to be an artist but studied medicine out of practical necessity. He hated it. Of his time in medical school, he wrote, “I was, body and soul, in a more indescribably hopeless, homeless, and friendless state than I ever want to be in again.” He completed his degree in 1869 but never practiced medicine. Instead, James met the anguish of his debilitating depression with a search for meaning. “I drifted into psychology and philosophy from a sort of fatality,” he reported. His search revealed a fascination with the connections between the mind and the body. This led him to a faculty position at Harvard University in 1873, where he developed one of the very first psychology courses the university ever offered, entitled The Relations between Physiology and Psychology. “I never had any philosophic instruction,” James confessed, “the first lecture on psychology I ever heard being the first I ever gave.”

In 1878, James wrote about the power of positive visualization. He imagined himself climbing in the mountains and getting stuck at a spot from which he needed to take a “bold dangerous leap.” He wrote, “I may wish to make the leap, but I am ignorant from lack of experience whether I have the strength for it.” James describes two possible outcomes. In the first, he believes what he desires. He imagines that his confidence gives him the strength to make the leap successfully. The confident James believes in himself, he jumps, and he makes it.

The second outcome imagines self-doubt. This doubting James hesitates, wavers, and then, “weakened and trembling, compelled to take the leap by sheer despair, I miss my aim and fall into the crevasse.” James concludes that in a situation like this, “I should be a fool if I did not believe what I wished, as my belief happens to be a preliminary condition which is essential to the accomplishment of the end which it affirms.” In other words, James imagines a situation in which his beliefs make themselves come true. As such, a wise person would have faith, and that faith would bring success.

When I first read James’s account, I took it as a persuasive argument for the benefits of optimism. There are undeniably situations in which the belief in a positive outcome increases the chances that one will choose to go for it, and thereby increases the opportunity for the positive outcome one expects. If believing you can leap the crevasse increases the chance that you jump, it must also increase the chance that you make it. On the other hand, the fear of failure can easily scare you off from the attempt.

All parents have observed their children talking themselves into failing. Children are routinely reluctant to try new things: a new class, a new sport, a new food. “Try it, you might like it,” you encourage gently. The child insists that she can’t: “What if I mess up? What if I hate it?” Her imagined failure keeps her from trying, making her fear a self-fulfilling prophecy. Yet at other times, parents get to witness the thrill of a child jumping enthusiastically into a new project with both feet. When she does, her enthusiasm begets a persistence that increases her chances of success. Once again, her belief is borne out by reality.

One implication you might be tempted to take away is that positive visualization leads to good outcomes in life. You would not be the first to think so. However, you would also be right to be skeptical of this. While there is some evidence for the benefits of visualization, that evidence suggests that its value is limited. While simply imagining your company’s rocket successfully making it into orbit may be pleasurable, it will not directly increase the odds for a successful launch any more than visualizing yourself as the pope will transport you to Rome.

In one study of the effectiveness of visualization, psychologists instructed college students to visualize getting good grades on a midterm examination. When that instruction included visualizing the process of studying and preparing for the exam, it led to longer hours of actual study and to better exam performance. By contrast, when the visualization centered exclusively on the positive outcome—that is, the exam result—the researchers did not find that it increased either the hours spent studying or the student’s performance on the exam. For visualization to be effective, it has to influence the actual behavior that affects the fantasized outcome, such as studying, practicing, or working out. In case you’re wondering whether those subliminal audio tapes to which I listened as a teenager might have helped, they probably didn’t do anything for me or any of the other people who paid good money for them.

Further reflection on William James’s story of his alpine adventure raised additional concerns that I had failed to consider the first time. If the doubting James really expected to fall into the crevasse, then making the jump seems like a bad call. If it were me, I’d like to think I would explore other ways out of the predicament that were less likely to end in my death. But more important than that, if the moral of his story is that it’s always better to believe that you can jump the crevasse, that seems wrong. Sometimes you can’t jump the crevasse. Sometimes it is just too wide.

Perhaps the doubting James could jump a five-foot crevasse. Let’s say that believing in himself could get him a foot farther. Then if the crevasse were less than six feet across, James should repeat some empowering self-affirmations and go for it. But if the chasm were twenty feet wide, no amount of positive self-talk would get him across. Believing in yourself, if it prompts you to jump to your death, would qualify as a mistake—however much it displays an admirable confidence in your capabilities. It is an error that could be avoided with better-calibrated confidence. There are many domains in life in which confidence can help you perform, but its benefits will be limited. Believing in myself did not prevent my feet from getting burned on the fire walk. Setting impossible goals can be like leaping into a twenty-foot chasm: inspiringly ambitious but doomed to end badly.

THE RISKS OF OVERCONFIDENCE

“Pride goeth before destruction, and an haughty spirit before a fall,” predicts the book of Proverbs. There are many situations in which being too sure of yourself can indeed undermine the very success you envision, as the work of the psychologist Jeffrey Vancouver shows. He has examined circumstances in which an increased sense of personal self-efficacy impairs future performance. Such experiments require manipulations of self-efficacy. If researchers had simply measured self-efficacy and performance, they would find many reasons why the two correlate that had nothing to do with any causal effect of confidence. For instance, those who were capable at a given task because they had prior success would be both confident and successful.

Experimental evidence is required in order to answer the question of whether one should choose to be confident, since the question is about the causal effect of confidence. This is exactly what Vancouver and his colleagues did. In one study, research volunteers played the game Mastermind, which involves guessing the color and order of hidden pegs. Vancouver’s devious computer program rearranged the hidden pegs to make it easier or harder for participants to win the game. Those who easily won then became more confident in their own abilities, and they reported significantly higher feelings of self-efficacy. How did they do in subsequent rounds of play without the computer helping them out? They did worse than those who had lower feelings of self-efficacy. When performance depends on effort, being too sure of yourself can reduce your effort and undermine success.

The students in my class who are most confident that they will ace the exam, and who therefore do not study, are not those who get the best grades. The skydivers, mountain climbers, and bungee jumpers who are most convinced of their invulnerability are not those with the longest life expectancies. The psychologist Gabriele Oettingen has spent her career identifying the many ways in which imagining success can increase failure. She has examined people who want to lose weight, students who want to perform well on exams, and the lovelorn who want to find romance. She finds that those who fantasize most about a positive future do not actually obtain what they desire; on the contrary, they tend to obtain worse outcomes.

Michael Raynor writes about the threats overconfidence poses to businesses in The Strategy Paradox: Why Committing to Success Leads to Failure. The problem he points out is that prior success breeds complacency, and this complacency undermines an organization’s ability to respond to new market challenges. Some of this is due to the inertia that afflicts organizations as they grow and age. Successful companies are built on successful people, products, and processes. These people, products, and processes naturally resist change if they fear that doing so could undermine their future success, prestige, or influence. But success can also breed confidence and persistence, especially for those at the top of an organization. All of this makes it difficult for businesses to adapt to changing market conditions.

For a time, parents were encouraged to build their children’s confidence by telling them they could do anything. Now many developmental psychologists worry about the unintentional harm we may have done to our children by trying to pump up their confidence and increase self-esteem. In her book The Self-Esteem Trap, Polly Young-Eisendrath argues that affirming our kids with positive and empowering messages can lead to greater disappointment and disengagement. This message echoes that of the psychologist Carol Dweck, who maintains that by telling our children that they are smart, talented, and capable, we make them afraid to take chances that come with some risk of failure. Kids are aware of their own limitations and worry that by failing they might expose our confidence-boosting affirmations as false.

And what would Elon Musk the businessman have to say to William James the scholar? To what degree were Musk’s audacious goals a key to success at Tesla and SpaceX? Did Musk successfully make so many bold, dangerous leaps simply because he believed he could do it? When asked, Musk has insisted that goals work best when they are realistic and achievable: “I certainly don’t try to set impossible goals. I think impossible goals are demotivating.” He has also seen the chaos created by excessively optimistic goals, and he has resolved to do better: “I’m trying to recalibrate to be a little more realistic.”