Prediction is very difficult, especially if it's about the future.

—Chinese Proverb, Neils Bohr, Yogi Berra

We humans have a great desire to predict things. We want to know if we'll marry the person we just met, if we'll get that new job, if it's going to rain this weekend, or if the stock we just bought will skyrocket. Our desire to know the future permeates many aspects of our personal and professional lives. As we've seen, however, wanting something often biases our beliefs and decisions. It turns out that our strong desire to predict future events has led us to believe that we can predict things that are essentially unpredictable. And we spend considerable time and money trying to predict them.

PSYCHICS AND ASTROLOGY

The World Trade Center tragedy on September 11, 2001, was a defining moment in our history. We were shocked, emotionally drained, and outraged. In our search for information about the disaster, many people turned to the Internet. CNN reported that the top three Web topics on September 20 were, in order, Osama bin Laden, Nostradamus, and Afghanistan. Nostradamus was number two!

Our history shows that we have always had a desire to predict future events. Written records from five thousand years ago indicate that the ancient world was attempting to foretell the future by using everything from animal entrails to celestial patterns.1 Alexander the Great had his psychics read the insides of slaughtered animals to ascertain future events. This desire still finances a number of activities, from reading tarot cards, palms, tea leaves, and crystal balls, to listening to mediums who supposedly receive messages from the dead or some unseen power. Many people believe that psychics and astrologers who've been dead for centuries, like the sixteenth-century French astrologer Nostradamus, can predict today's and future events, while others listen to a host of more current foretellers. Major companies have even employed psychics in their personnel hiring decisions, and some police departments have used them in attempting to solve crimes.²

So is there any value to psychic predictions? People point to some pretty amazing success stories. For example, Jeanne Dixon supposedly foretold the assassinations of John Kennedy and Martin Luther King, while many people believe that Nostradamus predicted both world wars, the atom bomb, Hitler, and the 9/11 tragedy. When evaluating these predictions, a couple of things must be kept in mind. First, we have to ask ourselves, are the predictions unambiguous, and second, are the predictions more accurate than what we would expect from chance.

Consider the following prophecy of Nostradamus that many people believe predicted the 9/11 disaster:

Earthshaking fire from the centre of the earth,
Will cause tremors around a new city,
Two great rocks will war for a long time,
Then arethusa will redden a new river.³

It certainly seems applicable to the Twin Towers collapse in New York City, referring to “two great rocks” and “a new city.” In fact, researchers found that 68 percent of people surveyed thought that the verses might have predicted 9/11.⁴ But is that because they were looking for such evidence? To find out, the researchers gave the same prophecy to another group of people and asked if it might have predicted the London blitz, where Germany bombed London for fifty-seven straight nights during World War II. It turned out that 61 percent thought the prophecy referred to the London blitz, where “earthshaking fires” and “tremors” were experienced throughout the city.5 The bottom line is that the prophecy is ambiguous enough to allow multiple interpretations. To further prove the point, the researchers randomly selected lines from different prophecies. Amazingly, 58 percent of the people surveyed thought that this scrambled version accurately predicted World War II!

Even Nostradamus experts can't agree on the meanings of his predictions. For example, two noted experts have interpreted the same verses in very different ways. One expert thought that a certain prophecy predicted the role of Emperor Haile Selassie in World War II, while the other said it referred to Henry IV and the siege of Malta in 1565.⁶ In effect, the verses are so ambiguous and open to interpretation that anyone can read whatever they want into them, and so, they are veridically worthless.

In addition to considering their ambiguity, we also have to question whether psychic predictions are more accurate than what we would expect from mere guessing. Thousands of forecasters make millions of predictions every year. With such a large number, some of the predictions will be right some of the time. As we have seen, coincidences happen quite often because of the sheer number of events that occur. In some instances, a coincidence will take place when a psychic foretells an event and the event actually occurs. Many people interpret that happenstance as proof of psychic ability. Why? When we test the hypothesis—psychics can predict the future—we naturally attend to evidence that confirms the hypothesis. Thus, we focus on the few times that psychics may have been somewhat right, and disregard the vast number of times they were wrong.

Psychologists Scott Madey and Tom Gilovich interestingly demonstrated this biased reaction to data. They gave subjects a diary of a student who supposedly had prophetic dreams. The diary contained a number of the student's dreams, along with events that occurred later on in her life. Half of the dreams appeared to come true, while half did not. When subjects were later asked to remember as many of the dreams as they could, they remembered many more that came true. When it comes to prophecies and fortune telling, we remember the hits and forget the misses.⁷

As we've seen, however, we have to pay attention to all the evidence when deciding if psychic predictions are associated with future events. Consider how many times psychic predictions were way off the mark. Even if we forget about the ambiguity in the Nostradamus quatrains, he supposedly predicted that in 1999, “A great king of terror will descend from the skies,” and the War of Wars will be unleashed. Did it happen? Of course, we may not have noticed since he also said we would have been at war with the antichrist since 1973.8 Jeanne Dixon predicted in her 1969 book My Life and Prophecies that Castro would be removed by 1970, Spiro Agnew's career would prosper, America would have severe food shortages after 1979, and a comet would strike the earth causing earthquakes and tidal waves in the mid 1980s.⁹ Another popular psychic predicted in the year 2000 that Bill Bradley would win the presidential election and David Letterman would call it quits in 2001. Bush won the election and Letterman signed a $32-million-a-year contract in 2002 to continue his nightly TV show.¹⁰

We also have to ask ourselves, if psychic ability was real, why are so many major events not foretold? The Committee for the Scientific Claims of the Paranormal (CSICOP) noted that the biggest embarrassment for psychics in 1997 was their failure to predict the death of Princess Diana. No psychic predicted the Oklahoma City bombing, the World Trade Center tragedy, or the two Iraqi wars. And, if psychics can predict the future, why aren't they making millions in the stock market? They say they have no need for great wealth—but why don't they do something good for society and donate the money to needy charities? On close examination, psychic ability turns out to give us nothing more than interesting conversation.

Some people also believe that astrology can predict the future. As noted, even Nancy Reagan used an astrologer to determine the best time for her husband's presidential speeches and meetings with heads of state.¹¹ Astrologers claim to have many clients in large Wall Street investment firms, and some technical stock analysts willingly say that they use astrology to predict the market. You, as stockholders, may unknowingly be paying for that advice!¹²

Do you think there must be something to astrology because it's based on ancient wisdom? If so, just remember that the ancients also used to foretell the future by reading the entrails of animals. The Babylonians invented astrology and hepatoscopy. While astrology uses the alignment of the stars and planets, hepatoscopy makes predictions by evaluating an animal's liver. Does it make any sense to believe that animal livers can give us an indication of the future? Probably about as much as believing that planets a million miles away affect our personality and future.

There are over ten thousand professional astrologers in the United States today. So does astrology work? Numerous studies demonstrate that it does not. For example, one study gave thirty prominent astrologers the natal charts of 116 subjects. They were also given three different personality profiles for each subject—one was the subject's profile and two were selected at random. All the astrologers had to do was match the subject's natal chart to his or her correct personality profile, a job that should have been quite easy for these professionals. It turned out that they chose the correct profile only 34 percent of the time, exactly what we would get from simply guessing. In essence, the astrologers' predictions were no better than chance accuracy.13

So why do people believe in astrology, palm reading, and other psychic prognosticators? In addition to wanting to believe, a major reason concerns a well-documented phenomenon known as the Forer effect.¹⁴ Mentioned earlier, the Forer effect refers to the fact that we often see some of our own personality traits in very general personality descriptions. Essentially, we see an ambiguous, general description and think that it refers specifically to us. My favorite example of the Forer effect occurred when a scientist put an ad in a Paris newspaper offering free horoscopes. He sent the same horoscope to all one hundred and fifty people who replied, and amazingly, 94 percent said they recognized themselves in the description. I wonder how they reacted when they found out the horoscope was drawn up for a French serial killer!¹⁵

PREDICTING THE STOCK MARKET

October. This is one of the peculiarly dangerous months to speculate stocks in. The others are July, January, September, April, November, May, March, June, December, August and February.

—Mark Twain

While some people believe that fringy prognosticators like psychics and astrologers can predict future events, most of us think that such predictions are worthless. It turns out, however, that there are things that many of us think we can predict quite accurately, which, in fact, we can't. Remember my friend Chris? Like many people, he thought that if he spent the time to learn about the stock market he could make a killing by buying and selling stock. Many people believe they can beat the market, and are willing to spend considerable money for advice from the “experts.” So, can we predict the market? We saw earlier that technical analysts can't accurately predict future stock prices, but can anyone else do it? Let's take a look.

One of the largest professional groups that attempt to forecast the future are investment advisers. There are about two hundred thousand investment advisers, and nearly half are stockbrokers. Most all of them are involved in the $71 billion industry that attempts to predict the future movements of investments.16 Stock analysts are on TV, writing newsletters, and calling their clients to clue them in on the next hot stock pick. Do people listen to them? Just consider how you would react to the following information.

You get a letter in the mail saying, “Accurate stock predictions! Professional stock analysts have recently developed a groundbreaking technique, based on years of research, that's proven successful at predicting stock price fluctuations. These stock predictions will be published in a new financial newsletter that comes out every month.” The letter doesn't ask you to subscribe, but it gives you the newsletter's latest pick and asks that you check it out. It says that Macrotech's stock will go up next month. You don't think much of it, but at the end of the month, you notice that Macrotech did rise. Of course, your first reaction is “lucky guess!” The next month you get another letter predicting that Macrotech will go down in the following month. When you check the price that month you find that the stock did fall. A third letter predicts that Macrotech will increase next month, and, amazingly, it does go up. Now you're getting intrigued. You then receive a fourth letter predicting that Macrotech will rise again in the following month. A check of the price reveals that it increased once more. Unbelievable! At that point you get a letter asking if you're interested in subscribing to the newsletter for a mere $400 a year. It sounds like a good deal, so you jump on it.

Now just imagine how those letters could have been generated. Someone sitting in their kitchen could have sent out two thousand letters to different people in the phone book. Half of the letters said that Macrotech would go up and half said it would go down. After Macrotech went up that month, the writer sent letters to only those 1,000 individuals who were initially told that Macrotech would rise. Once again, half the letters said that Macrotech would go up and half said down. When Macrotech went down, letters were sent to the 500 people who were told it was going down, half saying it would rise and half saying it would fall during the following month. The next letter was sent to only the 250 people who received the last correct prediction, with half saying Macrotech will rise and half saying it will fall. Now, you happen to be one of the 125 individuals who received four correct letters in a row. With that track record you figure the newsletter is worth the $400—so you write a check. If the other 124 people feel the same way, the writer sitting in her kitchen has just made $50,000 by sending out a number of cheap form letters!17

Seems like a big scam, doesn't it? But think about the stock analysts who market themselves as being superior because they perform above average four years in a row. You often see advertisements touting the ability of a stock analyst to beat his peers in the financial newsletters and on TV. However, out of one thousand stock analysts, about sixty-two will perform above average four years in a row by chance alone. As Martin Fridson explains, this beat-your-peers fallacy “is repeated every day in the financial markets. Investment advisers regularly win new clients on the strength of performance records that are not demonstrably better than chance results. Market forecasters routinely persuade investors of their excellence by stringing together just a few correct predictions. If these representations of superior scale can be discredited so easily, why do people persist in relying on them? Investors simply do not understand the basic principles of probability.”¹⁸

Stock analysts imply that if we follow their suggestions we can “beat the market.” What does that mean? The market is comprised of thousands upon thousands of stocks, actively traded on exchanges like the New York Stock Exchange. While we're not going to invest in every stock, we can invest in an index fund, which is a fund that reflects the returns we would earn from a broad category of stocks. For example, one index fund mirrors the returns earned by the five hundred stocks included in the Standard & Poor's 500 listing. Index funds are not actively managed—an investment expert is not using her expertise to select the stocks she thinks are valuable, as is the case with the vast majority of mutual funds. Presumably, if stock analysts use their expert knowledge they should be able to do better (i.e., earn a higher return) than an index fund. If not, their advice isn't worth much. So the question is, Can the experts beat the market? Let's look at the evidence.

Fund Managers

One type of evidence comes from the performance of fund managers. These individuals spend their entire professional lives analyzing the market, and buying and selling individual stocks for the funds they manage. In 2005 there were over eight thousand such funds marketed.19 Do their managed portfolios earn a higher return than an index fund? It turns out that the majority of funds do not beat the average market return in any given year. Considerable research demonstrates this fact, going all the way back to the 1960s. Michael Jensen, a University of Rochester professor, demonstrated that mutual funds from 1945 to 1964 were “not able to predict security prices well enough to outperform a buy the market and hold policy.”²⁰ And that fact holds true today. If you just bought an index fund, which mirrors the returns offered by the overall market, you would have beaten 50 percent to 80 percent of the fund managers in the 1970s, 1980s, and 1990s.²¹

Now you may say, some funds do beat the market. Maybe those fund managers have superior knowledge that they can capitalize on. But the data argues otherwise. While some mutual funds have performed better than an index fund over short time periods, they typically can't achieve that superior performance over the long term. The top twenty performing mutual funds in the 1980s were understandably better than the S&P 500 index in that decade (18.0% vs. 14.1% return). However, those same funds did worse than the S&P index in the 1990s (13.7% vs. 14.9%). A more dramatic example of a fund's inability to stay on top can be seen in the last few years. Riding the Internet bubble, the top twenty funds in 1998–1999 achieved a 76.72 percent return, a staggering three times higher than the S&P 500 index of 24.75 percent. However, those same funds came crashing down to a -31.52 percent return in 2000–2001, a loss three times greater than the S&P index of -10.50 percent. The number one ranked fund in 1998–1999 (Van Wagoner: Emerging Growth) dropped in rank to 1,106, the second best fund (Rydex: OTC Fund; Investor Shares) dropped to 1,103, while the third best fell to 1,098. As can be seen, there is no consistency in superior performance. In fact, the average large cap mutual fund earned about 2 percent less per year than the S&P 500 index fund over a twenty-year period ending December 31, 2001!22

Remember our discussion on chance and coincidence? Before we start to attribute some cause for an event, we need to demonstrate that the event was not the result of chance. So can mutual fund performance be explained by chance? What if we selected a sample of one thousand mutual funds and analyzed the percent that would be in the top half in performance over a number of years by chance alone. If we had one thousand people flip a coin, about five hundred heads would come up. If these five hundred advanced to the second round and flipped again, about two hundred fifty would be heads. About one hundred twenty-five will be heads in the third round, sixty-three in the fourth, and thirty-one in the fifth. So by chance alone, 50 percent of the funds would be in the top half in year one, 25 percent would be in the top half two years in a row, while 12.5 percent, 6.25 percent, and 3.1 percent would be in the top half for three, four, and five years in a row, respectively. And so, by chance alone, 3 percent of funds should perform in the top half five years in a row. Is mutual fund performance significantly different from these results?

William Sherden analyzed the performance of all mutual funds with assets over $500 million between 1991 and 1995. For each of the five years he determined the number of funds that performed in the top half for two to five years in a row.²³ As can be seen in figure 10, 50 percent of funds were in the top half in the first year, 27 percent were in the top half two years in a row, 17 percent performed above average for three years, 4 percent for four years, and 3 percent for five years. These numbers are just about what chance would predict. As Burton Malkiel has stated, “The few examples of consistently superior performance occurred no more frequently than can be expected by chance.”²⁴

Figure 10. Comparison of mutual fund investment performance with flipping a coin. The graphs show the percentage of major mutual funds that performed better than average over five consecutive years, as compared to the percentage predicted by flipping a coin. (From W. Sherden, The Fortune Sellers: The Big Business of Buying and Selling Predictions, Copyright © 1998 by W. Sherden. This material is used by permission of John Wiley & Sons, Inc.)

The performance of so-called superior funds is no better. Analyses of the Forbes honor role funds over the period 1973 to 1998 indicate that they underperformed the S&P 500 stock index.25 The inability of experts to predict the market even led the Wall Street Journal to start a dartboard contest in the early 1990s, where the stock picks of four experts were compared to stocks selected by throwing four darts at a stock listing. By the late 1990s, the experts seemed to be ahead by a slight margin. However, if you measure the performance of the experts from a relevant date after their selections were announced in the Wall Street Journal (which is more appropriate since their predictions may affect stock prices), the darts were actually slightly ahead.²⁶

The above discussion concerns the performance of mutual fund managers, but don't think they're the worst of the lot. It turns out that other investment managers do no better. A number of studies have examined the performance of investment managers from insurance companies, pension funds, foundations, college endowments, state and local trust funds, trust funds administered by banks, and individual accounts handled by investment advisers. Their investment performance is no better. Of course, there are exceptions, but as Burton Malkiel notes, there is no scientific evidence to indicate that the performance of professionally managed portfolios, as a group, is any better than a randomly selected group of stocks.27

The Gurus

What about those few individuals who have attained considerable fame for predicting major movements in the stock market? One stock market guru amazingly predicted the 1987 stock market crash. William Sherden analyzed thirteen market predictions that she made between 1987 and 1996 (i.e., predicting that the market would either go up or down), and found that she was right only five out of the thirteen times, worse than what you would get from just flipping a coin. According to Sherden, she never made another long-shot forecast that turned out to be correct, and the fund she managed outperformed the stock market in only one of the six years it was in existence. While it's value increased by 38 percent, the S&P 500 index increased by 62 percent over the same period.²⁸

A number of gurus publish financial newsletters that purport to predict the market. While one of these gurus had some early success in the 1970s, he later advised his clients to sell everything when the Dow was in the 800s. The market proceeded to rise to about 1200. According to Sherden, the guru's performance over eight years, ending January 1994, was 38 percent below the market average. Yet another guru predicted the 1980s bull market. However, shocked by the 1987 crash, he stated that the bull market was over and that the Dow would fall to 400 by the early 1990s. Instead, the Dow climbed to 4,000 by 1994. His performance over the ten-year period ending December 1996 was 64 percent below the market average.²⁹ The bottom line is, people pay up to $500 per year for financial newsletters that help them pick stocks, yet an index fund beats the newsletter picks 80 percent of the time.³⁰

Frequent Trades Equal Poorer Performance

Given our belief that we can beat the market, many of us actively manage our own portfolios. We buy and sell stock frequently, attempting to capitalize on some hot issue. However, the data indicate that this is a bad strategy. Consider the following two statements.

 

(A) Between 1984 and 1995, the average stock mutual fund earned an annual return of 12.3 percent, while the average bond fund earned 9.7 percent per year.

(B)  Between 1984 and 1995, the average investor in a stock mutual fund earned 6.3 percent, while the average investor in a bond fund earned 8 percent per year.

 

Both of these statements are true!31 How can that be? It turns out that instead of investing and holding a fund for an extended period of time, most investors move in and out of funds on a regular basis.³² Oftentimes, investors move their money into a fund that has experienced good recent performance. However, statistics tell us that we have regression to the mean. That is, if a fund is currently outperforming the market, its performance is likely to drop in the future to bring it back to average. And so, if we buy into a fund right after it has posted recent gains, we're likely to be in for a fall. In effect, going after strong past performance often means we take money out of funds that are likely to rebound, and put it into funds that are ready to drop.³³

In fact, one study compellingly demonstrated that frequent stock trading results in poor performance. The stock investment performance of individual investors from over sixty thousand households was analyzed between February 1991 and December 1996. The average household earned an average return of 16.4 percent. More importantly, the top 20 percent of investors who traded the most earned an average return of 11.4 percent.³⁴ These individuals actively traded their stocks because they thought they could beat the market, when in fact, that trading actually resulted in poorer performance.

Why Can't We Predict the Market?

Many researchers argue that the reason we can't predict future stock prices is because the market is efficient. The efficient market hypothesis states that all publicly available information is quickly impounded into a firm's stock price, so we can't beat the market by finding over- or undervalued stocks.35 While there's considerable support for this hypothesis, there are also anomalies that make some people question whether markets are totally efficient. However, even if the efficient market hypothesis doesn't hold in all cases, that doesn't mean we can predict future stock prices. As Malkiel states, “although I believe in the possibility of superior professional investment performance, I must emphasize that the evidence we have thus far does not support the view that such competence exists.”³⁶

One reason for the unpredictability of the stock market is that, even when prices are rising, they don't increase in a gradual, steady manner. Instead, a few days a year have big gains, while the rest of the year is relatively flat. As Gary Belsky and Tom Gilovich state, “the stock market is much like that common description of war: long periods of boredom interrupted by episodes of pure terror.” Studies show that if you missed the forty best-performing days out of the 7,802 trading days from 1963 to 1993, your average annual return would have dropped from just under 12 percent to slightly above 7 percent.³⁷ The problem is, we have no reliable way to predict which days are going to be the top performers.

What causes these big shifts in the market? William Sherden offers a plausible explanation when he states, “The stock market is clearly driven by irrational herd mentality and mass psychology. Speculative binges cause stocks to surge to price levels way beyond their economic value in terms of future earnings potential. Panics cause the equally irrational effect in the opposite direction. The stock market is a psychological soup of fear, greed, hope, superstition, and a host of other emotions and motives.”³⁸

Just look at Black Monday, October 16, 1987, the day the stock market crashed. The crash came and went without any good cause. Compared to the preceding Friday, stocks were worth 30 percent less, and no real new information accounted for the drop. This irrationality suggests that the efficient market hypothesis doesn't give us the whole story about the market. A more complete explanation is that the market is a complex system that has both rational and irrational forces at work. As Sherden states, “While rational forces drive the market toward its fair value, irrational forces of speculation and panic cause the market to diverge from rational value. These irrational forces give rise to explosive nonlinearities that make the market unpredictable.”39 And yet, we are spending billions of dollars to predict it.

So what can we take away from all this? In some sense, the stock market is like flipping a coin. If we flip a coin 1,000 times we can be pretty sure we'll get about half heads and half tails, but we can't tell what will happen on each individual flip. Similarly, if we invest in the stock market, we can be pretty sure that it will go up over the long-term, but it's very difficult to consistently predict which specific stock will outperform the market.⁴⁰ As a result, it makes more sense to invest in an index fund, and to hold onto that fund for the long haul, because we can then reap the rewards of the general upward movement in the market. But what about all those stock experts with their hot tips? You have to ask the old adage, “If these people can predict the market, why aren't they all incredibly rich?” The fact is, if stock analysts could predict future prices, it would make infinitely more sense to keep that valuable information to themselves, because they could make a killing in the market if they did. But they don't. They sell it to you.

But We Still Want to Believe

So we have considerable evidence to indicate that it's not worth our time to analyze and pick individual stocks. Of course, if you're doing it for sport—if you enjoy the game—go right ahead. I go to a casino knowing that the odds are stacked against me in the long run, but I enjoy the act of gambling. However, if you think that you can pick stocks to beat the market, you're fooling yourself. And yet, very intelligent people continue to think they can.

I was talking recently with a fellow colleague of mine—one of the most scholarly professors in our school. He told me that he could pick superior stocks by using fundamental analysis, a technique that attempts to determine the intrinsic (true) value of a stock based on a firm's underlying economic variables. The technique assumes that when the market price is below the intrinsic value, the stock is undervalued and the price will rise when the market properly adjusts.⁴¹ It seems to make sense, and so many people believe that fundamental analysis can work. In fact, my friend told me he had supporting evidence, and referred to the research of Benjamin Graham, the father of fundamental analysis. As Professor Burton Malkiel notes, however, “The academic community has rendered its judgment. Fundamental analysis is no better than technical analysis in enabling investors to capture above average returns.”42 In fact, Graham himself reluctantly concluded that fundamental analysis could no longer produce superior returns, stating, “I am no longer an advocate of elaborate techniques of security analysis in order to find superior value opportunities. This was a rewarding activity, say, 40 years ago, when Graham and Dodd was first published; but the situation has changed…. [Today] I doubt whether such extensive efforts will generate sufficiently superior selections to justify their cost…. I'm on the side of the ‘efficient market’ school of thought.”⁴³

I saw my friend a few days later, gave him the quote, and said, “Wasn't this the person you were referring to for support?” His reaction was telling—he ripped up the quote and exclaimed, “That doesn't mean a thing!” Even though the person he was using as support said the technique is no longer effective, my friend refused to believe it. Instead, he reiterated, once again, that fundamental analysis has given him many successful stock picks. When I pressed him on this issue, he said, “Sometimes I'm right, and sometimes I'm wrong.” I said, “What if you're wrong 60 percent of the time and your portfolio drops?” He replied, “Then it's my fault and I'll learn from it.” I said, “What if it doesn't work year after year?” He responded, “Then I have to learn how to do it better.” In effect, he wasn't willing to question his belief in fundamental analysis—even in the face of considerable empirical evidence to the contrary. Why is that? He was relying on his personal experience instead of scientific investigation, and, as we've seen, relying on anecdotal information is one of the main causes of erroneous beliefs.

Fundamental analysts often attribute their gains to using fundamental analysis, and their losses to alternative explanations, such as a downturn in the overall economy. But with this rationale, fundamental analysis is unfalsifiable, and something that's unfalsifiable is worthless. What should you do if you decide to pick individual stocks for investment? Keep track of your wins and losses; in effect, pay attention to both the hits and the misses. As time goes by, you'll likely realize that you're not earning any more than if you just invested your money in an index fund. Without such an approach, you'll be doomed to make extremely costly mistakes, as another colleague of mine recently discovered. He had his money in an index fund until he was convinced by a mutual friend to pick individual stocks—which went on to lose 40 percent of their value when the overall market was rising!

ECONOMIC FORECASTING

While some people pay astrologers to foretell the future, and others pay stock analysts for stock tips, we, as a society, spend billions of dollars on economic forecasts. A number of government agencies and hundreds of private organizations sell economic forecasts. How good are they? A review of twelve studies on forecasting accuracy, covering the periods 1970 to 1995, concluded that economists can't even predict the major turning points in our economy.44 One study analyzed the error rates in forecasting the gross national product (GNP) growth and inflation eight quarters into the future for six major economic forecasters: the Federal Reserve, the Council of Economic Advisers, the Congressional Budget Office, General Electric, the Bureau of Economic Analysis, and the National Bureau of Economic Research. Forty-six of the forty-eight forecasts did not predict our economy's turning points.⁴⁵

Another study revealed that the forecasting track record of the Federal Reserve was, again, worse than chance. Over the period 1980 to 1995, the Fed called the major turning points in real GNP growth only three out of six times (the same as chance), and was unsuccessful in predicting the two turning points in inflation. Thus, the Fed was accurate only 38 percent of the time (three out of eight). The turning point predictions of the Council of Economic Advisors (CEA) and the Congressional Budget Office (CBO) between 1976 and 1995 faired no better, with overall accuracy rates of only 36 percent and 50 percent, respectively.

These data indicate that the major economic forecasting organizations can't predict whether there'll be a major turning point in our economy. As William Sherden notes, “Economic forecasters have routinely failed to foresee turning points in the economy: the coming of severe recessions, the start of recoveries, and periods of rapid increases or decreases in inflation…. In fact, they have failed to predict the past four most severe recessions, and most of them predicted growth instead for these periods.”⁴⁶

Most economists were forecasting a severe downturn after the stock market crashed in October 1987, and yet the economy expanded vigorously during the last quarter of 1987. The bottom line is, most economic forecasts are no better than just predicting that next year will be about the same as this year. In fact, they can actually be worse, because if change is predicted, there's a good chance the direction will be wrong.47

In addition, the expertise of economists or the sophistication of economic models does not improve forecast accuracy. Predictions based upon large models having over one thousand equations are no better than predictions from simple models with only a few equations. No matter how sophisticated the models are, they still can't reliably predict the future. A particularly telling test was revealed in 1995, when the magazine the Economist published the results of a contest held in 1985. They asked people with different backgrounds to accurately predict the British economy ten years into the future. Who won? A group of sanitation workers tied for first place with a panel of four chairman of multinational firms.⁴⁸ Once again, the amount of knowledge we have in a certain area will not help us predict what will happen if the events are inherently unpredictable. As Michael Evans, founder of Chase Economics, states, “The problem with macro [economic] forecasting is that no one can do it.”⁴⁹

Also, no specific economic ideology does a better job than the others in predicting the economy's future. Forecasts are typically influenced by the particular beliefs and assumptions that an economist holds, and those assumptions can lead to vastly different predictions about our economic future. In fact, economics seems to be the only discipline where two people can receive a Nobel Prize for saying the exact opposite thing. This has led to a belief in the “First Law of Economics: For every economist, there's an equal and opposite economist.”⁵⁰ When different economists predict very different things, it's difficult to place much credence in those predictions. Why are there such diverse views from economists? One reason may be that economics typically does not use the scientific method, where hypotheses are tested by observing what goes on in the economy. Instead, economists often develop elaborate theories that may be logically consistent, but are often based upon unrealistic concepts. For example, a basic assumption of economics is that people are consistently rational in their behaviors. However, we are psychological and social beings who often display inconsistencies and errors in decision making. Money is not our only motivator—we are also affected by conformity, power, love, revenge, charity, laziness, etc. Have you ever heard an economist's joke? Here's one: Two economists are walking down the street. One says, “Hey, there's a dollar bill on the sidewalk.” The other says, “That can't be so—if there was, someone would have picked it up.”51

Given their poor track record, some say that forecasters are actually worse than useless because they can cause long-term damage to the economy.⁵² Why? People can make bad financial decisions by relying on erroneous forecasts. Interestingly, more and more people have recently questioned the value of those forecasts. In fact, a few corporations have disbanded their economics departments. However, many economists still attempt to predict the future, and the cost of that useless information to our society runs into the billions. So why do we do it? Our desire to predict the future is extremely powerful, whether we're trying to predict matters in our own personal life or the economy as a whole.

IT'S GOING TO BE SUNNY—UNLESS IT RAINS

Weather forecast for tonight: dark.

—George Carlin

You're planning a ski trip at the end of the week, so you turn to the Weather Channel on Monday to get the five-day forecast. “It's going to snow on Friday,” it reports. Psyched, you can't wait for the weekend. On Tuesday, you check the weather again to see if Friday's storm will be huge, but now they say it's going to be sunny. What happened to the storm? Disappointed, you check the forecast a little later and this time it predicts rain for Friday. We hear long-range forecasts all the time, and they're reported with considerable certainty. And yet, those forecasts are constantly changing. What's going on? It turns out that we can't predict the weather that far ahead. In fact, weather prediction is reasonably accurate only about twenty-four to forty-eight hours in advance.⁵³

And yet, we spend billions of dollars to make long-range forecasts. The World Meteorological Organization estimated that the global budget for weather forecasting was around $4 billion in 1995. About half of that is spent in the United States by the National Oceanic and Atmospheric Administration, which is the parent organization of the National Weather Service, the Climate Analysis Center, the Severe Storms Forecasting Center, and the National Hurricane Center. The National Weather Service issues daily forecasts as well as three-to-five- and six-to-ten-day forecasts. The Climate Analysis Center gives us monthly and seasonal outlooks, predicting weather patterns for the next thirty days, ninety days, and even eighteen months into the future. We do this knowing that the weather is a chaotic system, and is therefore impossible to predict that far ahead.54

Research indicates that the National Weather Service can produce reasonably accurate forecasts of temperature, cloudiness, and rain twelve to forty-eight hours ahead of time. Heavy snows are more difficult to predict. For example, New England experienced the great blizzard of 1978 on February 6. The day before the blizzard, the Boston Globe made no mention of a big storm, and predicted that the winds would only be ten to fifteen miles per hour in an easterly direction. As William Sherden notes, “For the period more than 48 hours in the future, weather forecasting enters the twilight zone, where accuracy and reliability decline to a point of very limited usefulness. In fact, the prediction of the specific time and places of precipitation beyond two days becomes indistinguishable from random guessing.”⁵⁵

It's no surprise, therefore, that the long-range forecasts of the Climate Analysis Center aren't any better than chance. If you wanted to predict whether a certain part of the country would have normal, above-, or below-average temperature over the next three months, which is what the Climate Analysis Center does, you would do just as well by throwing darts at a map to predict those three options.⁵⁶

Some people believe that the Old Farmer's Almanac is quite accurate in its weather forecasts. In fact, the editors of the almanac claim that their forecasts are 80 percent to 85 percent accurate. Sounds impressive, doesn't it? But consider the evidence. William Sherden analyzed the monthly average temperatures for the previous thirty years in Omaha, Nebraska, and found that the almanac was accurate 49 percent of the time in predicting whether they would be above or below seasonal norms. Given that there's a 50/50 chance of being right in this prediction, we might as well just flip a coin. The almanac's accuracy in predicting temperature was 73 percent, which is close to the 80 percent that it claims. Sounds pretty good—but to determine the real value of the almanac forecast we have to compare its accuracy to some benchmark, like that attained from a naive forecast. What if, for example, we just used the seasonal average temperature to naively predict the current temperature. Amazingly, we would get an accuracy rate of 90 percent. So relying on the almanac gives us less accurate predictions than if we just used past seasonal averages, which require no forecasting skill at all.57

What can we take away from this evaluation of weather forecasting? In recent years, meteorologists have made great strides in storm detection and short-range forecasting over a one- to two-day time horizon. In fact, Sherden notes that meteorology is the only forecasting profession that shows signs of improvement. However, long-range forecasting is just voodoo. So you have to wonder, why are we spending billions of dollars to make long-run forecasts? It seems more like wishful thinking than anything else.

TECHNOLOGICAL AND SOCIAL TRENDS

Technology gurus constantly make predictions about the new gadgets that will change our lives. Are they accurate? Professor Stephen Schnaars of Baruch College analyzed the forecasts published between 1959 and 1989 in sources like the New York Times, the Wall Street Journal, and Business Week, and found that they were wrong about 80 percent of the time. The predictions of major think tanks fair no better. In fact, Futures magazine once analyzed 1,556 technological forecasts and found that the experts were no better than nonexperts in their prediction accuracy.⁵⁸

So-called technological experts over the past forty years have predicted that, by now, most people would be using video telephones, ultrasonic dishwashers and showers, moving sidewalks, and jet-powered cars. Some experts even predicted that the average person would be working only about ten hours a year—by the year 2000. Moreover, innovations that have had a major impact on our lives, such as cellular telephones, atomic energy, compact discs, and the computer, were not predicted. In fact, the dictionary definition of a computer was “a person who computes” as recently as 1950. In 1956 RCA predicted that by the year 2000, there would be only 220,000 computers in existence.59 The evolution of technology is so complex and uncertain that it is just about impossible to predict what the next breakthrough innovation will be. As Sherden states, “The main difference between technology forecasting and science fiction is that the former is sold under the pretense of being factual.”⁶⁰

One reason it's tough to predict which consumer product will take off is because it's not always the best product. For example, Sony's Betamax came out at about the same time as VHS. Most experts thought that Betamax was a superior product, but VHS gained a market share advantage in the video rental stores, which pushed Betamax into oblivion. In the early days of personal computers, many experts thought the Macintosh operating system produced by Apple Computer was superior to Microsoft's DOS operating system that was used in IBM computers. However, Apple refused to license the Macintosh software until 1995, so when other companies started mass producing low-cost computers, they installed the DOS system.⁶¹ Just compare the market share of the two companies today.

What about all those futurists who predict major trends in our society? As you might surmise, they're typically wrong. For example, if you believed Paul Ehrlich's book The Population Bomb, published in 1968, you would have thought that war, pestilence, and famine would engulf us by the 1990s, killing around 500 million people. In 1970 Alvin Toffler wrote Future Shock, in which he predicted we would experience a psychological melt down by the 1990s because of too much change in our lives. In fact, we appear to be adapting quite well. Once again, it seems like the only difference between these types of books and works of fiction is that they are sold as nonfiction.⁶²

WHY WE CAN'T PREDICT—BUT STILL TRY TO

As we've seen, we attempt to predict many things that are essentially unpredictable. Why aren't they predictable? Two major reasons come to mind: chaos and complexity. The weather is inherently unpredictable beyond a couple of days because it is a chaotic system. Chaos theory applies to a limited number of physical systems, like the weather and fluid turbulence. With chaos, turbulent behavior is determined by nonlinear laws that magnify small errors in the initial conditions of the system, making it extremely unpredictable beyond a short period of time. You may have heard of the butterfly effect, where something as small as a butterfly flapping its wings in a far-off place may have consequences for the weather you're experiencing here and now.

The economy and other social systems are unpredictable because they are complex systems. In complex systems, order emerges from the complex interactions among components of a system, influenced by one or more guiding principles. For example, our ecosystem developed because of the complex interactions of many different life-forms guided by the principle of natural selection. Complex systems are unpredictable because it's practically impossible to know the outcome caused by countless interacting variables. These systems have periods of order interrupted by unexpected turmoil, and they can evolve, exhibiting new, unexpected behaviors. And so, chaos relates to many aspects of the physical world, while complexity relates to our biological and social world. In either case, accurate predictions are just about impossible to make.63

So why do we believe predictions that have no basis in reality? As with many of the topics discussed so far, we believe we can predict the future because we want to believe it. We abhor uncertainty. We like to feel in control, and the thought that we can predict the future gives us a better sense of control. Also, we are easily influenced by people who are in authoritative positions. We attribute expert status to economists, meteorologists, and stock analysts because they've had years of training in their respective disciplines. But as we've seen, some things are essentially unpredictable, no matter how much training a person has.

As William Sherden says, “The theories of chaos and complexity are revealing the future as fundamentally unpredictable. This applies to our economy, the stock market, commodity prices, the weather, animal populations (humans included), and many other phenomena.”⁶⁴ In effect, some things are just not knowable. We would be well advised to follow the lead of Winston Churchill, who gave up trying to predict what was to come, complaining that the future was just one damned thing after another. The sooner we realize that many things in our environment are essentially unpredictable, the sooner we'll be able to make more informed decisions on what to believe and how to use our resources.