Twenty Two

How to Outguess the Stock Market

Some have made fortunes out of gyrating markets, and others have lost them. One who’s done both is Victor Niederhoffer. Son of a policeman, Niederhoffer took an economics PhD at the University of Chicago, the bastion of the random walk theory, which holds that short-term changes in stock prices are completely unpredictable. “I criticized all those who had concluded that markets were random, including most of the professors in the room,” Niederhoffer said. “Further, I cautioned them that their failure to disprove a hypothesis… was methodologically inadequate to support a conclusion that prices were random. When I put it in the vernacular, ‘You can’t prove a negative,’ pandemonium broke loose.”

Niederhoffer collaborated with M.F.M. Osborne on a paper that could be called the Magna Carta of high-frequency trading. Osborne was another outsider to the economics profession, an astrophysicist who worked for a navy think tank. He had done important work on the random walk hypothesis. But in “Market Making and Reversal on the Stock Exchange,” published in the Journal of the American Statistical Association (1966), Niederhoffer and Osborne argued that stock price movements are not random at all. They described a way to outguess the market.

Here is a chart redrawn from one in their paper. It shows a few minutes of trading in the stock of Allied Chemical. At that time, stock prices were quoted in eighths of a dollar (12½ cents).

The line in this chart doesn’t look too random, and it’s not. At upper left, the Allied Chemical price zigzags between two price points like a Ping-Pong ball. Elsewhere, it seems that traders liked the prices of $56 and $55¾, for there were streaks of trades executed at these values. Niederhoffer and Osborne proposed that a floor trader could predict these price movements with enough accuracy to turn a profit.

Average investors imagine that a stock has a single price fluctuating in time, like the “market price” lobster on a summer resort’s menu. The reality is that there are always two prices, a bid and an ask. The bid price is the highest amount that buyers are willing to pay. The ask price is the least that sellers are willing to accept. In a well-regulated market, the difference is minimal.

Some investors place market orders to buy at the current price, whatever it is. In the 1960s humans executed these orders, much as operators routed phone calls. Someone wanting to buy at the market price would get the ask price, and someone wanting to sell at market would get the bid price.

When market orders come in haphazardly, with as many to buy as to sell, you’d have a Ping-Pong pattern as at upper left in the chart. The execution price fluctuates between the bid and ask price ($56 and $56⅛ in this case). This goes on until there are no more limit orders—commitments to trade at the current bid or ask price. Then the price crashes through the ceiling or floor and moves to another level.

Niederhoffer and Osborne discovered that traders setting limit orders greatly preferred round numbers. They would give instructions to buy or sell at $56, say. To a lesser degree they liked half-dollar amounts like $55.50; to a still lesser degree they favored quarter-dollar amounts like $55.75. Least popular were odd fractions like $55⅝. This has important consequences.

A trader who wants to make a quick profit would do well to place a buy order a little above a round number. Price movements are constrained by whole-dollar values and simple fractions. Should a trader buy at $38.01 (today prices are quoted in cents), and should the market sag, the backlog of buy orders at $38 would be a firewall delaying movement below that. That might give the trader time to sell at $38, for a penny’s loss.

It’s equally likely that prices would move upward. In that case, the trader would be in great shape. The price might go up to $38.10 or $38.50 or $39. The alert trader would have ample opportunity to make a profit. Because the likely profit would be much larger than the penny at risk, the odds are in the trader’s favor.

Niederhoffer and Osborne were not the first to realize this fact. In their paper they admitted that “professional traders will recognize these rules or their equivalent as quite familiar.” That was one of the most interesting things about their system. It allowed them to pick the brains of the pros. When a stock was frequently traded a little above or below a round number, it meant that the smart money was interested in that stock. These may have been people who had some kind of inside information, and that information was more often than not correct.

In effect, Niederhoffer and Osborne described an early high-frequency trading algorithm. It was purely mechanical, involving no traditional stock-picking skill. It profited from the market’s short-term jitters and flows of information rather than from the long-term growth of businesses.

Osborne, a scientist at heart, did not commercialize his insights. Niederhoffer did. As computing power came online, he wrote one of the first trading algorithms in the 1970s. George Soros gave him money to invest. Niederhoffer’s personal wealth, and the wealth under management, increased over the next two decades. The gimmick above was soon overexploited, but Niederhoffer kept devising new ways to beat the market. He was done in by an indicator more unusual than the off-dollar trades: cigarette butts.

Niederhoffer wanted to invest in Southeast Asia. He delegated a friend, Steven Keeley, to scope out the region. Keeley was a veterinarian. He had a theory that a nation’s economic health could be divined from the length of discarded cigarette butts. When people were feeling wealthy, they left longer butts. Another leading indicator was the hygiene in houses of prostitution. After several months of fact-finding, Keeley reported that Thailand was A-OK. Niederhoffer invested, and in 2007 the Thai economy collapsed. He lost his clients’ money and much of his own.

Can a nonprofessional outguess the market? The prospects for the amateur stock picker have probably never been worse. Small investors are outgunned by the high-speed algorithms that account for about half of today’s trading (and which haven’t been doing that well themselves lately). Professionals who devise new ways to predict the market’s movements with greater-than-chance accuracy can often profit from that. The profit taking in turn tends to eliminate the predictability. Thus many market inefficiencies vanish almost as soon as they are discovered and become known. This is how the market polices itself and remains almost completely efficient.

There are a few inefficiencies that have persisted despite being generally recognized. They are based on trader psychology that is presumably unlikely to change, and they are difficult or impractical for the professionals to arbitrage away. Some are hot hand effects.

In the NBA the hot hand is a mirage. In the financial markets it can be a self-fulfilling prophecy. Traders are predisposed to see trends in the noise of stock prices. When many traders come to believe that stocks are on a winning streak, they buy more stock, bidding up the prices. The shared fiction becomes a reality. As long as enough buyers believe that the market will keep going up, it will.

The cycle needn’t stop there. A rising stock market makes consumers feel prosperous. They may buy more goods, boosting the economy and justifying further stock price increases. But eventually the house of cards collapses. Emotions cool, and stock valuations struggle to get any higher. A streak of market declines leads traders to see the slump as a grim portent. Many pull out of the market, forcing further declines. In this way hot hand and cold hand thinking help create bull and bear markets. That’s been going on ever since there were markets and isn’t likely to change in the foreseeable future.

One of the small investor’s few advantages is time. He or she has the luxury of pursuing strategies that may play out over decades. That puts small investors in a different niche of the financial ecosystem from the star managers, hedge funds, and traderbots that must prove their worth every quarter. Individuals of moderate means are also likely to have much of their portfolios in tax-deferred retirement accounts. Trades within the account do not trigger taxes. That puts these individuals in a position to profit from the market’s mood swings—if they’ve got the discipline to do it.

Discipline is very much the issue. The ongoing and well-concealed scandal of the investment industry is that average investors don’t make anything like the “average” returns you hear about. It’s not that market averages are a fiction. It’s just that most people buy and sell at the wrong times. The more investors trade, the lower their realized return, on average. The shortfall goes by the name of the behavioral penalty. As a study by Brad Barber and Terrance Odean put it, trading is hazardous to your wealth.

The research firm DALBAR computed that the average stock investor in the period 1990 through 2009 underperformed the S&P 500 index by 5.03 percentage points a year. The reason investors underperformed is that they were trying to beat the market. Like a carnival game, it looks so easy. “People tend to be overconfident about their own abilities,” said Teresa Ghilarducci, an economist specializing in investor behavior. “They tend to focus on the short term rather than thinking about long-term consequences. And they tend to think that whatever the current trend is will always be the trend.”

That’s hot hand thinking. Bryan Harris, an analyst at Dimensional Fund Advisors, reported that investors sold $266 billion of US stock mutual funds from March 2009 through June 2011. The selling peaked almost precisely when the market was at its bottom.

How do you tell when stocks are a good buy? It’s not rocket science. The ultimate justification for stock prices is earnings. This is a truism of Economics 101. It hardly merits mentioning except that most investors don’t believe it, or don’t act as if they believe it.

From the investor’s standpoint, a stock is a machine for producing earnings. Were you buying an apartment building as an investment, you’d ask how much rent it was producing. You would want to buy as much rental income as possible for your money. Corporate earnings are the stock market’s rent. Any dividends will be paid out of those earnings. Earnings not paid out may be reinvested in the company, increasing the value of shares.

Take a corporation’s earnings per share and divide it by the share price. This is the price-to-earnings (PE) ratio. It’s the standard, quick-and-dirty measure of how attractive the stock is as an investment. A PE of around 15 is typical. When the PE is low, the investor is getting a lot of earning power for not so much money. When the PE is high, the investor is paying a lot for a trickle of income.

There are plenty of reasons why a stock’s PE may be high or low. When a company’s earnings are growing rapidly, its PE is usually high, and this might make sense. When a company is in a declining industry or has financial problems, its PE may be low, and this also could be reasonable.

You can compute PE ratios for a market index like the Dow Jones Industrial Average (of thirty blue-chip stocks) or the S&P 500. The S&P 500 index covers the broad American stock market, and there are scores of index funds tracking it. Historically, the median PE for the S&P 500 (or an S&P 500 index fund) has been around 16. It has varied wildly, though. At times it’s been 30 or more; other times it’s been in the single digits.

This doesn’t make sense. The S&P 500 is all of America’s big companies averaged together. Even in boom times, the whole S&P 500 can hardly merit the valuation of a growth stock. In the worst of times, it doesn’t deserve the low PEs of a doomed company. Yet that’s the way the market works. Stock prices, and PEs, are a lot more volatile than corporate earnings are.

To be sure, index earnings sometimes do fall off a cliff. In 2009 the S&P 500’s earnings fell to less than a tenth of where they’d been before the subprime mortgage crisis. This was due to banks and other companies writing off a mountain of bad debt all at once. After the purge, earnings promptly bounced back. The market is supposed to take a long-term outlook and price stocks according to the whole future stream of earnings. But investors, like everyone else, believe in the representativeness of small samples. PE valuations have too much to do with the latest news cycle, the latest quarter, and the last few years.

Yale economist Robert Shiller devised a better way of gauging stock market valuation. It’s the current S&P 500 price divided by a ten-year moving average of earnings. This has the merit of smoothing out business cycles and much of the duplicity in corporate earnings reports—for there are cycles of candor as well as profit. A ten-year average of corporate earnings is about as truthy as these things get.

Shiller’s idea wasn’t entirely new. At least as far back as 1934, pioneering value investor Benjamin Graham recommended using five to ten years of earnings in computing PE ratios. Graham was talking about PEs of individual stocks, but the notion could equally apply to indexes. Shiller’s Cyclically Adjusted PE ratio (aka the CAPE ratio, the Shiller PE ratio, or the PE 10) is computed by dividing the current S&P 500 value by the average of the past ten years’ earnings of the S&P companies. Shiller adjusts those past earnings for inflation. In that way the ratio’s value is comparable to a regular PE.

A first reaction might be that Shiller has thrown the baby out with the bathwater. The name of the game is to predict future earnings. Why bother with the past? Surely we can forecast earnings.

Don’t bet on it. The best way to demonstrate that astrology is nonsense is to compare predictions. When different astrologers have different forecasts for Libra, they can’t all be right. Well, the best way to show that earnings forecasts are of dubious value is to compare the predictions of Wall Street analysts. These forecasts are often alarmingly different and biased toward optimism. One Federal Reserve Board study found that analysts’ average expectations for the current year’s S&P 500 earnings were too high in nineteen out of twenty-one years (from 1979 to 1999). Just like investors, analysts are too obsessed with the current trend. Shiller’s ten-year PE enforces a big-picture perspective.

What if S&P earnings rise dramatically over the ten-year period? It’s not going to happen. A few smart and lucky companies will have skyrocketing earnings; the broad market indexes won’t. Remember, that hot technology company is cannibalizing the market of older companies that are in the index, too. When you subtract the fake growth of inflation, the real, averaged-out S&P earnings don’t change much over ten years. They haven’t in the past, anyway.

The Standard & Poor’s 500 index is a fairly recent invention, inaugurated in 1957. It is intended to track the 500 largest companies, by market value, publicly traded in the US stock market. Shiller backtracked and projected what 500 companies would have been in the S&P index, had it existed before 1957. He used earnings reports to reconstruct the ten-year PE back to January 1881. Here’s a chart of it.

It is hard to explain the huge variations as reasonable changes in the outlook for future earnings. Look at the rises to the big peaks in 1929 and 2000, and the equally insistent drops afterward. These were famous stock market bubbles driven by hot hand beliefs.

Shiller found that his backward-looking ten-year PEs have considerable power in predicting future returns. This is demonstrated in the chart below. Every dot represents a month, from January 1881 through January 1993. The dot’s position is determined by that month’s ten-year PE value (on the horizontal axis) and the return that an investor would have achieved had he invested a lump sum in the S&P 500 stocks that month and held that investment for twenty years (this return on the vertical axis). These are average annual returns over the twenty-year-period, adjusted for inflation. It’s assumed that dividends are reinvested, but this does not account for commissions, management fees, or taxes, all of which can vary a good deal. (To save words, hereafter all quoted returns will be adjusted for inflation, and PE will mean Shiller’s ten-year PE.)

The dots are not scattered all over the chart. Instead, the dot cloud forms a diagonal swath from upper left to lower right. That means that future market returns are predictable, albeit with a good deal of noise. The most important conclusion: The lower the PE when a long-term investor enters the market, the higher that investor’s likely return.

This is true even though the great mass of investors pay no attention to ten-year PEs and have no awareness of how predictable their collective actions are. This predictability also exists despite the fact that many smart traders were and are trying to take advantage of it. In an all-wise market, investors should have bid up the prices of stocks during those low-PE periods, nullifying the advantage of buying at those times. Investors also should have shunned stocks during high-PE periods, lowering prices and increasing returns. The result would have been a straight horizontal line of dots or (more realistically) a fuzzy horizontal dot cloud. I will show you a chart like that in a moment. But that’s not what we see here. The diagonal dot cloud shows that there’s a penalty for buying at high PE and a bonus for buying at low PE.

The average of the twenty-year returns is 6.58 percent after inflation. This is in line with the “average” stock return values touted by those who want to sell you stocks and mutual funds. When those values aren’t adjusted for inflation, and usually they aren’t, the average is around 10 percent.

There is only one dot with a negative return. An investor unlucky enough to have jumped into the market in June 1901 (at a PE of just over 25) would have lost buying power. After twenty years of inflation, the average return would have been -0.24.

Every other twenty-year period eked out a positive real return, and most beat the returns that could have been achieved with bonds. (Typically, the safest bonds have offered a real return of a little over 2 percent.) Many of the dots represent periods that spanned the 1929 crash and the Depression. Those investors still came out ahead. This fits in with the credo that stocks are not that risky for the long-term investor willing to stay the course. A very few of the periods achieved real returns over 12 percent. All those charmed spans started with a low PE.

Today’s investors have every right to feel cursed. They have had few opportunities to buy at average PEs, much less low ones. In mid 2013 the Shiller PE was around 23. The average real return at that valuation is something like 2 percent over the coming twenty years. Never has the twenty-year stock market returned as much as 3 percent annually (after inflation) when the PE was 23 or higher.

Many investors would say that a 23 PE is not all that high. It’s not that high in the experience of anyone born after the mid-1970s. Forget the “lost decade.” There’s a lost generation of stock investors who have never had a decent shot at making so-called average stock market returns.

The correlation between PE and return makes perfect sense. Why don’t most investors believe it?

Opposite is another chart of return versus PE. This time it’s the return over a one-year period, after buying the S&P stocks at a given PE.

The dot swarm is now horizontal. An investor’s prospects at PE 30 aren’t much different than at 10. This is what you’d expect of an efficient market in which investors price in everything known and guessable about future earnings. There is no free money to be had by investing at low PE for a one-year holding period.

The PE’s effect on returns comes with a time lag. That’s unfortunate because most of us have trouble making good decisions when effects do not immediately follow causes. High PEs are bad for you, like sugary drinks are bad for you, but it may take decades for that to become apparent. During bull markets, investors want to do what feels good at the moment. They join the crowd and buy the stocks that are supposedly making everyone rich. The chart of one-year return doesn’t reveal a problem with that. Some buying months had positive returns even when the PE was over 40. The most obscenely overvalued markets may extend their winning streak another year. It’s only when you look at much longer periods that returns correlate with PE.

“One might have thought that it is easier to forecast into the near future than into the distant future,” Shiller wrote, “but the data contradict such intuition.” Investors, traders, and analysts are trying to predict the wrong thing: the market’s next minutes, weeks, and quarters. Most are banging their heads against a brick wall. Certainly the small investor is.

There are long regimes where investors pay scant attention to PE valuations. These periods are terminated by the mass epiphany that stocks are over- or underpriced. Once the market starts correcting, the pendulum usually swings through the mean and keeps going. It may take a decade or more for this process to come to completion. That pummels the twenty-year returns when stocks are bought at high PEs, and augments returns when investing at low PEs.

The effects of buying at high or low PE can persist much longer than twenty years. Those promoting stock investments often quote an “average” return from 1920 to the present. This sounds so reasonable because, without the need of anyone pointing it out, we all recognize that the period includes the 1929 crash, the market’s worst catastrophe. Not everyone realizes that PEs were at rock bottom in 1920, dipping below 5. The PE is currently almost five times that. That PE increase alone adds about 1.7 percentage points a year to an “average” return calculated from 1920 to the present. Whenever you hear someone bragging about great returns, it’s a safe bet that he was riding a PE cycle up.

“Long-term investors would be well advised, individually, to lower their exposure to the stock market when [the PE] is high, as it has been recently, and get into the market when it is low,” Shiller wrote in 2005. This observation may seem incontestable. Yet the reaction to the notion of the small investor outguessing the stock market has been schizophrenic.

On the one hand, a legion of high-visibility experts talks as if it’s easy and routine to “time the market.” When valuations are high, you “take money off the table.” When valuations are low, “stocks are on sale” and it’s a great time to buy. Pundits are usually not talking about Shiller’s long-term changes but rather about the market ups and downs happening all the time, even on a daily basis. They propose trading a lot. Many of these experts are sponsored by brokers or mutual funds.

Paul Samuelson memorably expressed the opinion of many economists. “Suppose it was demonstrated that one out of twenty alcoholics could learn to become a moderate social drinker,” Samuelson said. Then the wisest course would be to pretend that it wasn’t so. “You will never identify that one in twenty, and in the attempt five in twenty will be ruined.” In Samuelson’s view, friends shouldn’t let friends try to beat the market. Even if it’s possible, they are more likely to lose than to profit from the attempt. As we’ve seen, there is ample evidence for that proposition.

Lately the behavioral penalty has been co-opted by the mutual fund industry to promote buy-and-hold investing. They’re not entirely motivated by the public good. Fund managers collect fees only while investors own their funds. It’s easier to justify high fees for stock funds than fixed-income funds. Therefore, the fund industry is reluctant to admit that there might be times when it’s not worth owning stocks. They have concocted bogeyman stories to scare investors into buying-and-never-selling.

One of the favorite pitches is that most of the stock market’s long-term return is due to a few halcyon days that post big gains. You don’t know when those days will be; ergo, you have to be fully invested all the time. Take the ten largest daily percentage gains for the Dow Jones index. Had you been invested in Dow stocks, you could have almost tripled your money (a 195 percent gain) in just ten days.

A reasonable response is So what? Here’s a statistic they don’t offer. Take the ten worst days for the Dow. You could have lost over two-thirds of your wealth (a 68 percent loss) in just ten days.

Seven of the ten “best” days came after the 1929 crash and in the ensuing chaos, through early 1933. The treasure chests are mixed in with the landmines.

Thoughtful investors realize there are times when the stock market isn’t very attractive relative to the alternatives. In December 1999 the market achieved its highest-ever ten-year PE, 44.20. The earnings yield—the maximum the S&P companies could have paid in dividends, had they decided to distribute every penny of their earnings—was a miserable 2.3 percent. Meanwhile, Treasury bonds were paying 6 percent. Duh. A sane long-term investor would have favored virtually risk-free bonds over risky stocks. At that valuation, stocks were very risky. Less than three years later, the S&P 500 had shed almost half its value.

Daily changes are noise. Market tops and bottoms are noise. The regular investor can’t predict them and shouldn’t try. The mistake is in thinking that the investor can’t predict anything. Given the evidence that market returns are moderately predictable over decades, the long-term investor should make use of that.

The ten-year PE is not unique in its forecasting power. There are many other ways of assessing fundamental value, and they all work, to a greater or lesser degree. You can look at S&P 500 dividend yields, asset value, or a ten-year moving average of the index itself. All have substantial predictive power (and their predictions correlate reasonably well with those of the PE10). Shiller’s main point is that the underlying value of America’s corporations does not change too much or too quickly. It’s emotion-driven stock prices that get out of line.

There are two good reasons for an investor to favor Shiller’s PE as a predictor of return. One is that the economic case for the importance of earnings is especially compelling. The other is that the Shiller PE is easy to look up on the Web.

A number of workable trading systems using PEs have appeared in books and the financial press. Ben Stein and Phil DeMuth’s 2003 book, Yes, You Can Time the Market! describes several of them. In one you make an annual determination of whether the market is overvalued or undervalued, using a moving average of S&P 500 PE ratios (Stein and DeMuth favor a fifteen-year term rather than Shiller’s ten-year frame). You add to your stock fund holdings only in years when the market is undervalued. Otherwise, you bank that year’s contribution to your portfolio, awaiting a better buying opportunity. When favorable valuations return, you don’t buy in all at once (for that would probably be at a PE only a little lower than average). Instead, you buy in gradually, at a rate of twice your intended annual contribution, as long as valuations remain below average.

This system is simple and well tailored to the saver who adds money to a retirement account each year. It guarantees that all stock funds are bought at lower-than-average valuations. That alone is enough to boost returns. For a twenty-year holding period, a typical gain is about half a percentage point a year.

The Stein-Demuth system works entirely by deferring stock purchases. It makes no assumptions about market tops and bottoms, and it’s about as stress- and regret-free as a market outguessing system can be. One limitation is that it works on new contributions only, not on capital you may already have. Someone who wants to get the most value from the PE’s predictive power needs to be willing to sell as well as buy. Let’s look at ways to do that.

Warren Buffett said that the first rule of making money is to not lose money. A realistic goal is to use PEs to exit the US stock market during most of the biggest plunges. These generally happen when PEs are high.

Here’s the simplest PE-based system of all. You invest in a low-cost S&P 500 index fund, buying low and selling high (in PE terms). When the ten-year PE hits a specified high value (the sell trigger), you sell and put the proceeds in a low-cost fixed-income fund (offering the return of ten-year US Treasury bonds, let’s say). You stay in the bond fund until the PE hits a particular low value, the buy trigger. Then you buy back into the stock fund, and the cycle repeats. To make things as easy as possible, I’ll assume that you’re very busy and can check the PE—and trade when indicated—only once a month.

I tested all the plausible whole-number pairs of buy and sell limits, computing the compound return that could have been realized over the period January 1881 to January 2013. In each case the portfolio started 1881 (when the ten-year PE was 18.47) fully invested in stocks.

The table below shows the most interesting part of the results. Sell trigger values are at top, and buy trigger values are on the left. Each cell within the table gives the average real annual return, for the corresponding pair of sell and buy thresholds, over the entire 132-year period. Returns are adjusted for inflation but not for trading expenses, management fees, or taxes.

The average real return of the S&P 500 stocks and their precursors during this period was 6.23 percent. Any cell containing a value higher than 6.23 percent marks a strategy that would have beaten the market. In fact, most of the strategies shown did outperform the market.

I’ve crossed out the cells in the lower left corner, as they represent policies that clearly make no sense (buying higher and selling lower, in PE terms). The other cells are colored as a heat map. The region of light-shaded cells on the left, in the topmost row, and at upper far right shows strategies that would have returned less than a buy-and-hold S&P 500 portfolio. The white cells are strategies that beat the market by up to a percentage point a year, and the medium-shaded cells at upper and center right outperformed the S&P index by at least 1 percent a year. The highest return was 7.62 percent, the result of buying at 6 and selling at 32.

Would it be wise to adopt a policy of trading at 6 and 32? No, not unless you can replay the last century. The chart’s returns are noisy. There is much cell-to-cell variation, owing to luck in trading at a particular crest or trough in the historic record. The 6 and 32 limits happened to call the bottom and top of the 1920s bull market rather closely. This would have been profitable enough to spike returns even when averaged over our 132-year period.

No one should expect that kind of luck. Given that the future won’t be exactly like the past, you should pay more attention to the general pattern of returns. Picture yourself tossing a dart at the chart, unable to control the exact cell you hit. Where would you aim, to have the best chance of landing in a cell with a good return?

It would be unwise to aim for the “best” return, at 6/32, as it’s adjacent to cells that underperformed the market. A not-so-accurate dart thrower would be better off aiming for somewhere in the right center of the chart. Let’s say you picked 13 and 28. That would have earned a return of 6.70 percent, beating the stock market by 0.47 percentage points a year.

If that doesn’t sound like anything special, take a look at this chart. It shows the (hypothetical!) growth of a $1,000 portfolio invested since 1881. A buy-and-hold investment in the S&P stocks would have grown to an inflation-adjusted $2,932,724. A buy-low, sell-high portfolio, with PE limit values of 13 and 28, would have grown to $5,239,915.

The return is only half the story. Look at how smooth the upper line is, compared to buy-and-hold, over the past twenty years. A PE-directed investor would have sold out of stocks in January 1997, sparing herself a couple of agonizing crashes. She would have been in safe, steady fixed-income investments through January 2013. The PE investor’s superior return is due entirely to avoiding losses.

The maxim is that risk and return are trade-offs. To get more return, you have to accept more risk. The historical data says that downside risk goes up with PE, just as long-term return goes down with it. By exiting the market during the least profitable times, you also skip some of the riskiest times.

In fact, all the returns in the table are more appealing when you factor in risk. As long as you manage to buy into the stock market at a low PE and sell at a higher one, you’re almost certain to realize a greater-than-“average” stock return. The buy and sell trigger pairs that underperformed the market did so because they had too much downtime in bonds. Say you had picked 8 and 22 as your limits. This would have averaged only a 4.85 percent return. You might think that was a dud. But with those limits, the PE-directed portfolio would have been in stocks only 37 percent of the time. It beat the market while it was in stocks and offered the safety of fixed-income investments the rest of the time. By risk-adjusted return, that’s not so bad.

To top the S&P 500’s return, you need to be more selective about limit values. The historical record suggests that this is not too difficult to do. Many pairs of buy and sell thresholds would have beaten the market by half a percentage point a year.

Most trading schemes have you trade a lot to eke out a minuscule advantage. With this system you hardly trade at all. Had you used 13 and 28 limits, you would have made only three trades in the past 132 years. You would have sold out of stocks in July 1929, three months before the crash, and bought back in in September 1931. Then you would have sold out of the market in January 1997, before the dot-com crash. In retrospect these actions look almost psychic.

These three timely moves would have avoided horrific losses, thereby beating the market handsomely over a 132-year period. But notice that a trader who began using the buy-low, sell-high system in the mid-1930s could have spent an entire investing lifetime without ever getting a trading signal or benefiting from the system in any way! On the other hand, had you been in the market in 1997, or 1929, you would have appreciated the system’s guidance.

There is a more powerful way to make use of PEs. The self-fulfilling nature of hot hand beliefs often creates momentum. Stocks will keep going up (or down) for a while. An index fund investor can take advantage of that with a trick similar to a trailing stop order.

Traders of individual stocks may place a standing order to sell a stock when it drops below a specified price. Say you own Apple and it’s trading at $420. You’re going on an Antarctic cruise (no Internet connection south of Tierra del Fuego) and can’t stand the idea that you might come back to find that the stock has dropped sharply. You could put in a stop loss order for $380 a share. This tells your broker to sell Apple if and when it dips below $380.

A stop loss order is insurance. As with any type of insurance, you hope you won’t need it. It limits your losses when something bad happens. In the case of a decline in Apple stock, the broker will likely be able to sell at a little below $380.

A stop loss order can be more sophisticated. It’s possible that Apple may shoot up first and then plunge, all while you’re incommunicado. You’d like to get some of the upside action. With a trailing stop order, you tell the broker to sell whenever the stock price drops 10 percent (or any percentage you name) from a high. The high is the stock’s maximum price during the time the order is in effect.

Example: Apple is selling at $420 when you place the trailing stop. The next day it rises to $428. That resets the maximum. Thereafter the broker will sell only if Apple drops 10 percent from that new high-water mark of $428. Should the stock continue to rise without much of a pullback, the maximum price will ratchet upward. Maybe it’s a roller-coaster couple of weeks and Apple surges to $483 before crashing down to $392. The broker would attempt to sell at a price 10 percent below the high. You could return from the Antarctic to find you’ve scored a nice profit—even though Apple is down from where it was when you left.

The index fund investor can create a do-it-yourself trailing stop, using PEs. As before, you choose a pair of PE values as buy and sell thresholds. The difference is that you don’t trade the instant the threshold is reached. You keep a record of the highest level the PE has reached since the sell threshold was crossed and sell whenever it drops X percent below that running high. I’ll continue to assume that you check the ten-year PE only once a month.

For buying back in, it’s the reverse. Once the PE sinks below the buy limit, you keep a record of its lowest monthly value. You buy back into the market the first month in which the current PE is at least X percent above the running low.

Stock traders typically set trailing stops at 10 to 30 percent. There is reason to keep X small. You will be selling at a discount of at least X percent from the peak, and buying at least X percent above the market’s bottom. The trade-off is that when X is very small, regular month-to-month volatility will quickly trigger a trade. You want a less itchy trigger finger—a larger X—in order to piggyback on any momentum that might cause valuations to move beyond the threshold.

As it turns out, the precise value of X was not too crucial. Here is a table of returns for a 6 percent trailing limit, over the period 1881 to 2013.

I’m using the same range of buy and sell values as before, and the same heat-map shading. There are more threshold pairs beating the market by 1 percent and even 1.5 percent (the darkest shading, in the center). Once again, the individual cells matter less than broad patterns. In this case, the highest returns—14, 15, or 16 to buy and 24 or 25 to sell—form a bull’s-eye, surrounded by other high returns. A dart thrower’s reasonable aiming point might be 15/24. It returned 7.93 percent in this period, beating the S&P 500 by 1.70 percentage points a year.

These thresholds are less extreme than those of the simpler buy-low, sell-high system because you’re not necessarily trading at them. When there is momentum, the trailing limit trick can often hitch a ride. This also has the effect of producing a few more trades. That helps ensure that the system will produce an advantage in the investor’s lifetime.

Let me now justify the trailing stop value of 6 percent. Here is a chart showing how returns varied with X. I’m holding 15 and 24 constant as the buy and sell triggers. All the trailing stop values through 20 percent performed respectably, though choices from 3 to 11 percent did best. Six percent had the highest return, barely.

You can likewise vary the checking interval. Checking the PE just once a year—and trading if called for—would have produced a respectable return of 7.46 percent. That a monthly or yearly checking regimen performed so well may come as a shock to mobile device addicts who check the market multiple times a day. A lot can happen in a month, and that was true even in slower times. The market dropped 26 percent in October 1929. But it had dropped 11 percent the previous month. Someone selling on a 6 percent drop would have gotten out before the crash and sold at a price about 11 percent off the peak.

Pretend an ancestor of yours started with a $1,000 US stock investment in January 1881 and that capital was completely invested with a PE momentum system ever since, using 15 and 24 as the limits. The system would have sold out of the stock market four times and bought back in four times. That’s eight trades in 132 years, or an average of about one trade every seventeen years. A typical investor can expect several trades in a lifetime.

Below is a chart of the ten-year PE with shaded bars marking the times when a 15/24 momentum investor would have been invested in the stock market. The unshaded strips represent periods when the system would have been in fixed-income investments.

Over short time frames, changes in the ten-year PE are almost all due to changes in stock prices. The chart’s steep declines in PE correspond to steep declines in stock prices. The momentum system skipped the 1929 crash and both crashes of the 2000s with remarkable timing. It sold near the top of the 1901 and 1966 bubbles, though it bought back in before the ultimate low.

Here’s a chart of portfolio value. These are real gains after allowing for the diminishing dollar (but not transaction costs, management fees, and taxes). During this 132-year period an investment in ten-year US Treasury bonds might have turned an initial $1,000 into $18,704, inflation adjusted. I do not chart that because the line would hug the axis so closely as to be indistinguishable from it.

A buy-and-hold stock investor in the S&P 500 and its precursor companies would have turned $1,000 into about $2,932,653. An investor using the PE momentum system to switch between stocks and bonds would have realized $23,836,362. That’s over 8 times the wealth of a buy-and-hold investor.

Nobody invests for 132 years. Let’s look at something a little easier to relate to, the twenty years from January 1993 to January 2013. This time we start with $1,000 in 1993. A bond investment would have grown to $1,617 in real terms. A buy-and-hold stock investment would have risen to $3,173. The PE momentum system would have ended up with $5,517.

It would have done that by making just two trades. The buy-and-hold portfolio was ahead of the momentum system for brief periods at the peaks of the dot-com and subprime mortgage bubbles. It didn’t stay ahead for long, though, and it didn’t finish ahead. The buy-and-holder weathered two horrific declines. In the first, the stock portfolio lost 43 percent of its value. In the second, it lost 50 percent. (These figures may be different from what you’ve heard, as they account for reinvested dividends and inflation.)

At the bottom of the market’s second plunge, in March 2009, the stock portfolio’s value was nearly tied with that of the bond portfolio. That’s one demonstration that stocks do not always outperform bonds over fairly long periods.

The dotted line labeled “Capitulator” represents the possible fate of someone who considered himself a buy-and-hold investor but who was spooked by the second big plunge in a decade. Exasperated by weeks of declines, the capitulator threw in the towel. He sold, vowing never to invest in stocks again.

The capitulating investor was lucky enough to sell somewhere above the very bottom. Still, as the dotted line shows, the capitulator ended up with less than the strict buy-and-hold investor. This is the behavioral penalty.

Investors have been told that the booms and busts of recent years are unprecedented. Really, booms and busts are two of the few things that haven’t changed. A 50 percent drop in the market is not an anomaly. The long-term investor should have a strategy in place for dealing with such declines, rather than hoping to come up with something in the heat of the moment. This is an area where intuition is all too fallible.

Nobody should stake everything on US stocks. Small investors are advised to build a portfolio of low-cost funds investing in bonds or Treasury bills, inflation-indexed bonds (TIPS), American and foreign stocks, and commercial real estate. The PE momentum system is to be applied only to that part of your portfolio allocated to US stocks (ideally, an S&P 500 fund). Some may prefer to apply it only to part of their US stock allocation. If you can’t bear the thought of ever being completely out of the US market, you might use it for half your allocation, or any fraction you like.

To recap, here’s how the system works. Put a reminder on your calendar, once a month, to check the ten-year PE. (One source is at www.multpl.com, where you click on “Shiller PE”). As long as the PE remains below 24, you do nothing.

The first month that the PE moves above 24, make a note of the PE value. Don’t sell anything yet. Each month afterward, record the latest PE value. You sell only on a downturn. When the current month’s PE is at least 6 percent less than the highest monthly value since it rose above 24, sell your US stock index fund (or whatever fraction is being managed with this system) and put the money in safe fixed-income investments. Since you’ll probably have a permanent allocation to fixed income as well, you’ll need to keep a record of how much is the stock fund proceeds.

Example. The Shiller PE is above 24 when you check in June. By November you’ve got this list:

June, 24.03

July, 24.60

August, 24.41

September, 27.45

October, 27.32

November, 25.53

The highest value in the list is 27.45, for September. The current (November) value is 25.53, which is more than 6 percent short of the 27.45 maximum. That means you should sell.

Note that there were also pullbacks in August and October, but neither was big enough to trigger a sale. In August the highest value on the list was 24.60 for July (obviously you didn’t then have the values beyond August). The August pullback was only 1 percent from the July maximum and should not have triggered a sale.

You remain parked in fixed-income investments until you get a buy signal. This will usually take several years. Keep checking the PE monthly. When it first dips below 15, note the month and PE. Each month thereafter, add the latest PE value to the list. Eventually there will be an upturn. When the current month’s PE is at least 6 percent above the recent low value, buy back into the stock index fund, using the proceeds of the fixed-income investments you bought at the previous sell signal (plus reinvested interest).

With that the cycle renews. You go back to waiting for a sell signal. Meanwhile, reinvest dividends and capital gains distributions, and keep making contributions. All reinvestments and new contributions go to the current allocation.

A well-informed (and disciplined) investor can make adjustments for interest rates and other factors that we haven’t considered. In recent years interest rates on fixed-income investments have been extraordinarily low by historic standards. That makes bonds less attractive as an alternative and presumably makes higher PE valuations less risky. You might therefore move the 15 and 24 thresholds up a bit. But for those who don’t trust their instincts, the 15 and 24 limits would have worked well in recent decades without any adjustments.

Can someone using a PE momentum system expect a similar advantage going forward? No, that would be unrealistic for a couple of reasons. The familiar caveat is that the future could be different from the past, perhaps in ways we can’t envision. This applies to any trading system, including having no system at all.

There is also a subtler issue. A trading system can be “overfitted” to the data. Economist and money manager David J. Leinweber supplied a classic example. He searched UN statistics to determine that the best predictor of S&P 500 performance was… butter production in Bangladesh.

The connection was, of course, just a coincidence. Leinweber’s point was that not all that correlates is gold. While no one would be so daft as to use butter production as a buy signal for stocks, it’s not always easy to tell what’s a useful predictor. It’s been seriously or semiseriously proposed that hemlines, sunspots, and the political party in the White House predict stock market returns.

It’s best to favor systems offering a believable economic reason for their outperformance. How much you pay for earnings affects how much you can expect to realize from dividends and capital appreciation, and how attractive bonds are as an alternative. This is true even without the also-reasonable supposition that PEs that go up must eventually come down.

It would be asking too much to expect that the limit values that were optimal for the past 132 years would be exactly optimal for the coming decades. Even so, a look at the table of returns shows that not-quite-optimal limits can beat the market while reducing risk.

Ideally, you’d want to apply a similar strategy to other volatile parts of your portfolio: developed market funds, emerging market funds, and funds investing in real estate investment trusts (REITs). Like the S&P 500, they have shown wild swings in PE valuation. Unfortunately, it’s not easy to look up ten-year, inflation-adjusted PEs for these investments, or to find the historical earnings needed to calculate them.

In the absence of earnings data, a ten-year, inflation-adjusted moving average of price can be a helpful benchmark. Anyone comfortable with spreadsheets can compute that. A current price exceeding 150 percent of the ten-year moving average is high. You should scale down your return expectations accordingly—and may want to trim your allocation as well. It would be realistic to restore the allocation when the price dips below the ten-year average.

When an investing system really works, people adopt it. Widespread adoption usually reduces or nullifies the benefit. What if PE-based trading systems become popular?

PE ratios have been a standard measure of valuation since the early twentieth century. The Shiller PE is now part of the education of any earnest investor. Yet most people don’t act on what PEs are telling us. Investors continue to have short memories and believe that the recent past foretells the future.

Above all, the PE momentum system demands patience. It involves a few extra trades in an investor’s lifetime, and they may be spaced decades apart. You have to stick with the PE-checking regimen through years of inactivity. You have to ignore the “new era” apologists who pop up like mushrooms in the loam of every overvalued market. Few investors have that kind of staying power. But that may boost the likelihood that the system will continue to work for those who stick with it.

Recap: How to Outguess the Stock Market

• When the Shiller PE is high, future stock market returns are likely to be low. Investors should defer buying stock funds when the PE is high and wait for better opportunities.

• For better returns yet, investors can sell stock funds when the PE is sufficiently high. A simple system is to sell when the Shiller PE hits 28 and buy back into the market when it hits 13.

• For still better returns, take advantage of hot hand momentum. When the Shiller PE rises above 24, check the market monthly and sell whenever the market falls 6 percent or more from a recent high. Keep the proceeds in fixed-income investments. Then, when the Shiller PE falls below 15, check the market monthly and buy back into the market whenever the PE rises 6 percent or more from a recent low.

• By sitting out the market’s most turbulent times, these systems reduce risk, and that may be as important as the extra return.