From Flash Crashes to Economic Meltdowns: Feedback
Without calling the overall national issue a bubble, it’s pretty clear that it’s an unsustainable underlying pattern.
—Alan Greenspan
On Thursday, May 6, 2010, at 2:32 p.m. Eastern Standard Time, a sequence of events began that led to chaos in the securities markets for the next half an hour. During the first few minutes of this period, major United States equity indices plummeted 5–6 percent (see Figure 3.1). At 2:45 p.m. a five-second trading pause was imposed on the market, and the indices miraculously rebounded. However, the tsunami that started in the indices began to wash over the entire equities market. Trading prices for more than three hundred individual stocks deviated by more than 60 percent of their previous values. As liquidity dried up in some stocks, markets failed, and at the extreme the prices of shares in formerly well-regarded companies began to fluctuate wildly, with a company such as Accenture, previously trading at $40, selling for only a penny, and shares of Apple quickly moving from $250 to $100,000. By 3:00 p.m. the tsunami had subsided, and the markets returned to more normal behavior.
What could have possibly started such turmoil? Was there some news report of a cataclysmic event, like a major war breaking out or the assassination of a key world leader? Did some European country suddenly default on its debt? Was there some terrorist strike on the homeland or cyberattack on the trading system? Alas, the triggering event was at once far more mundane, and far more worrisome, than any of these.
The proximate cause of the above turmoil appears to have been a set of trades initiated by a money-managing firm whose address was a post office box in Shawnee Mission, Kansas. This firm used a computerized trading program to sell some securities, relying on an algorithm that tied its trading behavior only to the current volume of trades on the market, rather than to a more obvious factor such as the security’s price. While in hindsight it is easy to see how such a program could induce a ripple in the market waters, the far greater concern is how the ever-growing set of interconnections and interactions across the complex financial system was sufficient to allow this ripple to grow into a full-fledged tsunami that, at least for half an hour, wreaked havoc upon the financial shores.
In September 2010, the US Commodity Futures Trading Commission and the US Securities and Exchange Commission released a joint report on the aptly named “flash crash,” entitled Findings Regarding the Market Events of May 6, 2010, and many of the market details below are drawn from this source. The report provides a detailed economic autopsy of the events that unfolded on that day, and it’s rather good reading (and easily available by download), all things considered, for those inclined to explore the intimate details. Like any good autopsy, contained within its dry descriptions and careful analyses is a remarkable story of how the death came about. Yet the real intrigue comes from what is not discussed, namely, the mystery about who did it, why, and whether it could have been avoided.
May 6, 2010, had begun with the financial markets already on edge. The European debt crisis dominated the political and economic landscape, especially the possibility that Greece might default on its debts. Changes in various market indicators, such as the expected stock market volatility, the premium on debt insurance, the exchange rate of the euro, and the prices of gold and safe securities such as Treasuries, all reflected the unease brought about by these conditions. These changes likely pushed the markets toward a critical state (an idea that we will explore in Chapter 11), where even a small event had the potential to cascade into a much larger chain reaction.
The start of the tsunami began innocently enough. A firm that managed mutual funds wanted to hedge its existing equity positions against future changes in the US equities market. This is a common desire, as presumably the firm wanted to lock in some profits it had recently gained against the possibility that the US equities market might decline. To accomplish this hedge, the firm wanted to sell 75,000 E-mini futures contracts that would become due that June. E-minis are a derivative security, that is, their value is tied to something else—in this case, each one is worth fifty times the value of the S&P 500 Index (which represents about 70 percent of the market capitalization of all US-listed equities). Thus, if the S&P 500 Index is at $1,000, each E-mini is worth fifty times that, or $50,000. The 75,000 contracts represented roughly 3.4 percent of the average daily volume traded during 2010, and they were worth a total of about $4.1 billion at the prevailing price. Having a single individual wanting to sell 75,000 contracts is unusual, although there had been two occasions in the preceding twelve months where trades of this size or larger had been conducted.
Such trades are not without hazards. The problem with selling such a large number of shares all at once is that you can easily cause the price to plummet if you are not careful. Suppose you decide to sell a large number of shares by just dumping them all on the market at once. Initially, if the market is liquid, there are some buyers around to purchase your shares at roughly the going price. As these buyers’ demands are satisfied they leave the market, and your shares begin to flow to preexisting offers to buy that have been recorded by the exchange in its “order book.” As your shares satisfy the highest of these preexisting offers, they next flow to lower ones, and on and on. As your orders eat well into the order book, prices continue to fall, and any potential new buyers coming onto the market recognize what is happening and make much less aggressive offers in anticipation of lower prices, putting even more downward pressure on prices. While any large offer to sell will have a tendency to depress the overall price regardless, given the microdynamics of the order book, dumping the shares on the market all at once will have a larger, short-lived impact, resulting in the seller getting much worse prices overall than she could have gotten if the shares were sold more slowly.
Thus, to get the best prices possible for a large order, the seller needs to carefully manage the block of trades and slowly release shares onto the market. This allows new buyers to find their way to the market during the sale, refilling the order book in the process, and ultimately resulting in the seller getting much higher prices overall for the entire lot.
One way to manage the sale of a large block of trades is to employ an automated trading algorithm that executes the orders in a reasonable manner. Such an algorithm should be programmed to track key data from the market, such as the current trading volume, price, and time of day. Based on this information, the computer will release the shares so as to get the best deals possible consistent with moving the entire block in a timely manner.
The firm that catalyzed the flash crash used such an algorithm. Of course, the devil is in the details, and in this case there was a devil indeed. The firm’s algorithm had one simple rule: feed in orders to the market so that these orders constitute less than 9 percent of the overall trading volume during the previous minute. Note that this algorithm completely ignores the trading price. That being said, at some level it is not a completely absurd algorithm, as normally volume is a good indicator of the market’s liquidity, and liquidity is tied to stable prices. In theory, if you remain a small part of the market (here, less than 9 percent) and the market is functioning in a “normal” way, this algorithm should result in stable and reasonable prices. In essence, the algorithm hitches a free ride on the volume information coming from the market and uses this as a proxy for reasonable prices. By doing so, it avoids having to make any difficult predictions about when to sell.
Unfortunately, there have been two recent changes in trading that have made this volume-based proxy quite dangerous. First, the rise of derivative securities has interconnected various markets. E-minis are linked to the S&P 500 Index. There are other derivatives with slightly different designs, such as S&P Depositary Receipts (known as SPDRs or “Spiders” and traded under the ticker symbol SPY), that are also tied to this index. If the price of one of these derivatives differs substantially from the others, there is an arbitrage opportunity to lock in a profit, regardless of what happens to the underlying prices, by selling the more expensive security and buying the cheaper one to make good on the previous sale. An alternative arbitrage opportunity interconnects the derivatives market to the broader markets: given that the price of the derivatives is tied to a bundle of individual stocks (which make up the index), you can always make a profit by offsetting the purchase (or sale) of the derivative by selling (or buying) the underlying bundle of stocks whenever the price of that bundle differs from the price of the associated derivative.
This latter opportunity is facilitated by the second major change we have seen in markets, namely, the ability to get information about the trading conditions across a vast array of securities and markets, calculate potential opportunities, and execute any needed trades, all in the blink of an eye. This revolution in trading is due to the rise of the computer and, indeed, things now happen far faster than the blink of an eye (which takes a somewhat poky 350 milliseconds, a length of time in which an electron can travel more than 65,000 miles).
The combination of highly connected markets and quickly conducted trades has formed a new kind of complex system unforeseen even a decade ago. A trade in one market reverberates across the others, as the various inconsistencies it induces get corrected. Of course, those corrections can start their own reverberations. If the (unintended) feedback loops emerging from the various connections are negative, the reverberations in the system slowly die out, and the markets are better for the experience, as prices realign with one another. If, however, the feedback loops are positive, we end up with the reverberations amplifying one another, creating something akin to the horrible screeching sound we hear when a microphone is held too close to a loudspeaker.
Over the past few years, a new kind of trading firm has arisen on the market scene: the high-frequency trader (HFT). These firms have fully embraced the information age and have created algorithmic traders that watch over markets and execute any desirable trades on a remarkably short time scale. For this type of trading, getting your buy or sell message to the exchange before anyone else is so important that factors such as where you physically locate your computer hardware matter—an electron travels about a foot each nanosecond, so every foot closer to the exchange’s machines gives you a nanosecond advantage over your competitors.
HFTs now account for an enormous volume of trades. In general, they don’t like holding too many shares at any one time. Thus, while they might buy a lot, they also sell a lot, so at the end of the day (though the new reality is that with global, interconnected markets, there really is no end of the day) their net holdings of any given security are small.
The existence of HFTs certainly alters the dynamics of markets. In the late 1980s at the Santa Fe Institute, my colleagues Richard Palmer, John Rust, and I created the Double Auction Tournament to test some core ideas about markets. Academics, professional traders, and interested amateurs tested their trading strategies over the web (to our knowledge, this was the first Internet-based auction) and then submitted final versions to us for analysis.
One of our interests was what would happen in a hybrid market composed of both machine and human traders. When we ran such a market without any compensation for the innate speed differences between humans and machines, we found that when the market opened there was a flurry of trades by the machines before the humans could even react. After that point, the machines just stayed quietly in the background as the humans traded with each other, activating only when a human made a bad offer, in which case a machine would jump into the market and steal the deal.
One suspects that our current market system, with both human traders and HFTs, may behave in an analogous way to the hybrid Double Auction Tournament. HFTs’ distinct speed advantages may be causing flurries of machine trading, punctuated by quieter periods where the machines remain in the background waiting to take advantage of human errors. When we eliminated the speed advantages of the machines, humans and machines easily coexisted and were difficult to tell apart given the data, with the one exception being that humans tended to place offers that ended in digits of either zero or five, while machines were not so constrained.
Returning to that fateful day in May, at 2:32 p.m. our trader presumably pushed the enter key in response to some innocuous-sounding prompt along the lines of “Are you sure you want to execute these trades?” and a stone was dropped into the market pond, causing a small ripple. The market was able to absorb the initial trade volume, as HFTs and other intermediaries bought the newly offered contracts. Over the next ten minutes, the HFTs accumulated quite a few of these contracts, and in order to balance out their positions, they began to sell. A game of high-stakes hot potato ensued, in which the HFTs began to buy and sell to one another, with only occasional leaks of the contracts out to other market participants.
It is at this point that the fatal flaw in the algorithm becomes apparent. The game of hot potato started to generate a lot of market volume, with more than 100,000 shares being exchanged in a very short time. The algorithm, blind to everything but the volume, saw this increased activity as a sign that the market was liquid and that prices were stable, and it began to dump even more shares into an already volatile mix. This new action destabilized things even further, as any real liquidity in the E-mini shares dried up and the prices began to plummet. In the thirteen minutes after the enter key was pressed, the algorithm sold 35,000 contracts, and the remaining 40,000 contracts were sold off in a scant seven minutes more. Thus, all of the initial 75,000 contracts were sold in under twenty minutes—whereas in the past, using more standard algorithms, it had taken around six hours to dispose of similarly sized lots. The initial trades and subsequent activity, not surprisingly, resulted in a substantial drop in the price of the E-mini contracts (see Figure 3.2).
However, at 2:45:28 p.m. an event happened that likely prevented an even deeper disaster. At that time, an automated mechanism paused trading for five seconds. This mechanism had been put in place by the exchange, and it was designed to recognize market conditions in which the execution of further trades would result in unnaturally large price swings. While five seconds seems like an inconsequential amount of time, it is an eternity in an era when nanoseconds rule. It was long enough to allow other traders to enter the market and get things on the road to recovery. Over the next twenty-three minutes, buyers with a more fundamental focus began to flood the market, and prices rebounded.
The proximate cause of the bad behavior in the E-mini market is easily tied to the flaw in the trading algorithm. By linking the number of trades to only volume, a positive feedback loop was unintentionally embedded in the algorithm: if the initial trades cause a big increase in volume, then the algorithm trades even more, which will further increase volume. If the HFTs had not been in the picture, the naive trading algorithm might not have induced enough extra trades to trigger the feedback loop. However, with the rapidly trading HFTs and their desire to maintain relatively neutral share positions, a new market dynamic formed that embedded a positive feedback loop into the system.
If this were just the story of the E-mini market, it would be worth telling as a parable about algorithmic—actually, human—hubris and the dangers of unintended consequences and positive feedback. But the story does not end here.
Given the interconnectedness of markets, what happens in the E-mini market does not stay there. As the E-mini declined in value, traders started to look for arbitrage opportunities elsewhere—in this case, either in SPDRs or in the stocks that make up the index itself. While the E-mini’s price was rapidly declining, driven by the positive feedback loop, the prices of SPDRs and the stocks that formed the index moved much more slowly. This created a new opportunity to profit by buying the relatively cheap E-minis and selling their more expensive equivalents in the form of SPDRs or the bundle of underlying stocks.
In a well-functioning market system, the arbitrage opportunities created by the collapsing E-minis would normally dampen the price dynamics. The profit-seeking activities of the arbitrageurs would raise the price of the E-mini (given the newfound demand to buy) and lower the price of the SPDR or bundle (given the newfound desire to sell), and the prices would realign and remove the opportunity for profitable arbitrage. Unfortunately, given the preexisting turmoil and the positive feedback loop, the markets failed to realign very quickly, and the arbitrage opportunity remained. This resulted in trading pressure on the other markets, and they started to eat into their respective order books as well. Moreover, the newly generated chaos made many potential market makers nervous, as nothing in the incoming data streams—which by this time were starting to falter given the massive influx of trading—could account for the large price changes being observed. This triggered data integrity checks, where firms paused their trading activity. Other firms withdrew entirely from the market as automatic systems that continuously monitor a firm’s position and potential exposure to financial risk began to exceed preset limits and halted the firm’s trading. Finally, in some firms, humans overseeing all of this bizarre activity simply lost their nerve (or behaved wisely) and withdrew their offers from the market.
As the market makers withdrew, the order books began to empty out, leaving only long-standing orders and, at the extremes, automated “stub” orders set at ridiculous price points just to ensure that there would always be someone willing to buy or sell any given share. Thus, the transactions that did occur were happening at prices that became more and more extreme over time. More than three hundred stocks experienced price changes of as much as 60 percent (more than 20,000 trades, constituting 5.5 million shares, were executed at such extremes). At the most extreme, securities were traded at their stub prices, with some shares going for a penny and others for $100,000.
The aftermath of the events of May 6, 2010, was significant. In the short term, there was a realization that the events were far from the “fair and orderly” markets that the exchanges want to oversee, and the trades that took place far from the prices prevailing just before the chaos began were broken by the exchanges, as they were considered “clearly unrealistic prices” that were “clearly erroneous” given the severe market conditions. While exchanges have always had the power to break such trades (always read the fine print), the actual mechanisms used for determining “clearly erroneous” were not well defined, and this has prompted a reform in this area. The second major reform has altered how various circuit breakers get deployed. Individual markets often have mechanisms designed to halt trading when unexpected conditions arise, and in practice, even very short halts have allowed markets to stabilize quickly and resume in an orderly fashion. Unfortunately, even the existence of circuit breakers can have unintended consequences, as multiple halts in a given security might cause market makers to withdraw their liquidity. Also, given global connectivity and many markets trading the same security, a trading halt in one market might just shift the displaced trades to another market, circumventing the original breaker and exacerbating the problem.
The one area that has not been reformed is limiting the HFTs. For example, the feedback loops induced by the HFTs could be dampened by imposing transaction taxes or redesigning markets to lessen the importance of nanosecond-scale speed.
Even the above repairs do not address the fundamental problem that caused the flash crash. We have unknowingly created a complex adaptive financial system that we do not understand and cannot control. At each stage of its creation, we have accrued additional complexity in the name of added benefits: connecting markets with one another will ensure that price discrepancies will be eliminated quickly, having high-frequency traders will guarantee a ready trading partner for any transaction, using derivatives will provide a means for farmers to hedge the risks of bad weather and for pension funds to insure their portfolios, and so on. While each of these individual pieces makes sense, the collection may not.
As we have already seen, reductionism does not imply constructionism. Thus, while the motivation for, and understanding of, any single piece in the system may be sound, that should not give us any confidence in the behavior of the whole. The flash crash occurred not by design but through emergence.
The flash crash was a surprisingly gentle warning that we must heed. The events during that thirty-minute period in May, while striking, were reversible. While careful autopsies of dramatic events are useful, we need to be in a position to prevent the appearance of the bodies in the first place. Unfortunately, the flash crash has shown us that, however good our retrospective investigations might be, our prospective knowledge is weak. We can’t even begin to grasp the implications of the financial systems we have built.
While the flash crash was driven by greed in the pursuit of profit, it fortunately involved ignorance, not malice. Imagine the chaos and long-term devastation that could happen if malice and a bit more forethought were involved. How difficult would it be for, say, a terrorist organization or rogue state to infiltrate either the computer or human systems that underlie our markets and wreak havoc on a much larger, and longer-lasting, scale? This does not seem all that hard. Attacks on the cyber infrastructure, such as cracking the actual systems of the exchange or those of the numerous decentralized trading operations, or somehow disrupting or altering the communication flows that direct or report trades, seem possible, especially given examples such as the Stuxnet computer worm, which hampered Iran’s ability to enrich uranium. The human systems connected with financial institutions are also vulnerable. Indeed, there are examples where the actions of a single trader brought down an entire institution, as with the fall of the 233-year-old Barings Bank in 1995. Thus, inserting one or more traders into the system with enough access to the trading desks to launch a carefully coordinated, malicious attack is feasible. A more ambitious approach might include setting up an apparently legitimate fund or HFT operation that gets privileged and unfettered access to the trading systems, or, if that is too bothersome, simply executing a large number of simultaneous transactions spread across legitimate traders. The impact of such an attack is hard to predict, but at the very least it would seriously erode confidence, and it could be far more consequential, leading to a partial collapse of the very markets that ensure our economic survival.
Unfortunately, the story encapsulated in the flash crash may not be all that unique. Indeed, the recent worldwide financial collapse that started in 2008 has similar undertones.
At the heart of the 2008 financial collapse was an economic crisis that fully embraced all of the seven deadly sins. Gluttonous fixed-income-asset buyers, for the promise of slightly higher returns, were willing to buy up newly formed collateralized debt obligations. Extravagant home buyers, hoping that rising house prices would allow refinancing in the future, opted for houses and ballooning mortgage payments well beyond their current means. Greedy mortgage brokers, able to pass on even suspect mortgages to firms that created and quickly sold off mortgage-backed securities, were willing to qualify almost any buyer. Envious firms, wanting to boost their bottom lines, began leveraging themselves while marketing suspect derivatives to their customers. Slothful rating agencies, relying on the word of the firms and outdated statistical models, gave absurdly high ratings to novel securities while collecting commissions. Prideful government agencies, relishing the increase in home ownership and the power of the unregulated market, stood idly by. As for wrath, hell hath no fury like a complex economic system scorned.
The point of the previous paragraph is not to tell some modern morality tale but rather to emphasize how, at each level of the system, the entities involved were following perfectly understandable—though perhaps not virtuous—incentives. Thus, in a very real sense, economists and policy makers were fully equipped to understand each part of the system. Unfortunately, as we have seen before, thinking that understanding the parts of a system implies that you understand the whole system is a sin that is committed all too often.
As we saw in the case of the flash crash, positive feedback mechanisms amplify small events into large ones. The housing market is rife with positive feedbacks. If mortgage money becomes easier to get, the demand for houses goes up, resulting in higher house prices. These higher house prices make lenders more willing to grant mortgages, as rising prices ensure that sufficient collateral exists to lower the risk of the loans.
In the US housing market, the positive feedbacks tended to reinforce every part of the system. Higher house prices encouraged more buyers, lowered lending standards, and resulted in less risky derivatives and easier government policies, and each of these fed back on the others, reinforcing the chain of effects. Alas, the same forces that amplified the system on the way up accelerated its demise on the way down. Unfortunately, unlike with the flash crash, there were few circuit breakers, or anything like them, in place during the financial collapse.
The interactions and connections among the various parts of the system are critical here. Imagine any of the key markets associated with the financial collapse as a timber farm along a lightning-prone ridge. Every now and then, lightning strikes, and if it hits a tree, that tree goes up in flames and ignites any neighboring trees. If you want to maximize the timber harvest, you must make a trade-off between growing more trees to get more timber and keeping land fallow to contain neighboring fires. The best choice here depends on various underlying factors, such as the frequency of lightning and the growth rate of trees, but whether the best choice gets made depends on who owns the ridge. If a single person owns the ridge, it will be in her interest to include a few firebreaks, so that a single spark won’t lead to a conflagration that takes out the entire ridge. Unfortunately, such firebreaks may not arise in a system where each potential tree site is owned by a different individual following her own incentives. In this situation, while all individuals would benefit from the inclusion of firebreaks, no individual wants to be the person who provides the firebreak, since she would have no timber to harvest. In economic terms, firebreaks are underprovided, and this results in far more destructive fires and much lower harvests than are possible under a more coordinated regime.
So it was with mortgages at the start of the financial crisis. No entity wanted to forgo any possible trade and lose some immediate profit. Thus, a single bank finds it individually profitable to hold securities issued by another bank, even though that other bank has bought securities from another bank, and so on down the line, to the point where the failure of a very distant bank can cause the whole system of promises to unravel. Similarly, a single firm may simultaneously buy and sell insurance-like policies on the risk of default (known as credit default swaps) and feel that its position is safe, since any loss to one of the policies will be perfectly offset by a gain to the other. However, if one firm fails to meet its obligation to pay in the case of a default (think American International Group, aka AIG), the entire system unravels. In these and countless other situations, what is important here is the chain of connections that results from individually rational, but globally irrational, arrangements. Without well-placed firebreaks, these systems are subject to small events having catastrophic consequences.
In both the 2008 financial collapse and the flash crash, we saw systems that were vital and thriving at one moment suddenly become quiescent. This type of switching happens in a variety of complex systems. For example, a living organism exists in a dynamic state where its many interacting parts result in a vital and robust organism. Now introduce, say, a well-placed shock, and the once vital organism is pushed into a death state where none of its parts interact. Unfortunately, this too is a robust state.
Expectations often keep social systems, and especially markets, operating. Expectations can lead to self-fulfilling prophecies, both good and bad. Thus, in a flash crash, once liquidity dries up, the expectations of market makers may change to the point where they believe they will no longer be able to find reasonable trading partners, which causes them to withdraw their orders and realize their expectations, further exacerbating the liquidity crisis. Once a housing bubble begins to pop, the downward spiral of house prices alters the expectations of the lenders, and they become wary of granting new (or refinancing old) mortgages without extreme levels of collateral, which in turn causes prices to fall and the newly formed expectations to be reinforced. In both cases, feedback loops concerning expectations exacerbate a bad situation.
When emergence is working for you, the invisible hand of Adam Smith is a wonderful thing. Life would be a lot more fun, albeit much less intriguing, if emergence arose only when it led to good things. Unfortunately, we have seen the dark side of emergence, in which a seemingly innocuous event triggers a cascade that leads to disaster. Complex systems, whether intentional or not, are playing an increasingly important role in our world. While we might not ever be able to fully control such systems, we may be able to mitigate their downsides through the clever introduction of metaphorical firebreaks such as the circuit breakers that are used in financial markets. Our understanding of how to create such controls is lagging well behind our need to implement them, and we must quickly develop this knowledge so that the kingdom won’t be lost for want of a nail.