5

Physics Hits the Street

IN FEBRUARY 1961, FISCHER BLACK’S PHD ADVISOR, ANTHONY OETTINGER, WROTE TO HARVARD’S COMMITTEE ON HIGHER DEGREES: “I have reason to be concerned about [Black’s] intellectual discipline so that, while recognizing his ability and his desire for independence, I am concerned lest he lapse into dilettantism.” Two months later, Oettinger chaired an oral exam designed to determine whether Black was prepared to advance to the dissertation stage of his doctorate. Black passed — but with an explicit requirement that he produce “a coherent, lucid thesis outline” by January 1962. Within a week, Black was in jail for participating in student riots in Harvard Square, and when a Harvard dean went to bail him out, Black was unrepentant. He railed against police authority, against Harvard’s authority, and against his advisor. January 1962 came and went, and Black had done no more work toward his thesis. He was informed that he could not return to Harvard.

Today, Fischer Black is one of the most famous figures in the history of finance. His most important contribution, the Black-Scholes (sometimes Black-Scholes-Merton) model of options pricing, remains the standard by which all other derivatives models are measured. In 1997, Black’s collaborators, Myron Scholes and Robert Merton, were awarded the Nobel Prize in economics for the Black-Scholes model. Black had died in 1995 and so was ineligible for the prize (the Nobel is never awarded posthumously), but in a rare nod, the Nobel committee explicitly acknowledged Black’s contribution in its announcement of the award. Every two years, the American Finance Association awards the Fischer Black Prize — one of the most prestigious awards in academic finance — to an individual under forty whose body of work “best exemplifies the Fischer Black hallmark of developing original research that is relevant to finance practice.” MIT’s Sloan School of Management endowed a chair in financial economics in Black’s honor. And the list goes on.

In the broad history of physics in finance, Black is perhaps best seen as a transitional figure. He was trained as a physicist but was never successful as one — in large part because he was too wide-ranging and unfocused. Though he was more successful as a financial economist, his career was fleeting, as he quickly became bored with the projects that made him famous and turned to new ideas that were met with much more skepticism. Yet these very qualities — the qualities that Oettinger worried would lead Black to dilettantism — were what allowed Black to bring about a marriage long in the waiting. He was enough of a physicist to understand and develop the insights of people like Bachelier and Osborne, and yet he was enough of an economist to express his discoveries in a language economists could understand. In these ways he was like Samuelson, though he was never as intellectually distinguished. But unlike Samuelson, Black was able to communicate to investors and Wall Street bankers how the new ideas coming out of physics could be used in practice. Thorp was the first person to figure out how to use Bachelier’s and Osborne’s random walk hypothesis to make a profit, but he did so outside of the establishment, through Princeton-Newport Partners. Black, on the other hand, was the person who made quantitative finance, with its deep roots in physics, an essential part of investment banking. Black took physics to the Street.

Black first arrived at Harvard in 1955, at age seventeen. If anyone asked why he applied to Harvard and nowhere else, he would say it was because he liked to sing and Harvard had a great glee club. From the very beginning, he was determined to chart his own course through academia. He refused to do the work he was assigned and instead wrote papers on topics he decided were interesting. After a few semesters of introductory courses, he decided to enroll in graduate classes. He picked an interdisciplinary major called “social relations,” which combined several social science disciplines, and then promptly began conducting experiments with himself as the subject. For instance, he would modify his sleep schedule, alternating between four hours awake and four hours asleep, all while taking careful and copious notes on how his body reacted. He began taking drugs, including hallucinogens, and tracking the effects. Most of his friends were graduate students.

Come junior year, however, he started having second thoughts about his choice of major. Social relations was interesting, but Black wanted a career in research. Like Osborne and Thorp, Black was a natural-born scientist, constantly experimenting and coming up with theories to test, and he just didn’t see how social relations could get him the kind of job he wanted. So he turned toward the hard sciences, flirting with chemistry and biology before finally settling on physics. He wanted to do fundamental, theoretical work, and so the next year he applied to graduate school, once again only to Harvard, to do a PhD in theoretical physics. He won a prestigious National Science Foundation graduate student fellowship and Harvard admitted him. In the fall of 1959, Black started graduate school as a physicist.

But by the end of his first year, his attention had begun to stray again. He took only one physics course, filling his first year instead with electrical engineering, philosophy, and mathematics. He was a little interested in everything, but not enough interested in anything to stay focused for long. After just a few weeks, he switched departments, to study applied mathematics instead of physics; then, come spring semester, he was devoting all of his time to an artificial intelligence course at MIT, taught by AI pioneer Marvin Minsky; by fall 1960, he was back to the social sciences, taking two courses in psychology.

It would be wrong to say that Black did poorly in school. But his tack was certainly unconventional. On the one hand, he barely passed some courses — including the one physics course he enrolled in. During his second year he failed a psychology course because it emphasized “behavioralist” methods, while Black saw himself as aligned with the newer, more fashionable “cognitivist” school. But he was certainly one of the best minds at Harvard. In an open competition during his first year, he successfully solved a challenge problem offered by one of his mathematics professors, which earned him an endowed scholarship for the following year. And so his abilities were never really in doubt. It is nonetheless easy to see Oettinger’s worry: two years into graduate school, Black was no closer to settling on a major than he was as an undergraduate. If anything, the rate at which he swung from discipline to discipline was accelerating. As Black saw it, he was simply curious, and he wasn’t going to be pinned down by some stodgy old school’s rules about what constituted appropriate academic work — even if it meant leaving Harvard.

Ultimately, Black did earn a PhD in applied mathematics. But he took the scenic route. When Harvard asked him to leave, he found a job at Bolt, Beranek and Newman (BBN), a Cambridge-based high-tech consulting firm. BBN hired Black because of his computer skills, and most of his time there was spent working on computerized data retrieval systems for a project commissioned by the Council on Library Resources. As part of this project, Black wrote a program that used formal logic to try to answer simple questions. The program would take an input such as “What is the capital of Romania?” and try to deduce an answer based on a list of facts it had stored in a database. A major part of this project was devoted to simply parsing the question, trying to determine what the questioner was even after. Black’s work represented an important early contribution to the field known as computational linguistics, in which people try to figure out how to make computers understand and produce natural language.

Word spread quickly around Cambridge of Black’s work at BBN. In the spring of 1963, Minsky heard about Black’s question-answering program. He was sufficiently impressed — and sufficiently influential — that he negotiated readmission to Harvard on Black’s behalf. Minsky took responsibility for Black’s work, with a professor at Harvard named Patrick Fischer serving as the official advisor. Over the next year, Black turned his consulting project into a dissertation on deductive question-answering systems, which he successfully defended in June 1964.

But by this time, Black had had enough of academia — at least for a while. He had settled on a project long enough to write a dissertation, but this hardly signaled a lifelong devotion to artificial intelligence. He thought about becoming a writer, working on popular nonfiction projects. Or maybe he would go into the computer business. He considered applying for a postdoctoral fellowship to stay at Harvard and work on the interface between technology and society — a new subject, spurred by new postwar technologies. But ultimately nothing panned out, and so after graduating, Black returned to consulting. At least there, he could work on many different projects, and he had already discovered that solving concrete problems appealed to him.

Instead of returning to BBN, however, Black took a job with another local firm, called Arthur D. Little, Inc. (ADL), in the Operations Research Division. At first Black worked primarily on computer problems. For instance, MetLife had a state-of-the art computer, but the company still felt as if its computation needs weren’t being met. MetLife hired ADL to see if a second computer was needed. Black, collaborating with two others at ADL, discovered that the problem wasn’t the computer, which was working at only half capacity, but rather the way in which the computer stored data: instead of using thirty available drives, it used only eight drives in everyday tasks. So Black and his team worked out an optimization scheme for using all of the available drives.

Black worked at ADL for about five years. The experience changed his life. When he arrived, he was an operations research and computer science guy. He had unusually broad interests, but there’s no evidence to suggest that finance was among them. When he left in 1969, he had already laid the foundation for the Black-Scholes model. He was recognized, at least in some circles, as an exciting, if radical, up-and-coming financial economist. Wells Fargo immediately hired him to develop a trading strategy.

This transformation began shortly after Black arrived at ADL, where he encountered a slightly older member of the operations research section named Jack Treynor. Treynor had gone to Haverford College intending to major in physics but decided that the department wasn’t very good, and so he switched to mathematics. After college he went to Harvard Business School and then joined ADL in 1956, a decade before Black would arrive. Treynor and Black didn’t overlap at ADL for long: in 1966, Treynor was wooed away by Merrill Lynch. But the two practically minded mathematicians made fast friends. Black liked Treynor’s way of thinking and quickly became interested in his work, primarily on risk management, hedge fund performance, and asset pricing. Although Treynor didn’t have a formal background in financial theory either, his business school background had exposed him to a set of problems that he was well suited to work on, and so much of his work at ADL involved financial institutions. Meanwhile, he worked on more theoretical research projects on the side, often motivated by the kinds of problems ADL clients encountered.

By the time Black arrived at ADL, Treynor had already developed a new way of understanding the relationship between risk, probability, and expected value, now known as the Capital Asset Pricing Model (CAPM). The basic idea underlying CAPM was that it should be possible to assign a price to risk. Risk, in this context, means uncertainty, or volatility. Certain kinds of assets — U.S. Treasury bonds, for instance — are essentially risk-free. Nonetheless, they yield a certain rate of return, so that if you invest in Treasury bonds, you are guaranteed to make money at a fixed rate. Most investments, however, are inherently risky. Treynor realized that it would be crazy to put your money into one of these risky investments, unless you could expect the risky investments to have a higher rate of return, at least on average, than the risk-free rate. Treynor called this additional return a risk premium because it represented the additional income an investor would demand before buying a risky asset. CAPM was a model that allowed you to link risk and return, via a cost-benefit analysis of risk premiums.

When Black learned about CAPM, he was immediately hooked. He found the simple relationship between uncertainty and profit deeply compelling. CAPM was a big-picture theory. It described the role of risk in making rational choices in a very abstract way. Later in his career, Black would point to one feature of CAPM in particular that he was drawn to: it was (in his words) an equilibrium theory. “Equilibrium was the concept that attracted me to finance and economics,” Black wrote in 1987. CAPM was an equilibrium theory because it described economic value as the natural balance between risk and reward. The idea that the world was in a constantly evolving equilibrium would have appealed to Black’s sensibilities as a physicist: in physics, one often finds that complicated systems tend toward states that are stable under small changes. These states are called equilibrium states because they, too, represent a kind of balance between different influences.

Black set out to learn everything that Treynor knew about finance, so that when Treynor left ADL, just a year after he and Black first met, Black was the natural person to take Treynor’s place on ADL’s financial consulting team — and further perfect Treynor’s model. CAPM would form the foundation for virtually all of the work Black would go on to do.

If Jack Treynor initiated Black’s transformation into a financial economist, Myron Scholes brought it to fruition. Scholes arrived in Cambridge in September 1968, a fresh doctorate from the University of Chicago in hand. A fellow graduate student in Chicago, Michael Jensen, had recommended that Scholes look up Black — an “interesting fellow,” in Jensen’s estimation. Scholes called soon after arriving in Cambridge. Both men were young: Scholes had just turned twenty-seven, and Black was thirty. Neither was particularly accomplished, though Scholes’s recent appointment as an assistant professor at MIT was a promising sign. They met over lunch in the drab, institutional cafeteria at ADL’s Acorn Park campus. One rarely imagines history unfolding over a cafeteria meal shared by undistinguished men. And yet, that first meeting between Black and Scholes was the start of a friendship that would change financial markets forever.

Black and Scholes were polar opposites. Black was quiet, even shy. Scholes was outgoing and brash. Black was interested in applied work, but he had a theoretical, abstract mind. Scholes, meanwhile, had just written a heavily empirical thesis, analyzing piles of data to test the efficient market hypothesis, which by this point had been elevated to a central principle of neoclassical economics. It is difficult to imagine how that first conversation could have gone. And yet, something clearly clicked. The two men met again, and then again. Soon they had laid the foundation for a lifelong friendship and intellectual partnership. Scholes invited Black to participate in the weekly MIT finance workshop, which was Black’s first opportunity to fully engage with finance academics. Soon after, Wells Fargo approached Scholes with an offer for a consulting arrangement, to help the bank implement some of the new ideas in finance, like CAPM, that were just bubbling to the surface in academia. Scholes felt that he didn’t have enough time to do the work himself, but he knew someone who would be perfect for the job. Black quickly agreed, and in March 1969, some six months after that first meeting in the ADL cantina, Black quit his job at ADL and went off on his own. He started a new consulting firm, called Associates in Finance, with Wells Fargo as his principal client. He and Scholes were tapped to help Wells Fargo create a new, state-of-the-art investment strategy.

It was around this time that Black began thinking about ways to extend CAPM to different kinds of assets and different kinds of portfolios. For instance, he tried to apply CAPM to the question of how to apportion one’s investments over time. Should you change your risk exposure as you get older, as some people were suggesting? Black decided the answer was no: just as you want to diversify over different stocks at a given time, you also want to diversify over different times, to minimize the impact that any particular stretch of bad luck might have. The question of how to value options using CAPM was just one of the many such problems that Black was working on at this time. And as early as the summer of 1969, Black had already made progress, by deriving the fundamental relationship that would ultimately give rise to the Black-Scholes equation.

The essential insight was that at any given instant, it is always possible to create a portfolio consisting of a stock and an option on that stock that would be perfectly risk-free. If this sounds familiar, it’s because the idea is very similar to the one at the heart of Thorp’s delta hedging strategy: he, too, realized that if the prices of options and their underlying assets are related, you could combine options and stocks to control risk. The difference was that Thorp’s delta hedge strategy aimed to guarantee a profit, provided that the underlying stock’s price didn’t change too dramatically. This approach controlled risk, but it didn’t eliminate it altogether. (Indeed, if CAPM-style reasoning is correct, you shouldn’t be able to both eliminate risk and still make a substantial profit.) Black’s approach was to find a portfolio consisting of stocks and options that was risk-free, and then argue by CAPM reasoning that this portfolio should be expected to earn the risk-free rate of return. Black’s strategy of building a risk-free asset from stocks and options is now called dynamic hedging.

Black had read Cootner’s collection of essays on the randomness of markets, and so he was familiar with Bachelier’s and Osborne’s work on the random walk hypothesis. This gave him a way to model how the underlying stock prices changed over time — which in turn gave him a way to understand how options prices must change over time, given the link he had discovered between options prices and stock prices. Once Black had found this fundamental relationship between the price of a stock, the price of an option on that stock, and the risk-free interest rate, it was just a few steps of algebra for him to derive an equation for the value of the option, by relating the risk premium on the stock to the risk premium on the option. But now he was stuck. The equation he derived was a complicated differential equation — an equation relating the instantaneous rate of change of the price of the option to the instantaneous rate of change of the price of the stock — and Black, despite his background in physics and mathematics, didn’t know enough math to solve it.

After struggling for several months, Black gave up. He didn’t tell anyone about the options problem, or his partial solution, until later in 1969, when Scholes mentioned that one of his master’s students at MIT was interested in options pricing. Scholes began to speculate about whether CAPM could be used to solve the problem — at which point Black opened his desk drawer and pulled out a sheet of paper with the crucial differential equation written on it, and from then on the two men worked on the problem together. They had solved it by summer 1970, and the Black-Scholes equation for the price of an option made its debut in July, at a conference that Scholes organized at MIT, sponsored by Wells Fargo. In the meantime, a new colleague of Scholes’s at MIT, Robert Merton (himself an engineer by training, though he went on to earn a PhD in economics), had rederived the same differential equation and the same solution from an entirely different starting point. With two different approaches giving the same answer, Black, Scholes, and Merton were convinced they were on to something big.

Black and Scholes submitted their paper to the Journal of Political Economy, one of the most important publications in the field, soon after they had solved the problem. The paper was promptly rejected, without so much as a note of explanation (suggesting it wasn’t even seriously considered). So they tried again, this time with Review of Economics and Statistics. Again, they received a rapid rejection with no articulation of what was wrong with the article. Merton, meanwhile, held off on sending his alternative approach to journals, so that Black and Scholes could receive appropriate credit for their discovery.

Despite the early setbacks, however, Black and Scholes were not destined to labor in obscurity. Powerful forces in academia, in finance, and in politics were aligning in their favor. And some of the then-reigning academic gods were ready to intervene. After the second rejection, University of Chicago professors Eugene Fama and Merton Miller, two of the most influential economists at the time and leaders of the then-nascent Chicago School of economics, successfully urged the Journal of Political Economy to reconsider, and in August 1971 the article was accepted for publication, pending revisions.

In the meantime, Fischer Black had attracted attention at the University of Chicago. Economists there were familiar with his work with Scholes, both on options and at Wells Fargo; they’d seen him in action at the Wells Fargo conference. A few years earlier, in 1967, Black had traveled to Chicago with Treynor to present some of their collaborative work to the graybeards. Chicago economists didn’t need fancy journals to vet young academics: they knew talent when they saw it, and Black certainly had talent. And so in May 1971, they offered Black a job. At this point, Black had already been out of graduate school for seven years, yet he had only four publications, just two of which were in finance. He had a PhD, but in an unrelated field. None of this mattered. Chicago wanted him.

Chicago wasn’t working on a hunch that Black’s work would become important. The faculty there had some inside information: options were about to become a really big deal — and a formula that allowed investors to price them would prove essential. Two major changes to U.S. and international policy were in the works, both centered in Chicago, that would soon revolutionize the derivatives industry. Having someone like Black on one’s team could only help.

The first major change took place on October 14, 1971, just a few weeks after Black arrived in Chicago. The Securities and Exchange Commission (SEC) gave the go-ahead to the Chicago Board Options Exchange (CBOE), the first open, dedicated options market in United States history. Options had been around for hundreds of years, and they had been traded in the United States, often in the guise of warrants, since at least the middle of the nineteenth century. But they had never been traded on an open market before. Economists in Chicago had been agitating for the SEC to remove barriers to an open options exchange for years, until finally they convinced the Chicago Board of Trade (CBOT) to convene a committee to consider the possibility, in 1969. The head of that committee was James Lorie, a faculty member at the University of Chicago business school; later, Lorie and Merton Miller were essential in writing the report on the public impact of an options exchange that would become a major part of the CBOT’s proposal to the SEC in March 1971.

The CBOE and the Black-Scholes paper were greenlit within months of each other; two years later, the CBOE opened for trading, just a month before the Black-Scholes article would appear in print. On the first day of trading, nine hundred options were traded on sixteen underlying stocks. But volume grew at an astonishing rate: well over a million options were traded in 1973 alone, and by October 1974, the exchange began seeing single days in which as many as forty thousand options were traded, with regular volume above thirty thousand. Within a decade, this number would reach half a million. And competition from other exchanges popped up quickly: first the American Stock Exchange announced it would begin trading options, followed quickly by the Philadelphia and Pacific stock exchanges. In January 1977, the European Options Exchange was established in Amsterdam, modeled on the CBOE. Options trading was suddenly a big business, and, at least at first, investors were anxious to learn as much as they could about the new instruments. Black, Scholes, and Merton quickly became household names, at least in finance.

The second fortuitous policy change, as far as Black’s career was concerned, occurred almost simultaneously with the creation of the CBOE, though its impact on Black was slower. Once again, Chicago’s influential economists, and especially the famous monetarist Milton Friedman, were behind the initiative. In 1968, when Nixon was elected president, Friedman wrote him a letter urging him to abandon the so-called Bretton Woods system. Bretton Woods, named for the town in New Hampshire where the system was devised in July 1944, was the international monetary agreement put in place at the end of World War II. The Bretton Woods conference led to the creation of the International Monetary Fund (IMF) and the International Bank for Reconstruction and Development (now part of the World Bank). More important for our story was the fact that under the Bretton Woods system, major world currencies were valued at fixed exchange rates, based on the value of the U.S. dollar (and ultimately on gold, because the dollar was freely exchangeable for gold, at least for foreign governments). Changes in these exchange rates were infrequent, involving a long diplomatic process.

By 1968, however, when Friedman wrote to Nixon, the Bretton Woods system was beginning to show cracks. The main problem was that there simply wasn’t enough gold in the world to back the explosion in postwar international trade. While the United States held most of the world’s gold supply, gold continued to be traded on the open market, where its price could fluctuate. As long as the United States and its allies could keep open-market gold prices in line with the Bretton Woods price, there was no problem. But if the price of gold on the open market rose too high, as it naturally would with growing demand and a limited supply, there would be a risk of a run on the dollar (in the sense of a rush to convert dollars to gold), as foreign governments sought to settle their own debts by buying U.S. gold and selling it for a profit on the open market — in which case the system would simply collapse. Such a rush in fact occurred in late 1967, which was the impetus for Friedman to write his letter. But for a thinker like Friedman, the Bretton Woods system was ill conceived from the start: it was hopeless for governments to try to set exchange rates at all. Exchange rates, like anything else, should be determined freely in an open market.

Nixon didn’t listen to Friedman at first, but by 1971, with increased spending in Vietnam accelerating the accumulation of U.S. debt, he saw the writing on the wall. First West Germany and Japan pulled out of the Bretton Woods agreement and announced their currencies would no longer maintain parity with the dollar. Then, rather than wait for the world economy to collapse, Nixon administered the coup de grâce to the Bretton Woods system by ending the convertibility of U.S. dollars to gold. Over the next years, the fixed exchange rates gave way to floating rates, creating a system whereby the relative prices of currencies were determined on the open market.

Meanwhile in Chicago, Leo Melamed, the chairman of the Chicago Mercantile Exchange (CME), another futures exchange that had spun off from the CBOT in the early twentieth century, saw that global fiscal policy was in flux. Following a hint from Friedman, Melamed launched a new exchange of his own in May 1972, called the International Monetary Market (IMM), for trading futures contracts in foreign currency. As long as the Bretton Woods system was in place, trading currency futures wasn’t very interesting because currency values could change only through a laborious and public process. But once the exchange rate was allowed to float and be determined by open-market trading, futures markets became essential. Most important was that companies, and especially banks, could use currency futures to protect themselves against unexpected changes in currency values. Suppose that a company in the United States contracts with a company in the United Kingdom to send a shipment of cowboy boots in exchange for payment in pounds on delivery. The agreement is made at a particular time, but the payment won’t come in until the cowboy boots hit Britain. And in the meantime, the pound could change in value, so that the U.S. company’s profits (in dollars) would be lower than they were when the contract was made. To protect against such changes, the U.S. company could sell a futures contract for the amount it plans to receive when the shipment arrives, effectively eliminating the risk that the currency might change unexpectedly.

What does the IMM have to do with Black and Scholes’s options pricing formula? At first glance nothing — but within a few years, futures trading at the IMM had expanded to include new derivatives based on currencies, including options. Because currency risk is an important part of any international transaction, currency derivatives very rapidly became essential to the international economy. And once again, as at the CBOE, the Black-Scholes model became an integral part of everyday trading life. Even more importantly, Black and Scholes pointed a way forward for modeling other derivatives contracts, too, which rapidly grew at the IMM as businesses sought new ways of protecting themselves against currency risk. Between the IMM and the CBOE, Black and Scholes found a world that was perfectly poised to take advantage of their new ideas.

The options pricing formula that Black, Scholes, and Merton discovered was equivalent to the method that Thorp had worked out in 1965 for pricing warrants — though Thorp used a computer program to calculate options prices, rather than derive the explicit equation that bears Black’s, Scholes’s, and Merton’s names. But the underlying arguments were different. Thorp’s reasoning followed Bachelier’s: he argued that a fair price for an option should be the price at which the option could be interpreted as a fair bet. From here, Thorp worked out what the price of an option should be, assuming that stock prices satisfy the log-normal distribution Osborne described. Once he had a way of calculating the “true” price of an option, Thorp went on to work out the proportions of stocks and options necessary to execute the delta hedging strategy.

Black and Scholes, meanwhile, worked in the opposite direction. They started with a hedging strategy, by observing that it should always be possible to construct a risk-free portfolio from a combination of stocks and options. They then applied CAPM to say what the rate of return on this portfolio should be — that is, the risk-free rate — and worked backward to figure out how options prices would have to depend on stock prices in order to realize this risk-free return.

The distinction may seem inconsequential — after all, the two arguments are different paths to the same model of options prices. But in practice, it was crucial. The reason is that dynamic hedging, the basic idea behind the Black-Scholes approach, gave investment banks the tool they needed to manufacture options. Suppose you are a bank and you would like to start selling options to your clients. This amounts to selling your clients the right to buy or sell a given stock at a predetermined price. Ideally, you don’t want to make a risky bet yourself — your profits are going to come from the commissions you will earn on the sales, not on the proceeds of speculation. In effect this means that when a bank sells an option, it wants to find a way to counterbalance the possibility that the underlying stock will become valuable, without losing money if the option doesn’t become valuable. Black and Scholes’s dynamic hedging strategy gave banks a way to do exactly this: using the Black-Scholes approach, banks could sell options and buy other assets in such a way that (at least theoretically) they didn’t carry any risk. This turned options into a kind of product, something that banks could construct and sell.

Black stayed in Chicago until 1975, when MIT wooed him back to Cambridge. For a few years, academia seemed like the perfect fit for Black. He could work on whatever he liked, and at least in the early heyday of exchange-based options trading, it seemed he could do no wrong. He was an academic celebrity of the highest order, which brought both respect and freedom. His personal life, however, was a growing disaster: his (second) wife, Mimi, hated their Chicago life, which was an important part of the decision to move back to Cambridge, nearer to her family. But the move east didn’t help much. Increasingly alienated at home, Black devoted more and more time to his work, branching off now in new directions. He began to work on generalizing CAPM to try to explain cycles in the economy: why, in a rational world, would there be periods of growth, followed by periods of contraction? This led him to a new theory of macroeconomics, which he called “general equilibrium.” He also launched a crusade against the accounting industry, which he considered backward and unhelpful to investors.

But these other strands of his work were terribly received. It was as though Black had used up all his luck and timing with the options paper, and the string of other papers on derivatives and financial markets that followed it. His work on macroeconomics in particular was out of step with the times. Economists in the 1970s and 1980s were deeply engaged in an ongoing debate about economic regulation and monetary policy. On one side were the Chicagoans; on the other, the Keynesians, who favored government intervention throughout the economy. General equilibrium was a third way, thrust into a bipolar community. Black found himself attacked, and then ignored, by both sides. No one would publish his papers. His colleagues began to write him off as irrelevant. In less than a decade, he went from outsider to idol, and then back to outsider. By the early 1980s, Black was fed up with academia. He wanted out.

In December 1983, Robert Merton, Black’s old collaborator from the Black-Scholes days, was doing consulting work for the investment bank Goldman Sachs. Merton was doing at Goldman what Black and Scholes had been doing at Wells Fargo back in 1970: bringing in the new ideas from academia and trying to implement them in a practical setting. In this capacity, he argued to Robert Rubin, then head of the Equities Division, that Goldman Sachs should hire a theorist, an academic of its own, at a high enough level of the company that the new ideas would have a chance to seep through the culture. Rubin was convinced, and so Merton went back to MIT, brainstorming who among their current crop of graduate students he would recommend for this important position. Merton asked Black for his advice and received a surprising answer: Black wanted the job himself. Three months later, Black left academia for a new job at Goldman Sachs, to organize a Quantitative Strategies Group in the Equities Division. Thus he became one of the first quants, a new kind of investment bank employee with an intensely quantitative and scientific focus, as interested in intellectual innovation as in making a big trade. Wall Street would never be the same.

On October 4, 1957, the Soviet Union launched Sputnik, the first man-made object to enter Earth’s orbit. America panicked. Eisenhower immediately ordered the fledgling U.S. space program to schedule its own launch. The date was set for December 6. The event was televised live across the nation, as American scientists attempted to prove they were equal to the Soviets. Millions tuned in as the first American spaceship ignited on the launch pad, and then inched off the ground — for about four feet, before falling back to the tarmac and exploding. The performance was a humiliation for the American scientific establishment. Four years later, the Soviets did the Americans one better still, by propelling Yuri Gagarin into orbit and successfully launching the first manned spacecraft. Kennedy responded within the week by asking NASA to find a new challenge that the Americans could win. On May 25, 1961, Kennedy announced his commitment to put the first man on the moon.

Physics had been on the rise in the United States since World War II. But after Sputnik was launched, physics interest skyrocketed. About five hundred physics PhDs were awarded in 1958. By 1965, that number was closer to a thousand, and by 1969 it was over fifteen hundred. This rapid growth was in part a matter of nationalism: becoming a rocket scientist was a fine way to serve your country. But even more, it was a matter of funding. NASA’s annual budget increased by a factor of seventy from 1958 to its peak in the mid-sixties. In 1966, NASA was given almost $6 billion — 4.5% of the total federal budget — to devote to basic science. Other government funding agencies, like the Department of Energy and the National Science Foundation, were also flush (though none could compete with NASA). Even mediocre graduates of mid-tier doctoral programs were guaranteed jobs in science, as either professors or government researchers. Physicists were in high demand.

On July 20, 1969, Neil Armstrong and Buzz Aldrin became the first men to set foot on the surface of the moon. America and its allies rejoiced — finally, an American victory in the space race. And almost immediately, the physics job market collapsed. As the space race accelerated, so too did America’s commitment to the war in Vietnam. The success of the Apollo 11 mission gave Nixon an excuse to divert funds from NASA and other research groups to the military effort. By 1971, NASA’s budget was less than half of what it had been in 1966 (in real terms). Meanwhile, college enrollment began to drop, largely because the Baby Boom years were over. Once the “Boomers” had graduated, universities stopped hiring new faculty members.

Emanuel Derman was a South African physicist who experienced this funding roller coaster firsthand. He entered graduate school, at Columbia University, in 1966, at the high point of U.S. science funding. He worked on experimental particle physics — a field far from NASA’s central interests, but a beneficiary of the uptick in government support for physics nonetheless. Like most graduate students, he slogged through, living on a small stipend and working long hours. The students he knew when he first arrived in graduate school went on to positions at universities around the country. But by the time Derman graduated, in 1973, there were no permanent jobs left. Derman, and other physicists who had done excellent work, were barely able to scrape together a series of temporary research positions. Derman spent two years at the University of Pennsylvania, followed by two years at Oxford, and then two years at Rockefeller University, in New York. By the end of the decade, he was ready to give up. He considered quitting physics for medical school but decided to go to Bell Labs and work as a programmer instead.

As the seventies droned on, the number of physics PhDs awarded in the United States dropped, to about one thousand per year. While this was significantly lower than the peak in 1968, it was still far more than the flailing job market could support. This meant that by the time Black moved to Goldman Sachs, in 1983, there were thousands of very talented men and women with graduate degrees in physics and related fields who were either unemployed or underemployed.

Black’s move to Goldman Sachs coincided with another change, too. By 1983, options were a growing business, making people with training like Black’s attractive on Wall Street. But bond trading — already a mainstay of the financial industry — was in the midst of a sea change. Beginning with the Carter administration in the late 1970s, the U.S. economy entered a period of high inflation and low growth that has subsequently been dubbed “stagflation.” In response, Paul Volcker, the chairman of the Federal Reserve from 1979 through 1987, increased interest rates dramatically, so that the prime interest rate, the rate that determines how expensive it is for banks to lend to one another — and, by extension, to lend to consumers — reached an unprecedented level of 21.5%. Volcker was successful at reducing inflation, which he had under control by 1983. But this volatility in interest rates forever changed the previously sleepy bond industry. If banks couldn’t borrow from one another for less than 20%, surely corporations and governments that were trying to issue bonds would need to pay even higher rates (since typically bonds are more risky than interbank loans). The so-called bond bores of the 1970s, traders who had chosen to work in the least exciting of the financial markets, now needed to cope with the most variable market of all. (Sherman McCoy, the star-crossed antihero of Tom Wolfe’s novel Bonfire of the Vanities, was an eighties-era bond trader who took himself to be so important, given the changes in the bond markets during the late seventies and early eighties, that he privately called himself a “Master of the Universe.” The name has stuck, now used to refer to Wall Street traders of all stripes.)

The success of the Black-Scholes model and other derivatives models during the 1970s inspired some economists to ask whether bonds could be modeled in a similar way to options. Soon, Black and others had realized that bonds themselves could be thought of as simple derivatives, with interest rates as the underlying asset. They began to develop modified versions of the Black-Scholes model to price bonds, based on the hypothesis that interest rates undergo a random walk.

Thus, Black arrived on Wall Street at a moment when derivatives, and derivative models, were proving increasingly important, in unexpected ways. Black’s Quantitative Strategies Group at Goldman Sachs, as well as similar groups at other major banks, provided an answer to questions that many investment bankers, and especially bond traders, hadn’t known how to ask. At the same time, there was a large pool of underemployed physicists who were ready to step in and follow Black’s lead in changing financial practice. Once a few physicists and half-physicists had made their way to Wall Street, and once the usefulness of the ideas that Black had managed to translate from theory to practice was appreciated, the floodgates opened. Wall Street began hiring physicists by the hundreds.

Derman stayed at Bell Labs for five years. Starting in 1983, though, he began to get phone calls from headhunters sent from investment banks. He was unhappy enough at Bell Labs that he took these offers seriously, but when he finally received an offer from Goldman Sachs, he declined it on the advice of an acquaintance who had worked there before. But the world was changing. Derman found the next year at Bell Labs intolerable, and so when Wall Street came calling again, in 1985, he was ready to move. He decided to go with Goldman Sachs after all, and in December 1985 he made the leap. His job was in the Financial Services Group, which supported Goldman’s bond traders. By the time he arrived, Black was already an institutional legend.

Both Thorp and Black based their options models on Osborne’s random walk hypothesis, which amounted to assuming that rates of return are normally distributed. This might give you pause. After all, Mandelbrot argued throughout the 1960s that normal and log-normal distributions do not effectively account for extreme events, that markets are wildly random. Even if Mandelbrot’s claim that rates of return are Lévy-stable distributed and thus do not have well-defined volatility is false — and most economists now believe it is — the weaker claim that market data exhibit fat tails still holds. Options models assign prices based on the probability that a stock will rise above (or drop below) a certain threshold — namely, the strike price for the option. If extreme market changes are more likely than Osborne’s model predicts, neither Thorp’s model nor the Black-Scholes model will get options prices right. In particular, they should undervalue options that would be exercised only if the market makes a dramatic move, so-called far-out-of-the-money options. A more realistic options model, meanwhile, should account for fat tails.

Mandelbrot left finance at the end of the 1960s, but he returned in the early 1990s. One of the reasons was that many financial practitioners were beginning to recognize the shortcomings of the Black-Scholes model. Instrumental in this shift was the Black Monday stock market crash of 1987, during which world financial markets fell more than 20% literally overnight. Blame for the crash fell to a novel financial product based on options and the Black-Scholes model, known as portfolio insurance. Portfolio insurance was designed, and advertised, to curtail the risk of major losses. It was a kind of hedge that amounted to buying stocks and short selling stock market futures, the idea being that if stocks began to fall, the market futures would also fall, and so your short position would increase to offset your losses. The strategy was designed so that you wouldn’t sell too many futures short, because that would eat into your profits if the market went up. Instead, you would program a computer to gradually sell your stocks if the market fell, and you would short just enough market futures to cover those losses.

When the market crashed in 1987, though, everyone with portfolio insurance tried to sell their stocks at the same time. The trouble with this was that there were no buyers — everyone was selling! This meant that the computers trying to execute the trades ended up selling at much lower prices than the people who had designed the portfolio insurance had expected, and the carefully calculated short positions in market futures did little to protect investors. (In fact, investors holding portfolio insurance tended to do better than those who didn’t hold it; however, many people think the automated sell orders associated with portfolio insurance exacerbated the sell-off, and so everyone suffered because portfolio insurance was so prevalent.) The Black-Scholes-based calculations underlying portfolio insurance didn’t anticipate the possibility of a crash, because the random walk model indicates that a major one-day drop like this wouldn’t happen in a million years.

Several things happened in light of the crash. For one, many practitioners began to question the statistical predictions of the random walk model. This makes perfect sense — if your model says something is impossible, or virtually impossible, and then it happens, you need to start asking questions. But something else happened, too. Markets themselves seemed to change in the wake of the crash. Whereas in the years leading up to the crash the Black-Scholes model seemed to get options prices exactly right, in virtually all contexts and all markets, after the crash certain discrepancies began to appear. These discrepancies are often called the volatility smile because of their distinctive shape in certain graphs. The smile appeared suddenly and presented a major mystery for financial engineers in the early 1990s, when its prevalence was first recognized. Notably, Emanuel Derman came up with a way of modifying the Black-Scholes model to account for the volatility smile, though he never came up with a principled reason why the Black-Scholes model had stopped working.

Mandelbrot’s work, however, offers a compelling explanation for the volatility smile. One way of interpreting the smile is as an indication that the market believes large shifts in prices are more likely than the Black-Scholes model assumes. This is just what Mandelbrot had been claiming all along: that probability distributions describing market returns have fat tails, which means that extreme events are more likely than one would predict based on a normal distribution. In other words, market forces seemed to have brought prices into line with Mandelbrot’s theory. From the late 1980s on, Mandelbrot’s work has been taken much more seriously by investment bankers.

There’s an interesting, and rarely told, twist to the story of the rise and fall of Black-Scholes. The first major company to develop a quantitative strategy based on derivatives was a highly secretive Chicago firm called O’Connor and Associates. O’Connor was founded in 1977 by a pair of brothers named Ed and Bill O’Connor, who had made their fortune on grain futures, and Michael Greenbaum, a risk manager who had worked for them at First Options, an options clearinghouse the brothers ran. Greenbaum had majored in mathematics at Rensselaer Polytechnic Institute before joining First Options, and so he had some background with equations. He was one of the first people to realize that the new options exchange in Chicago offered a chance to make a killing, at least if you were mathematically sophisticated. He approached the O’Connor brothers with the idea of a new firm that would focus on options trading.

This much of the story is well known. But given the timing, many people assume that O’Connor was simply an early adopter of the Black-Scholes model. Not so. Greenbaum realized from the start that the assumptions underlying Black-Scholes weren’t perfect, and that it was failing to properly account for extreme events. And so Greenbaum built a team of risk managers and mathematicians to figure out how to improve on the Black-Scholes model. One of O’Connor’s first employees was an eighteen-year-old whiz kid named Clay Struve, who had worked for Greenbaum at First Options in a summer job, and who worked for Fischer Black as an undergraduate at MIT during the school year. During 1977 and 1978, Greenbaum, Struve, and a small team of proto-quants worked out a modified Black-Scholes model that took into account things like sudden jumps in prices, which can lead to fat tails.

O’Connor was famously successful, first in options and then in other derivatives — in part because the modified Black-Scholes model tended to outperform the standard one. Remarkably, according to Struve, O’Connor was aware of the volatility smile from very early on. That is, even before the crash of 1987, there were small, potentially exploitable discrepancies between the Black-Scholes model and market prices. Later, when the 1987 crash did occur, O’Connor survived.

There’s another, deeper concern about the market revolution initiated by Black and his followers that many people worried about in 1987 and that has become quite stark in the wake of the most recent crisis. Take the 2008 crash as an example. During the financial meltdown, even sophisticated investors, such as the banks that produced securitized loans in the first place, appear to have been mistaken about how risky these products were. In other words, the models that were supposed to make these products risk-free for their manufacturers failed, utterly. Models have failed in other market disasters as well — perhaps most notably when Long-Term Capital Management (LTCM), a small private investment firm whose strategy team included Myron Scholes among others, imploded. LTCM had a successful run from its founding in 1994 until the early summer of 1998, when Russia defaulted on its state debts. Then, in just under four months, LTCM lost $4.6 billion. By September, its assets had disappeared. The firm was heavily invested in derivatives markets, with obligations to every major bank in the world, totaling about $1 trillion. Yet at the close of trading on September 22, its market positions were worth about $500 million — a tiny fraction of what they had been worth a few months before, and far too little to cover the company’s loans. A feather’s weight would have led to a default on hundreds of billions of dollars of debt, leading to an immediate international panic, had the government not stepped in to resolve the crisis.

The mathematical models underlying dynamic hedging strategies specifically, and derivatives trading more generally, are not perfect. Bachelier’s, Osborne’s, and Mandelbrot’s stories go a long way toward making clear just why this is. Their models, and the models that have come since, are based on rigorous reasoning that, in a very real sense, cannot be wrong. But even the best mathematical models can be misapplied, often in subtle and difficult-to-detect ways. In order to make complicated financial markets tractable, Bachelier, Osborne, Thorp, Black, and even Mandelbrot introduced idealizations and often strong assumptions about how markets work. As Osborne in particular emphasized, the models that resulted were only as good as the assumptions that went in. Sometimes assumptions that are usually excellent quickly become lousy as market conditions change.

For this reason, the O’Connor story has an important moral. Many histories suggest that the 1987 crash rocked the financial world because it was so entirely unexpected — impossible to anticipate, in fact, given the prevailing market models. The sudden appearance of the volatility smile is taken as evidence that models can work for a while and then suddenly stop working, which in turn is supposed to undermine the reliability of the whole market-modeling enterprise. If models that work today can break tomorrow, with no warning and no explanation, why should anyone ever trust physicists on Wall Street? But this just isn’t right. By carefully thinking through the simplest model and complicating it as appropriate — in essence, by accounting for fat tails — O’Connor was able to anticipate the conditions under which Black-Scholes would break down, and to adopt a strategy that allowed the firm to weather an event like the 1987 crash.

The story that I have told so far, from Bachelier to Black, goes a long way toward showing that financial modeling is an evolving process, one that proceeds in iterative fashion as mathematicians, statisticians, economists, and quite often physicists attempt to figure out the shortcomings of the best models and identify ways of improving them. In this, financial modeling is much like mathematical modeling in engineering and science more generally. Models fail. Sometimes we can anticipate when they will fail, as Greenbaum and Struve did; in other cases, we figure out what went wrong only as we are trying to put the pieces back together. This simple fact should urge caution as we develop and implement new modeling techniques, and as we continue to apply older ones. Still, if we have learned anything in the last three hundred years, it’s that the basic methodological principles of scientific progress are the best ones we’ve got — and it would be foolish to abandon them just because they aren’t always perfect.

What’s more, since mathematical modeling in finance is an evolving process, we should fully expect that new methods can be developed that will begin to solve the problems that have plagued the models that have gotten us to where we are today. One part of this process has involved modifying the ideas that Black and Scholes introduced to financial practice to better accommodate Mandelbrot’s observations about extreme events. But that’s only the beginning. The final part of the book will show how models have continued to evolve outside of mainstream finance, as physicists have imported newer and more sophisticated ideas to finance and economics, identifying the problems with our current models and figuring out how to improve them. Black was instrumental in producing a new status quo on Wall Street, but his ideas were just the beginning of the era of financial innovation.