A BRIEF HISTORY OF AI
AI has captured the public’s imagination, but this is not the first time; we are actually in a third wave of AI progress and interest. It first happened in the early 1960s and again in the early 1980s. In both cases, expectations about progress toward artificial general intelligence (AGI) systems—with human-like intelligence—far exceeded reality. Both periods were followed by AI winters, in which investment and interest in AI plummeted. Will this time be different?
Figure 3.1 Timeline of AI hype cycles.
During the first two AI hype cycles, AI systems were developed to further the goal of AGI. Narrow AI did not exist. Because they failed to achieve AGI in the first two hype cycles, researchers created and focused on building narrow AI systems during the third cycle.
THE FIRST AI HYPE CYCLE
The first AI system was built in 1951 by Marvin Minsky and Dean Edmonds at Harvard University. It was built with vacuum tubes and simulated a rat learning to search a maze for food. John McCarthy coined the term artificial intelligence in 1956. The first AI conference was a two-month workshop held at Dartmouth College that same year. The goal was to bring together researchers in automata theory, neural networks, and the study of intelligence. The participants included several individuals who would go on to become leaders in the field of AI, including Minsky, McCarthy, Allen Newell, and Herbert Simon.
EARLY SUCCESSES
Early work in AI showed tremendous promise. In 1956, at Rand Corporation, Allen Newell and Herbert Simon created the Logic Theorist, which was able to prove many mathematical theorems.1
In 1960, when he was at MIT Lincoln Labs, Bert Green (who became my PhD thesis advisor eighteen years later) wrote BASEBALL, a computer program that could answer questions about the 1959 baseball season, like these:
•Who did the Red Sox lose to on July 5?
•Where did each team play on July 7?
•Which teams won ten games in July?
•Did every team play at least once in each park?2
For his PhD thesis at MIT in 1964, Daniel Bobrow3 developed the STUDENT system, which accepted a restricted subset of English in which it was possible to express a wide variety of algebra word problems such as this one:
If the number of customers Tom gets is twice the square of 20 percent of the number of advertisements he runs, and the number of advertisements he runs is forty-five, what is the number of customers Tom gets?
STUDENT transformed the input question into a set of simultaneous equations that it then solved. The success of STUDENT, along with other early successes, led Herbert Simon to proclaim in 1965 that “machines will be capable, within twenty years, of doing any work that a man can do.”4
EARLY FAILURES
During this early achievement period, there were also some significant failures, the most notable of which was machine translation— computer systems that translate text from one human language to another. Researchers first attempted machine translation in the 1950s by programming a computer to act like a human translator who had a pocket translation dictionary but did not know the target language. The technique was to first store the equivalent of a pocket dictionary in the computer. Then the program processed the source text word by word, placing the corresponding word in the target language into the output text. Some of the more sophisticated systems also used syntactic analyses to correct for differences in word order in the two languages. For example, the phrase the blue house in English might translate to the equivalent of “the house blue” in a language such as French: la maison bleue.
These early machine translation attempts produced disappointing results. The major problem was the mistaken assumption of a one-to-one correspondence between words in the source and target languages. For example, because of word ambiguity, a substantial proportion of the words in the target language had too many potential equivalents in the source language to come up with an accurate translation.
When forced to choose among the possible target language alternatives for each source language word, these systems often produced humorous results. Several realistic, even if not actual, anecdotes have become part of the lore of AI. For example, one system reportedly translated the English phrase water buffalo into the equivalent of “hydraulic goat” in the target language. Idiomatic and metaphorical material also caused problems. One system reportedly translated the English the spirit is willing, but the flesh is weak into the equivalent of “The vodka is good, but the meat is rotten” in Russian. Another system reportedly translated out of sight, out of mind into “invisible idiot.”
In a 1960 review of machine translation efforts, an Israeli linguist named Yehoshua Bar-Hillel5 asserted that human translators could not only understand the meaning of the source language but could also reason based on their knowledge of the world—abilities that computers had yet to demonstrate. For computers to be able to translate language, he argued, they would have to do the same thing in the same way.
Bar-Hillel’s point of view was reiterated six years later in the infamous ALPAC (Automatic Language Processing Advisory Committee) report, which convinced the US government to reduce its funding for this type of research dramatically.6 The ALPAC report concluded that machine translation was slower, less accurate, and twice as expensive as human translation and stated that there was no point in further investment in machine translation research. The US government canceled nearly all of its funding for academic machine translation projects that year. Not long after, Sir James Lighthill’s 1973 report to the UK Parliament criticized the failure of AI to achieve “grandiose” objectives.7 As a result, the UK government canceled its funding for most AI research in the UK.
Neural networks like the ones that power most of today’s high-profile AI applications were described in 1958 by Frank Rosenblatt at Cornell University.8 His pioneering paper explained how a neural network could compute a function relating inputs to outputs. The approach received a lot of press, including this quote from a 1958 New York Times article: “The Navy revealed the embryo of an electronic computer today that it expects will be able to walk, talk, see, write, reproduce itself, and be conscious of its existence.”9
Research on neural networks continued through the 1960s but was stopped in its tracks by the 1969 book Perceptrons, written by Marvin Minsky and Seymour Papert, which appeared to prove that further progress in neural networks was impossible.10 We now know that Minsky and Papert’s criticism, although it was accurate for the type of perceptron used by Rosenblatt, does not apply to neural networks in general. Nonetheless, the book was responsible for stopping most neural network research at the time.
For the next decade, research funding diminished, but some continued at universities. In the late 1970s and early to middle 1980s, excitement about AGI once again built to a frenzied peak, primarily as a result of research into natural language processing and expert systems.
Rather than giving up, these later AI researchers took Bar-Hillel’s critique as a head-on challenge. They started to think about how to represent the meaning of a natural language utterance and, more generally, how to represent commonsense knowledge in a computer system in a way that can support human-like reasoning.
For example, my former colleagues at Yale University, led by Roger Schank, developed a theory of how to use hand-coded symbols to represent meaning in a computer system.11 Schank and his students developed several impressive AI systems, such as FRUMP12 and SAM.13 FRUMP could read stories about train wrecks, earthquakes, and terrorist attacks directly from the UPI newswire and could then produce summaries of those stories. SAM read short stories such as “John went to a restaurant. He ordered lobster. He paid the check and left.” The computer could then answer questions such as “What did John eat?” with “Lobster,” even though eating was never mentioned in the story.
These systems, along with other natural language processing systems developed at research institutions throughout the world, left a strong impression that the development of computers that could understand language at an AGI level was right around the corner.
Other researchers focused on building what came to be known as expert systems. They built systems that performed tasks that previously required human intelligence by processing large bodies of hand-coded symbolic rules. For example, an auto-repair expert system might have this rule: IF the temperature gauge indicates overheating and the radiator is cool to the touch, THEN the problem is a sticky thermostat.
Researchers such as Stanford University’s Edward Feigenbaum achieved significant recognition for early expert systems, such as the MYCIN system,14 which diagnosed blood infections. With 450 rules, MYCIN performed as well as some experts.
Governments poured vast amounts of money into AI during the 1980s. In 1982, Japan announced its Fifth Generation project, intended to create intelligent computers within ten years. In response, DARPA (the US Defense Advanced Research Projects Agency) committed $1 billion in 1983 to the Strategic Computing Initiative, an effort to build AGI in ten years.15 At the same time, a group of twenty-four large US corporations banded together to start MCC (Microelectronics and Computer Technology Corporation). Its flagship project, headed by Douglas Lenat, was to hand code all commonsense knowledge in a system named Cyc. Commonsense knowledge includes hand-coded symbolic facts, such as “birds have wings,” “airplanes have wings,” and “if it has wings, it can fly.” Unless, of course, it is a chicken. The UK had its own effort named Alvey, and Europe formed ESPRIT (European Strategic Program on Research in Information Technology).
Our company, Cognitive Systems, became a hot property. At one point, I recall deciding that I had to stop doing newspaper and magazine interviews to get any work done. Cognitive had an IPO in 1986, despite not having much revenue. A sister company to Cognitive Systems named Nestor that was commercializing neural network technology also had an IPO in 1983, and Feigenbaum’s expert systems company, Teknowledge, had one in 1984.
AI was so hot in the 1980s that researchers created two companies with significant venture capital backing, Symbolics and LMI, to build computers specially designed to run the Lisp programming language, which was the preferred programming language of the AI community.
In the mid-1980s, AI was featured on the covers of many publications. For example, the March 1982 cover of Business Week proclaimed, “Artificial Intelligence: The Second Computer Age Begins.”
Company after company rebranded their products as “AI-based.” It seemed that AI researchers had figured out how to make AGI systems and to represent human knowledge symbolically. It appeared that the only thing standing in the way of AGI was a bit of elbow grease to encode all human knowledge and the types of reasoning needed for intelligent behavior.
I admit to having been a significant contributor to the hype of the time. Our Cognitive Systems brochure headline was grandiose: “The Vision to Change the World.” We were able to make a successful public offering in February 1986, without significant revenues, on the strength of the general hype around AI.
My next company, Intelligent Business Systems, raised over $50 million in venture capital ($120 million in 2019 dollars). Raising venture capital was as difficult then as it is now. But we eventually put together a consortium of VCs and scheduled the closing for noon on a Friday at a country club. The idea was to take care of the final contractual issues in the morning and play golf in the afternoon while the lawyers were preparing the final drafts. However, we did not resolve those final issues until midnight, and we did not get to play golf. I was so tired that I ran into the back of my cofounder’s car at the first stoplight. He was 6’5” and had played tight end in the NFL. The policeman who arrived on the scene commented that we must be friends because otherwise I would be in worse shape than my car.
We developed and sold a natural language–based accounting system. Users could ask questions such as “Who owes me money for over ninety days?” However, the company ended up failing when the second AI winter hit, and we could not raise a follow-on round of funding. The good news was that the skills of my employees were in high demand, and no one stayed unemployed for long.
My third company, Esperant Technologies, developed a natural language system that became one of the leading business intelligence products of the early 1990s. However, because AI had fallen into disfavor again, even though the product was successful, I had to strike all references to AI from the product and marketing literature.
AI failed for the second time because the symbolic hand coding of knowledge was time-consuming, expensive, and did not perform well in the real world. Expert systems could perform only very narrowly defined tasks, and natural language interfaces had limited coverage that required users to learn what they could and could not say. Symbolic AI came to be known as good, old-fashioned AI, fell into disfavor by 1990, and was out of favor for thirty years. But it might make a comeback in the 2020s.
The AI bubble of the 1980s deflated gradually, and by the end of the 1980s, the second AI winter had begun. The Japanese government disbanded its Fifth Generation project without it having produced any significant AI technology. DARPA significantly cut back on its AI funding. MCC wound down in 2000. The market for Lisp machines imploded. Professor Stuart Russell said that his AI course at the University of California at Berkeley had 900 students in the mid-1980s and had shrunk to only twenty-five students in 1990.16
THE THIRD AI HYPE CYCLE
During the second AI winter, DARPA continued to support AI research in universities. However, there was relatively little public awareness of AI, few products branded themselves as AI, and there was certainly no fear of it.
AI started to heat up again in the late 1990s and early 2000s, when the US National Institute for Science and Technology (NIST) held a periodic competition for machine translation systems during the first decade of the new millennium. In 2005, Google participated for the first time and blew away the competition. Rather than using symbolic, hand-coded translation systems based on grammar and other linguistic features, Google used only automated processes. They employed large computer systems to process massive databases of news stories and other text to determine which words and phrases occurred most often before or after other words and phrases. “Just like that, they bypassed thirty years of work on machine translation,” said Ed Lazowska, the chairman of the computer science department at the University of Washington.17
Especially since 2010, companies, government agencies, nonprofits, and other organizations are once again pouring money into research and development. However, this time, it is China that is making the gigantic push into AI.18 In July 2017, China’s State Council issued the Next-Generation AI Development Plan to become the “premier global AI innovation center” by 2030, and that calls for China’s AI business to be worth $148 billion. Russian president Vladimir Putin said the country that takes the lead in AI would rule the world.19
Companies, once again, are rebranding products as having AI inside. A study of European start-ups that advertise their AI capabilities found that almost 45 percent do not even use AI.20 As The Atlantic said, AI is “often just a fancy name for a computer program.”21
Private companies have recruited many of the leading academic researchers in the AI field. Google, Facebook, Microsoft, Apple, Netflix, Uber, Amazon, Nvidia, Baidu, IBM, Tencent, Alibaba, Salesforce, and many others have AI teams that publish significant numbers of academic papers and blog posts.
Several computer industry luminaries, including Elon Musk, Sam Altman, Greg Brockman, Reid Hoffman, Jessica Livingston, and Peter Thiel, committed a billion dollars of funding to create a large nonprofit AI research lab named OpenAI in San Francisco. Paul Allen, the late cofounder of Microsoft, invested a great deal of money into the Allen Institute for AI in Seattle. Vicarious, also in San Francisco, has raised $150 million from the likes of Jeff Bezos, Elon Musk, Mark Zuckerberg, and Ashton Kutcher. OpenAI, the Allen Institute, Vicarious, and another company, DeepMind, which was acquired by Google, are all prolific contributors to academic AI research.
Jeff Dean, head of Google AI, noted that the number of AI papers published had followed Moore’s Law with the number of publications more than doubling every two years from about one thousand per year in 2009 to over twenty thousand per year in 2017.22 Tickets to the 2018 Conference on Neural Information Processing Systems reportedly sold out fifteen minutes after registration opened on the website. The 2019 conference had thirteen thousand registered participants and 6,743 papers submitted.23
In early 2020, it is fair to say that AI has become hot once again, that it is all over the press, and that interest in AI is reaching new peaks every year. Are we in another bubble? Or is this time for real?
This time, it is clear that AI is succeeding. Search engines answer a lot of our questions intelligently. We can go to a webpage in a language we do not know, click translate, and easily understand the page’s contents. AI can identify the people in our photos. And new and exciting AI applications are being released daily.
That said, all of the progress has been in narrow AI systems; there has been little or no progress in building AGI systems since the 1980s. Yet the popular press confuses today’s narrow AI with AGI regularly, as in a 2018 Newsweek article’s title: “Robots Can Now Read Better Than Humans, Putting Millions of Jobs at Risk.”24 The reality is that computers can perform one small reading comprehension task at a human level by using clever tricks rather than AGI. AI systems cannot read and understand at an AGI level. Moreover, researchers do not know how to make them do so.
NO THIRD WINTER IN SIGHT
There have been three periods of high exuberance around the prospects for AGI. Each of the first two cycles ended in extended periods of low interest in AGI. The current cycle has staying power because of all the successful narrow AI commercial applications. However, this enthusiasm for narrow AI has fallaciously spilled over into optimism about AGI.
During the first two cycles of AI, researchers focused on hand coding symbolic knowledge and reasoning rules. This proved to be brittle and required far too much manual effort. The current cycle of AI relies on automated learning, which overcomes the difficulty of hand coding symbolic rules. Yet current AI techniques can only produce narrow AI systems, not AGI systems. This third cycle of AI will continue to be successful. However, it will fall far short of achieving anything close to AGI.