The Engineering Book: From the Catapult to the Curiosity Rover: 250 Milestones in the History of Engineering

2011

Watson

David Ferrucci (b. 1962)

Every now and again, engineers and scientists pop something into the world and it is completely unexpected. For example, we went from “no airplanes” to “airplanes” the day the Wright Flyer lifted off the ground. We went from “self-driving cars are way off in the future” to “self-driving cars are here today” the day Google announced 100,000 accident-free road miles for its self-driving technology.

And the same is true for Watson—IBM’s Jeopardy-playing computer, developed by a team lead by principal investigator David Ferrucci. Out of the blue, Watson had trounced the best human players. It was quite surprising.

So how did IBM engineers pull this off? Part of it was a very large hardware problem, and part of it was innovative software. The key, and perhaps most surprising, decision the software engineers made was to work with untagged data. In other words, Watson would read in unmodified versions of things like Wikipedia, the Internet Movie Database, and the dictionary. Watson would employ things like natural language processing, machine learning, and semantic analysis to make sense of all of it. No one would have to structure or tag the data in any way. From that raw text information, the algorithms would sort through it all to form a knowledge base, and then combine it with a natural language processing front end to understand the Jeopardy questions. A synthetic voice then provides the answers.

And Watson is fast. The first system—the one that played on national TV in 2011—is a supercomputer filling a room. That’s because it used nearly 100 servers, containing nearly 3,000 computing cores and 16 terabytes of memory, to run the software.

Watson’s Jeopardy-playing ability is just the beginning. The same software techniques can grind through all kinds of text information—medical, legal, scientific, Internet—and perform the same kind of magic. For example, imagine a search engine that “understands” your question. Or a system where a doctor can ask any medical question and find all the related research results. Watson is a great example of engineering on the cutting edge.

Contestants Ken Jennings and Brad Rutter compete against Watson at a press conference on January 13, 2011 in Yorktown Heights, New York.