Invention establishes relationships that did not previously exist. In its barest essence, the element of innovation lies in the completion of a pattern or in the improvement of a pattern that was unsatisfactory and inadequate.
—A. P. USHER, A History of Mechanical Inventions
PATTERN RECOGNITION is the act of making sense of complexity. Patterns enable us to anticipate what will happen next.
The human brain is the world’s best pattern recognition machine. It identifies faces and voices. It spots familiar shapes among clouds and stars or in wallpaper designs and ceiling tiles. It can create new musical forms and artistic figures. It enjoys the patterns of jokes and dramas, especially the unexpected twists. It tries to interpret the logic of its own dreams or to see parallels between current events and historical ones. It can recognize winning patterns on the chessboard and in other games. With the help of computers, it can model trends in crime or the weather. It can diagnose diseases and try to find ways to prevent them. It can make money by recognizing a new investment or business trend.
Invention is commonly considered to be a product of an inexplicable leap of insight. When viewed more closely, though, great inspirations are often less mysterious than they might otherwise seem. New solutions often are found in the recognition of new patterns. Rather than being inhibited by prior knowledge, this thinking tool shows that past experience is often integral to the incubation of insights. We’re often told that breakthrough innovators are the ones operating outside the box. On the contrary, inventors are the ones who learn to recognize boxes or other patterns in the first place. Inventors are constantly searching the world for patterns that can be reapplied and extended.
As Abbot Payton Usher suggests in a classic 1929 book, the history of invention is one of finding new patterns and then improving or completing them.1 Building on that point is architect Christopher Alexander. In his 1977 book A Pattern Language, Alexander observes that “each pattern describes a problem which occurs over and over again in our environment.”2 But as with many important patterns that have been discovered, he notes, “no pattern is an isolated entity. Each pattern can exist only to the extent that it is supported by other patterns: the larger patterns in which it is embedded, and the smaller patterns which are embedded in it.” New patterns describe “the core of new solutions.”
In the world of biology, the double helix is a granddaddy of a pattern, solving a great mystery but also giving birth to thousands of new ones. “It was a scientist’s dream,” remarked one biologist, “simple, elegant, and universal for all organisms.”3 But it could very well have been found by someone other than James Watson and Francis Crick. The soon-to-be famous duo was in a race with other scientists. Like fifteenth-century mapmakers, they were using both theory and real-world observation to find a fundamental pattern, a new way of looking at the available knowledge. The already legendary chemist Linus Pauling was hot on the trail. Only a few months beforehand, he proposed a triple-helix model that seemed somehow faulty to Watson. Meanwhile, Rosalind Franklin was studying X-ray images of genetic material. Working across town from Watson and Crick’s Cavendish Laboratory in Cambridge, England, she was only one tweak of a mental model away. “She had the data to do it,” Watson noted years later. But as one of her future colleagues remarked, Franklin “needed to break the pattern of her thinking.”
The double helix was obvious to virtually no one. Over the ensuing years and decades, it would become knowable to virtually everyone. As is typically the case with great scientific discoveries, this one would lead to great technological inventions. The discovery of this pattern wasn’t an end, in other words, but rather was a new beginning. Entire generations of medical researchers and biotech inventors would spend their careers working with this elegant model fixed in their heads. But what could they do next? The double helix seemed perfect. How could you improve on it? The answer was to construct related patterns. Get inside the double helix. Unlock its secrets. Extend the pattern in new ways. Find new patterns that complement it. Use it as a tool of invention.
Leroy “Lee” Hood has spent his entire career doing just that. Hood is a physically fit man of medium build who makes forceful hand motions whenever he launches into a scientific explanation. The Renaissance boy of his one hundred forty–student high school in Shelby, Montana, Hood starred on the football team; played the piano, clarinet, and oboe; acted in school plays; sang in the choir; joined the debate team; and edited the yearbook. In the summer, he rode horses and hiked mountains. In the parlance of cognitive science, his brain was becoming “well connected.”
In his junior year, Hood submitted a geological mapping of a nearby oil incline to the national Westinghouse science fair, with the hope he’d become the first student from his state ever to be selected as a winner. He was indeed chosen. The long train ride to Washington, D.C., was the first time he had left Montana. As they said back then, Lee Hood was going places. But where?
One of Hood’s most searing childhood memories takes him back to age nine, when his parents were having an argument. The conflict was over his one-year-old brother, Glen, who had been born with Down syndrome. “My mother wanted to keep Glenny in the family,” Hood recalls. “My father felt it would be better for him and the family if he were placed in a state home. My father won the argument. It was very hard on my mother. At some point, I began thinking, ‘What went wrong with Glenny?’”4 Lee Hood didn’t know it yet, but he would end up inventing instruments for finding new patterns in the wide-open field of biology, and his family’s experience may have prompted him to choose a life of genomic rather than geological mapping.
His study of biology began with an opportunity. During his senior year, the chemistry teacher was called on to teach a new course in biology to sophomores. In need of some help, the teacher recruited Hood, clearly one of the school’s brightest students. Hood began reading up on the subject to help prepare the lessons. The year was 1956. “I remember reading an article in Scientific American about Watson and Crick’s discovery [three years earlier of the double-helix structure of the DNA molecule],” he says. “I thought, ‘Wow!’ I didn’t begin to understand the implications. But I thought, ‘Gee, this is incredible.’ I understood it in the sense that this had to be the material that contained the genes that represented the instructions of life. My understanding was very simple compared to now. But it made me get very excited. I realized that there was something very concrete at the foundation of biology.”
For Hood, that “Wow!” was a cognitive snap, a sudden realization. On one level, the now-famous Watson and Crick model—a twisty ladder with the base-pair rungs—was only a simple pattern. On another level, however, that pattern organized a mind-boggling array of data and concepts into a distinct, indelible image that would thereafter be imprinted on Hood’s mind. He was among the first generation of kids to grow up in a world redefined by the double-helix pattern.
Hood wanted to experience more moments of pattern recognition. There were thousands of secrets hidden within the pattern of the double helix, and he was energized by the prospect of finding them. As a student at California Institute of Technology, he picked up patterns of thinking from some of the world’s greatest scientific minds.
For freshman physics, he had Richard Feynman, only a few years before Feynman would win the Nobel Prize for his work on quantum electrodynamics. Feynman liked to play intellectual jokes on the students. “He made you leave class thinking you understood everything,” Hood recalls, “and then you went home and tried to do your homework, and you realized you didn’t.” The lesson: If you didn’t comprehend the underlying pattern that organized the information, bits of data that you thought you understood didn’t lead to follow-on insights.
Feynman himself told a story of how he had learned “what patterns were like and how interesting they are.”5 At the age of three, he recalled, his father brought home bathroom tiles and demonstrated that they could be set up “in a more complicated way: two white tiles and a blue tile, two white tiles and a blue tile, and so on.”
Hood also attended Linus Pauling’s chemistry lectures. The winner of the Nobel Prize for his discovery of the nature of complex substances such as proteins and antibodies, Pauling often dispensed advice on creativity. “I am constantly asked by students how I get good ideas,” Pauling said. “My answer is simple: First, have a lot of ideas. Then, throw away the bad ones.” Knowing how to discriminate between the two, of course, is the trick.
Hood recalls being even more strongly influenced by the thinking patterns of other teachers and mentors, including his biology professor, George Beadle, who had won the Nobel Prize for medicine for his 1941 discovery that genes do their magic by controlling precise chemical events. Beadle confirmed that genes exist to execute a recipe for creating proteins, the building blocks of all living matter. Not only that, but Beadle also proved that genes could be manipulated. By shooting X-rays at certain cells, for example, the recipe could be altered, rewritten, artificially mutated. But how did it all happen? What use was all this knowledge? How exactly did genes create endless varieties of life? The race was on to crack the genetic code, to find the patterns by which genes construct proteins.
In the middle of his college career, Hood got hooked on this mind-set of unlocking biological patterns. A young French geneticist named Jerome Lejeune discovered the pattern deviation that caused Down syndrome. Reading Lejeune’s 1959 paper, Hood learned that his brother Glen’s condition was triggered by something called a “21 trisomy”: an extra copy of chromosome number 21 on the double-helix ladder of 23 base pairs of chromosomes. Learning this gave Hood a deep appreciation for how, as he puts it, “such a little thing could have such a profound effect.” He wanted to know more about human anatomy as well as human diseases and treatments, a topic not covered deeply at Caltech. “I was even more convinced than ever that I wanted to go into biology, but for reasons I never fully understood—perhaps in part due to my brother Glen,” Hood later acknowledged. He enrolled in medical school at Johns Hopkins without any intention of becoming a doctor. “Most of the students wanted to become family doctors in small towns,” he recalls.
Instead, Hood would go on to combine his knowledge of biology with the process of invention, cocreating a set of key instruments that would enable the eventual mapping of the entire human genome, humanity’s ultimate pattern. He would play a large part in epochal breakthroughs, setting off an explosion of new patterns and pattern deviations that continue to reveal themselves to this day. His inventions would also sit at the core of the multibillion-dollar biotech industry, which didn’t exist when Hood was in college and medical school.
But before we explore more precisely what Hood actually had a hand in inventing and how he did it, let’s think back to simpler times, when breakthrough inventions involved things we could see more readily with the unaided eye. For a more mechanical model of how perceiving and applying patterns can work in practice, let’s turn to a classic case.
The basic idea of the phonograph had been around for forty years before Thomas Edison reframed the problem. In 1857, inventor Leon Scott, an American of French origin, realized that sound vibrations could be visualized. He fixed a bristle to a stretched piece of pigskin. When he spoke loudly into the diaphragm, the bristle vibrated and etched patterns onto a soot-coated piece of paper or glass. Scott called it a “phonautograph.” Several inventors improved on the device, showing that visual patterns representing sound could be preserved for later study. Alexander Graham Bell was among those who built his own phonautograph from scratch.
To Edison, the problem wasn’t in recording sound but rather in playing it back. Whereas Bell saw the phonautograph as a way of understanding sound, Edison looked for a new opportunity. Could one design a machine that spoke back sound? His mental model of the problem seemed to form in a unique way. Being largely deaf, he preferred to experience sound not through his ears but “by clenching his teeth around a metal plate attached to the sounding apparatus,” according to Edison biographer Neil Baldwin.6 “Vibrations were conveyed through his resonating jawbone—meaning, in effect, that he virtually heard through his teeth.” By feeling the sound patterns reverberate through his jaw in this way, Edison hit on the question of whether one could record and play back the sound using the same machine.
For a solution, Edison turned to a pattern, a set of shapes and mechanical motions, that he was familiar with. “Any creative technologist,” notes Reese Jenkins, another Edison biographer, “possesses a mental set of stock solutions from which he draws in addressing problems.”7 In Edison’s case, one of his favorite stock solutions was the steel cylinder and the stylus. He had seen these simple but effective devices everywhere in his travels, and a pattern for creating new mechanical devices from these elements had already formed in his mind.
As a child, Edison sold newspapers on the Grand Trunk Railroad. By the age of fifteen, he was publishing his own small newspaper, the Weekly Herald, which contained gossip, advertisements for businesses along the rail line, and news about the railroad. Edison printed the newspaper in the baggage car of a train using a galley proof press. The machine was powered by a revolving, hand-cranked metal cylinder that pressed the sheets against a flat bed of movable type. The newspaper ceased publication after a chemical fire burned part of the baggage car, an accident that may have contributed to the loss of much of Edison’s hearing when an angry conductor whacked him in the head afterward. (Other accounts blame a bout with scarlet fever, as in the case of Mabel Hubbard, Bell’s future wife.)
Edison next became a roving telegraph operator, drifting from city to city in search of more interesting situations and higher pay. Most of the telegraph offices contained mechanical devices, such as relays and type-wheels, which were driven by rotating steel cylinders that transmitted electricity through a metal stylus. Was he beginning to recognize this pattern back then? We don’t know exactly what he was thinking, but we do know that he soon made his first invention, a telegraph message playback device designed after this kind of rotating cylinder. That same year (1867), the twenty-year-old Edison found work in a telegraph machine shop operating a foot-powered lathe, which rotated materials as he cut them into cylindrical shapes using a styluslike tool.
Ten years later, when Edison first achieved fame for inventing the “talking machine” at Menlo Park, reporters wanted to know how the inventor came to his sudden flash of insight. The first sketch of Edison’s phonograph, made in November 1877, shows a horizontal cylinder having grooves in its surface for encoding sound information. The drawing depicts a movable stylus that has an embossing point for making the grooves, which would be read for playback by another stylus. The pattern of oscillations was relayed electrically to a speaker. The sounds were to be preserved on paraffin and later on wax disks that wrapped around the steel cylinder.
Edison’s conception seemed directly patterned after his earlier telegraph message recorder and on the machines he studied in his early travels. It employed the same shapes and motions and was aimed at the same sort of mechanical function. It may have been a new invention, in other words, but it was based on a recurring pattern.
Edison first envisioned his phonograph as being used primarily to record and play back messages from the telephone, which had been demonstrated and disclosed in a patent by his rival, Bell, only a year earlier. Edison didn’t yet know of his lucky strike. He didn’t see the applications that would give rise to the musical recording industry a quarter century later. Edison, who had previously come close to inventing the telephone, now inspired envy in Bell, who marveled at the talking machine. “It’s a most astonishing thing to me,” remarked Bell, “that I could possibly have let this invention slip through my fingers when I consider how my thoughts have been directed to this subject for so many years past.”8
But it seems that Edison was able to arrive at the phonograph before Bell and all his other competitors because of the way Edison pinpointed the basic problem and then completed a pattern that no one else had seen in the same way. He recognized the versatility of a rotating, electromechanical cylinder. He kept seeing that shape repeatedly, drawing on the pattern in his mind. The printing press was based on a cylinder, as were many early telegraph relay devices and the machine shop lathe. Even the electromechanical printing wheels that Edison developed were cylindrical. The phonograph was considered Edison’s breakthrough invention, but it really was a continuation of a pattern that he had recognized years earlier.
Ten years after he achieved fame for recording and playing back sound, Edison again invoked his favorite shape. By then, he had relocated his laboratory to West Orange, New Jersey. “I am experimenting upon an instrument which does for the eye what the phonograph does for the ear,” Edison wrote in 1888, “which is the recording and reproduction of things in motion. The invention consists in photographing continuously a series of pictures occurring at intervals . . . at greater than eight per second . . . on a cylinder or plate in the same manner in which sound is recorded on the phonograph.”9
This invention, which Edison called the kinetoscope, was housed in a cabinet that had a viewing peephole fitted with a microscope lens. His first sketches depicted a large cylinder for advancing the film. This shape was soon abandoned in favor of rollers that held coils of film, much like latter-day microfilm threaders. As the viewer looked through the peephole at the top of the cabinet and cranked a handle, the frames went by, giving the appearance of a moving image. There is some dispute as to whether Edison was truly the first to develop a commercial-quality motion picture machine, but there is no doubt that he again applied a familiar pattern in a new domain. His mind stored successful patterns, and he continuously searched for analogous situations in which his favorite shapes and structures could be used.
The key to recognizing patterns that lead to new solutions is to spot something that already works, learn why it works, and then reapply or complete the successful model. That is the thinking habit that Thomas Edison learned early in his career. He recognized recurring patterns in his environment that others were missing, and he practiced reapplying and completing those patterns.
For some inventors, the patterns that needs to be recognized are hidden in data that would seem meaningless to most people. This was the case with a computer scientist named Max Levchin. Levchin is thin, with short black hair, and he often seems reticent about what he’s doing. As a child growing up in the Ukraine, Levchin drew pictures and took art classes, and these experiences may have helped him develop an appreciation of patterns. He also became obsessed with cryptography, the science of encoding information using secret patterns. Growing up under the old Soviet regime convinced him of the need to carry out communications in a way that would be undetectable by authorities.
In 1991 the teenaged Levchin immigrated with his family to Chicago. As a computer science student at the University of Illinois, he immersed himself in the mathematics of creating and breaking codes, not only making it the focus of his studies but also, he says, turning his pursuit into a “huge hobby” that consumed countless days and nights at the supercomputer center on the Urbana-Champaign campus.10
Dreaming that he would one day profit from his passion, Levchin aimed to start a company that would process financial transactions over the Internet, devising codes so unbreakable that hackers wouldn’t be able to read the data even if they intercepted them. To make good on his goal, Levchin moved to Silicon Valley after graduating in 1998. With the emergence of eBay, he noticed that a growing number of people were buying and selling goods to strangers. It was the old Soviet black market, only legal.
Levchin created in his mind the opportunity to replace the predominant way online buyers paid for their order. More than 80 percent of them were sending paper checks through the mail, a ridiculous delay for a real-time marketplace. Millions of part-time hobbyists, collectors, and small-business owners needed a simple way to exchange money instantly without having to open expensive merchant accounts to accept and process credit cards. Along with Peter Thiel, a financial hedge fund manager Levchin met at a Stanford University lecture, Levchin cofounded PayPal, a Palo Alto, California, company that suddenly became the leading processor of person-to-person (P2P) payments over the Internet.
As designed by Levchin, PayPal was a simple way of allowing person A to send an e-mail payment to person B. The parties didn’t have to know each other’s name or location. The PayPal software authorized person B to take money from person A’s bank account or to charge A’s credit card. The software therefore had to police the transaction.
As it turned out, using advanced cryptography was only a small part of PayPal’s success. Yes, financial data had to be stored and exchanged between computers using encrypted formats. But that technology had already been invented and built into standard Web browsers and databases. Millions of small businesses operating on eBay would adopt Levchin’s invention only if it proved to be secure enough. So Levchin went to work searching for patterns within all those financial transactions.
What Levchin detected was patterns of crime. He didn’t simply notice that there was a lot of fraud happening online. What he began to spot were specific tip-offs of online criminal behavior. People who sign up for PayPal provide certain information, such as name, postal address, e-mail address, bank account numbers, and so on. Levchin decided to look for correlations between the information PayPal had on each customer and the customer’s behavior in using the system. If a user attempted to conduct several transactions at once, tried to make transfers of high dollar amounts, or tried to send payments to notorious locations overseas or to unverified addresses, Levchin wondered what this behavior might say about the probability that the transaction was fraudulent.
There was no doubt that the patterns of crime were there to be found. Whereas Visa and MasterCard report an overall fraud rate between 0.05 percent and 0.07 percent, a Gartner Group study of Web merchants indicated that the figure soared to 1.13 percent for online transactions. In other words, buyers and sellers online faced a twenty times greater risk of not being able to recover the money or merchandise due to them. In May 2000, officials from the Federal Bureau of Investigation revealed the results of Operation Cyber Loss, a sweep of stings that led to the arrests of ninety alleged con artists charged with defrauding fifty-six thousand citizens out of $117 million, mainly through online auction fraud, stolen credit card numbers traded and used over the Internet, and wholesale identity theft. “Subjects and victims involved in this operation were scattered throughout the world,” Thomas T. Kubic, a deputy assistant director at the FBI, told a congressional hearing. “These cases reflect the nature of fraudsters to migrate from one fraudulent scheme to another, and [are] indicative of criminal behavior that would only continue to expand if left unaddressed.”11 Perhaps it would be more efficient to stop this kind of fraud before it happened rather than try to prosecute it after the fact.
Levchin invented financial surveillance software that closely monitored PayPal’s customers and alerted the company to any suspicious account activity. The original requirement was that the crime recognition program show results on one screen that, like a police blotter, “could be parsed by a human in seconds,” he says. As each transaction was placed, the screen would flash either the clear sign or a variety of red flags.
Levchin named his fraud detection program Igor, after a Russian hacker it apprehended early on, and PayPal filed for patents on Igor’s ability to mine the data for clues. The patterns that this twenty-five-year-old had built into the software were so fascinating and useful that officials from the FBI, the U.S. Secret Service, and the U.S. Postal Service routinely drop by PayPal’s offices to check cases against Igor’s data to learn some of the crime patterns it reveals.
A typical online crime pattern works this way: The software spots an aggregation of fake accounts, stolen credit card numbers, and money withdrawals. The scam works as follows. A criminal steals a bunch of credit card numbers, opens many fake PayPal accounts at once under the stolen identities, and links the accounts to the stolen card numbers. Lots of money is then transferred from other accounts to one central account. Then the hacker attempts to purchase consumer electronics online or to withdraw the money in cash. When this kind of likely scam is spotted, accounts can be frozen instantly and transactions can be halted or delayed.
Some of the patterns that Levchin began looking for aren’t obvious at all. Does the physical distance between buyer and seller have any correlation to fraud? If you opened your account in the middle of the night, does that increase the chance that you are a criminal? If buyer and seller are both transacting in the middle of the night in their respective time zones, does that increase the likelihood of fraud? If your password hint question happened to be “What is your favorite color?” and you said “Black,” what does that say about your criminal intent? If you capitalize the first letter of your name in your e-mail address, does that decrease the chances you’re a fraudster? “We looked at every little thing,” Levchin says. In other words, he was creating perhaps the closest thing we have to the predictive human–machine technology depicted in the movie Minority Report.
As in the movie, a single scrap of information may not be important in and of itself, but combining everything into a model can result in a powerful predictor. Levchin calls Igor “suspect-generation technology.” These small patterns could lead to big findings: funneling money to the Russian mob, say, or financing a small war in Asia. PayPal is a system for transferring U.S. dollars among people in dozens of countries, so none of these possibilities is far-fetched.
Levchin was positing relationships that didn’t previously exist and using new technology to do it. As a result, his invention was something that no one else had and that everyone seemed to want. The company raised more than $200 million in venture capital, with most of that coming after the dot-com collapse. Soon, PayPal was processing the payments for one in every four eBay transactions. With Levchin’s Igor program, PayPal was able to claim an online fraud rate of about 0.3 percent, thus eliminating 75 percent of the risk of sending payments online. Citibank and Bank One, and even eBay itself, started rival online payment services, but no one has been able to catch PayPal. Bank One soon had to shut down its system after experiencing fraud rates as high as 25 percent.
“Humans are extremely good at pattern recognition,” says Levchin. “Computers are only as good as programmed, which is a shame.” The best solution, he says, is to have the computer crunch through millions and millions of pieces of data and instantly find the patterns it has been told to find. Then a human can review the results and assess the significance of the finding: Is this transaction normal and legitimate, or is it alarming and dangerous? Is this person exactly the kind of customer we want, or one of America’s most wanted? Is this person a successful online merchant with a hot product, or an international rogue with only crime in mind? Among the other things Levchin recognized was a new pattern for getting computers and humans to work well together, a combination that is especially useful these days.