5
The Cambrian Explosion
TO GET TO the next milestone in brain evolution, we must leave the era when the first bilaterians were wiggling around and jump forward fifty million years. The ancient world this brings us to is the Cambrian period, an era that stretched from 540 to 485 million years ago.
If you peered around the Cambrian, you would see a world very different from the older Ediacaran. The gooey microbial mats of the Ediacaran that turned the ocean floor green would have long since faded and given way to a more familiar sandy underbelly. The sensile, slow, and small creatures of the Ediacaran would have been replaced by a bustling zoo of large mobile animals as varied in form as in size. This wouldn’t resemble a zoo you would enjoy—this was a world ruled by arthropods, the ancestors of insects, spiders, and crustaceans. These arthropods were far more terrifying than their modern descendants; they were massive and armed with hauntingly oversize claws and armored shells. Some grew to over five feet long.
The discovery of steering in our nematode-like ancestor accelerated the evolutionary arms race of predation. This triggered what is now known as the Cambrian explosion, the most dramatic expansion in the diversity of animal life Earth has ever seen. Ediacaran fossils are rare and sought after, but Cambrian fossils, if you dig deep enough, are all over the place, and they encompass a mind-boggling diversity of creatures. During the Ediacaran period, animals with brains were humble inhabitants of the seafloor, smaller and less numerous than their brainless animal cousins like the coral and anemones. During the Cambrian period, however, animals with brains began their reign over the animal kingdom.
Figure 5.1: The Cambrian world
Original art by Rebecca Gelernter
One lineage of Grandma Worm remained relatively unchanged and shrank in size, becoming the nematodes of today. Another lineage became the masters of this era, the arthropods. Lineages of these arthropods would independently develop their own brain structures with their own intellectual abilities. Some, such as the ants and honeybees, would go on to become impressively smart. But neither the arthropod nor the nematode lineage is ours. Our ancestors were likely not very conspicuous in the Cambrian cacophony of terrifying creatures; they were barely bigger than early bilaterians, only a few inches long, and not particularly numerous. But if you spotted them, they would have looked refreshingly familiar—they would have resembled a modern fish.
Fossil records of these ancient fish show several familiar features. They had fins, gills, a spinal cord, two eyes, nostrils, and a heart. The easiest-to-spot feature in fossils of these creatures is the vertebral column, the thick interlocking bones that encased and protected their spinal cord. Indeed, taxonomists refer to the descendants of this ancient fishlike ancestor as vertebrates. But of all the familiar changes that emerged in these early vertebrates, the most remarkable was surely the brain.
The Vertebrate Brain Template
The brains of invertebrates (nematodes, ants, bees, earthworms) have no recognizably similar structures to the brains of humans. The evolutionary distance between humans and invertebrates is too distant; our brains are derived from too basic a template in our bilaterian ancestor to reveal any common structures. But when we peer into the brain of even the most distant vertebrates, such as the jawless lamprey fish—with whom our most recent common ancestor was the first vertebrate over five hundred million years ago—we see a brain that shares not only some of the same structures but most of them.
Figure 5.2: Our Cambrian ancestors
Original art by Rebecca Gelernter
From the heat of the Cambrian explosion was forged the vertebrate brain template, one that, even today, is shared across all the descendants of these early fishlike creatures. If you want a crash course in how the human brain works, learning how the fish brain works will get you half of the way there.
The brains of all vertebrate embryos, from fish to humans, develop in the same initial steps. First, brains differentiate into three bulbs, making up the three primary structures that scaffold all vertebrate brains: a forebrain, midbrain, and hindbrain. Second, the forebrain unfolds into two subsystems. One of these goes on to become the cortex and the basal ganglia, and the other goes on to become the thalamus and the hypothalamus.
This results in the six main structures found in all vertebrate brains: the cortex, basal ganglia, thalamus, hypothalamus, midbrain, and hindbrain. Revealing their common ancestry, these structures are remarkably similar across modern vertebrates (except for the cortex, which has unique modifications in some vertebrates, such as mammals; stay tuned for breakthrough #3). The circuitry of the human basal ganglia, thalamus, hypothalamus, midbrain, and hindbrain and that of a fish are incredibly similar.
Figure 5.3: The shared embryonic development of vertebrates
Original art by Mesa Schumacher
The first animals gifted us neurons. Then early bilaterians gifted us brains, clustering these neurons into centralized circuits, wiring up the first system for valence, affect, and association. But it was early vertebrates who transformed this simple proto-brain of early bilaterians into a true machine, one with subunits, layers, and processing systems.
The question is, of course, what did this early vertebrate brain do?
Thorndike’s Chickens
Around the same time that Ivan Pavlov was unraveling the inner workings of conditional reflexes in Russia, an American psychologist by the name of Edward Thorndike was probing animal learning from a different perspective.
Figure 5.4: The brain of the first vertebrates
Original art by Mesa Schumacher
In 1896, Edward Thorndike found himself in a room full of chickens. Thorndike had recently enrolled in Harvard’s master’s program in psychology. His main interest was studying how children learn: How best can we teach children new things? He had numerous ideas for experiments, but to Thorndike’s chagrin, Harvard would not allow him to conduct experiments on human children. So Thorndike had no choice but to focus on subjects that were easier to obtain: chickens, cats, and dogs.
To Thorndike, this wasn’t all bad. A staunch Darwinist, he was unwavering in his view that there should be common principles in the learning of chickens, cats, dogs, and humans. If these animals shared a common ancestor, then they all should have inherited similar learning mechanisms. By probing how these other animals learned, he believed he might be able to also illuminate the principles of how humans learned.
Thorndike was both extremely shy and incredibly smart, so he was perhaps the perfect person to engage in the solitary, meticulously repetitive, and undeniably clever animal studies that he pioneered. Pavlov did his groundbreaking psychology work when he was middle-aged, after an already famed career as a physiologist, but Thorndike’s most famous work was his first. It was his doctoral dissertation, published in 1898, when he was twenty-three, for which he is most well known. His dissertation: “Animal Intelligence: An Experimental Study of the Associative Processes in Animals.”
Thorndike’s genius, like Pavlov’s, was in how he reduced hopelessly complex theoretical problems to simple measurable experiments. Pavlov explored learning by measuring the amount of saliva released in response to a buzzer. Thorndike explored learning by measuring the speed with which animals learned to escape from what he called puzzle boxes.
Thorndike constructed a multitude of cages, each with a different puzzle inside that, if solved correctly, would open an escape door. These puzzles weren’t particularly complex—some had latches that when pushed would open the door; others had hidden buttons; others had hoops to pull. Sometimes the puzzle did not require a physical contraption, and Thorndike would just manually open the door whenever the animal did something specific, such as lick itself. He placed various animals in these cages, put food outside to motivate the animals to get out of the boxes, and measured exactly how long it took them to figure out the puzzle.
Once the animal escaped, he would record the animal’s time, and then have the animal do it again, and again, and again. He would calculate the average time it took animals to solve a given puzzle on their first trial, compare that with the time for their second, and go all the way to how fast they solved it after as many as a hundred trials.
Thorndike originally wanted to probe the dynamics of imitation, a feature of learning he believed would exist across many animal species. He allowed untrained cats to watch trained cats escape from various puzzle boxes to see if it had any effect on their own learning. In other words, could cats learn through imitation? It seemed at the time that the answer was no; they didn’t get any better by watching (note that some animals can do this; stay tuned for breakthrough #4). But in this failure, he discovered something surprising. He found that these animals did all share a learning mechanism—it just wasn’t the one he originally expected.
Figure 5.6: Animals learning through trial and error
Images from Thorndike, 1898
When first placed in a cage, the cat would try a whole host of behaviors: scratching at the bars, pushing at the ceiling, digging at the door, howling, trying to squeeze through the bars, pacing around the cage. Eventually the cat would accidently press the button or pull the hoop, and the door would open; the cat would exit and happily eat its prize. The animals became progressively faster at repeating the behaviors that got them out of the box. After many trials, cats stopped doing any of their original behaviors and immediately performed the actions required to escape. These cats were learning through trial and error. He could quantify this trial-and-error learning with the gradual decay in the time it took for animals to escape (f igure 5.6).
What was most surprising was how much intelligent behavior emerged from something as simple as trial-and-error learning. After enough trials, these animals could effortlessly perform incredibly complex sequences of actions. It was originally believed that the only way to explain such intelligent behavior in animals was through some notion of insight or imitation or planning, but Thorndike showed how simple trial and error was all an animal really needed. Thorndike summarized his result in his now famous law of effect:
Responses that produce a satisfying effect in a particular situation become more likely to occur again in that situation, and responses that produce a discomforting effect become less likely to occur again in that situation.
Animals learn by first performing random exploratory actions and then adjusting future actions based on valence outcomes—positive valence reinforces recently performed actions, and negative valence un-reinforces previously performed actions. The terms satisfying and discomforting went out of favor over the decades following Thorndike’s original research; they had an uncomfortable allusion to an actual internal sensation or feeling. Psychologists, including Thorndike, eventually replaced the terms satisfying and discomforting with reinforcing and punishing.
One of Thorndike’s intellectual successors, B. F. Skinner, went so far as to suggest that all animal behavior, even in humans, was a consequence of nothing more than trial and error. As we will see with breakthroughs #3, #4, and #5 in this book, B. F. Skinner turned out to be wrong. But while trial and error does not explain all of animal learning, it undergirds a surprisingly large portion of it.
Thorndike’s original research was on cats, dogs, and birds—animals that share a common ancestor around 350 million years ago. But what about more distant vertebrate cousins, those that we share an ancestor with as far back as 500 million years ago? Do they too learn through trial and error?
A year after his 1898 dissertation, Thorndike published an additional note showing the results of these same studies performed on a different animal: fish.
The Surprising Smarts of Fish
If there is any member of the vertebrate group that humans bear the most prejudice against, it is fish. The idea that fish are, well, dumb is embedded in many cultures. We have all heard the folklore that fish cannot retain memories for more than three seconds. Perhaps all this prejudice is to be expected; fish are the vertebrates that are the least like us. But this prejudice is unfounded; fish are far smarter than we give them credit for.
In Thorndike’s original experiment, he put a fish in a tank with a series of transparent walls with hidden openings. He put the fish on one side of the tank (in a bright light, which fish dislike), and on the other side of the tank was a desirable location (the dark, which fish prefer). At first, the fish tried lots of random things to get across the tank, frequently banging into parts of the transparent wall. Eventually the fish found one of the gaps and made it through to the next wall. It then repeated the process until it found the next gap. Once the fish made it past all the walls to the other side, Thorndike picked it up, brought it back to the beginning, and had it start again, each time clocking how long it took the fish to get to the other side. Just as Thorndike’s cats learned to escape puzzle boxes through trial and error, so did his fish learn to quickly zip through each of the hidden openings to escape the bright side of the tank.
This ability of fish to learn arbitrary sequences of actions through trial and error has been replicated many times. Fish can learn to find and push a specific button to get food; fish can learn to swim through a small escape hatch to avoid getting caught in a net; and fish can even learn to jump through hoops to get food. Fish can remember how to do these tasks for months or even years after being trained. The process of learning is the same in all these tests: fish try some relatively random actions and then progressively refine their behavior depending on what gets reinforced. Indeed, Thorndike’s trial and error learning often goes by another name: reinforcement learning.
If you tried to teach a simple bilaterian like a nematode, flatworm, or slug to perform any of these tasks, it would fail. A nematode cannot be trained to perform arbitrary sequences of actions; it will never learn to navigate through hoops to get food.
Over the next four chapters we will explore the challenges of reinforcement learning and learn why ancestral bilaterians, like modern nematodes, were unable to learn this way. We will learn about how the first vertebrate brains worked, how they overcame these earlier challenges, and how these brains flowered into general reinforcement learning machines.
The second breakthrough was reinforcement learning: the ability to learn arbitrary sequences of actions through trial and error. Thorndike’s idea of trial-and-error learning sounds so simple—reinforce behaviors that lead to good things and punish behaviors that lead to bad things. But this is an example where our intuitions about what is intellectually easy and what is hard are mistaken. It was only when scientists tried to get AI systems to learn through reinforcement that they realized that it wasn’t as easy as Thorndike had thought.