The Princeton Guide to Evolution

Genetics of Speciation

H. Allen Orr and Daniel McNabney

OUTLINE

1. Genetics of prezygotic isolation

2. Genetics of postzygotic isolation

3. Summary

This chapter reviews current understanding of the genetic basis of speciation. Much progress has been made in the past several decades in uncovering how new species arise genetically. These recent studies analyze the barriers that prevent gene flow between closely related populations or species.

GLOSSARY

Dobzhansky-Muller Model. A model that explains how hybrid sterility and inviability can evolve unopposed by natural selection. The model emphasizes interactions among two or more loci.

Epistasis. Nonadditive interaction between loci.

Extrinsic Postzygotic Isolation. Reproductive isolation that results when hybrids are intermediate in phenotype and fall between parental niches.

Intrinsic Postzygotic Isolation. Reproductive isolation that results when hybrids suffer from developmental defects.

Pheromone. A chemical that can affect the behavior of other individuals, typically individuals that belong to the same species.

Most biologists define speciation as the evolution of reproductive isolation between populations (see chapter VI.1). This reproductive isolation is traditionally broken into two types: prezygotic and postzygotic. In prezygotic reproductive isolation, genes do not flow readily between populations or species because of barriers (e.g., courtship differences) that act before the formation of hybrid offspring. In postzygotic reproductive isolation, genes do not flow readily between populations or species because of barriers (e.g., hybrid sterility) that act after formation of hybrid offspring. (Genes cannot move between two populations if all hybrids between these populations are inviable or sterile.) In general, we are most interested in those forms of reproductive isolation—and in the genes giving rise to them—that appear early during the process of speciation. We are less interested, for example, in those forms of isolation that appear after the attainment of complete reproductive isolation. In practice, however, it is sometimes difficult to determine which forms of isolation arose or which genes diverged earliest during speciation.

A good deal is now known about the genetic basis of reproductive isolation, and much of this information has been obtained over the last two decades or so. Indeed, among contemporary evolutionary biologists, much of the progress in and excitement about speciation has focused on developments in the genetics of speciation. Here we summarize some of these developments. As will become clear, we know far more about the genetics of postzygotic than prezygotic isolation.

1. GENETICS OF PREZYGOTIC ISOLATION

Although the genetic basis of prezygotic isolation is less well understood than that of postzygotic isolation, evolutionary biologists have long appreciated its importance in speciation. For instance, Ernst Mayr—a key figure in the Modern Synthesis—argued that “if we were to rank the various isolating mechanisms of animals according to their importance, we would have to place behavioral isolation far ahead of the others.” Also, recent empirical studies in taxa including fruit flies and birds reveal that prezygotic isolation can evolve more quickly than postzygotic isolation, especially when populations or species occur in the same geographic area.

A number of theories explain how prezygotic isolation evolves between populations. Many emphasize the role of sexual selection in the evolution of traits. When such evolution occurs independently in two geographically separated populations, it can yield taxa that may no longer find each other attractive once they encounter each other again. (Variants of sexual selection include Fisherian runaway, good genes, sensory drive, and sexual conflict; see chapter VI.5.)

Three main classes of phenotypes can give rise to prezygotic reproductive isolation: ecological, behavioral, and gametic. Although we know something about the genetic basis of each class, some of these barriers are better understood than others. We consider each in turn.

Ecological Isolation

Adaptation to different local environments can lead indirectly to reproductive isolation. One of the best-understood examples of ecological isolation involves two species of monkeyflower (Mimulus). These species are adapted to different pollinators, and this differential adaptation gives rise to reproductive isolation. In particular, M. lewisii has pink flowers and is pollinated primarily by bumble bees, whereas M. cardinalis has red flowers and is pollinated primarily by hummingbirds. Despite overlap in part of their geographic ranges, these two species show essentially complete reproductive isolation.

Genetic analyses of 12 floral traits that differ between these species—differences that likely underlie adaptation to bumble bee versus hummingbird pollinators—reveal that a total of 47 different chromosomal regions are involved. For 9 of these 12 traits, a locus of major effect (defined operationally as a locus that explains at least 25 percent of the species difference) was found, suggesting that these trait differences might have a reasonably simple genetic basis. One of these chromosomal regions includes the YUP locus, which regulates the amount of yellow carotenoid pigment incorporated into the petals of flowers. Genetically moving the YUP region from each species into the other causes a dramatic shift both in flower color and visitation by bee versus hummingbird pollinators in the field.

Habitat isolation is thought to explain the persistence of a hybrid species of wild sunflower (Helianthus). The hybrid species, H. paradoxus, which formed between two salt-sensitive species, H. annuus and H. petiolaris, shows at least a fivefold increase in fitness under high-salt conditions relative to its parental species. Increased salt tolerance has allowed the hybrid species to invade brackish salt marshes in which neither parental species can survive. Filling this novel niche appears to isolate the hybrid species from the two parental types. Genetic analysis of the parental species shows that at least 17 chromosomal regions contribute to the ability to survive in high-salt areas; some evidence suggests that genes that are involved in calcium transport play a role in this salt tolerance.

Temporal isolation, which occurs when two species are reproductively isolated because of differences in the timing of breeding, represents another form of prezygotic isolation often connected to ecology. Unfortunately, rigorous studies of the genetic basis of temporal isolation appear to be lacking.

Sexual Isolation

Differences in courtship signals or rituals can also cause prezygotic isolation between species. During courtship, one sex may present a signal that must be interpreted correctly by the opposite sex. If the signal and/or the preference for the signal diverge between independently evolving populations, reproductive isolation can result. Indeed, in many taxa individuals prefer signals from individuals belonging to the same species relative to those from individuals belonging to other species.

To take a well-known example, differences in pheromones between populations or species can cause sexual isolation. A large body of work has examined this phenomenon in the fruit fly Drosophila. Using a simple method to transfer pheromones between species, Jerry Coyne and colleagues showed that pheromonal differences explain several examples of sexual isolation between species of the D. melanogaster group (which includes the species D. melanogaster, D. simulans, D. mauritiana, and D. sechellia). D. melanogaster and D. sechellia are both sexually dimorphic species (i.e., males and females differ in their predominant pheromone), whereas D. mauritiana and D. simulans are sexually monomorphic (i.e., males and females have the same predominant pheromone).

Studies have shown that at least five regions of the third chromosome contribute to differences in female pheromones between species of the D. melanogaster group. Genes that contribute to a difference in male pheromones between D. simulans and D. sechellia map throughout the genome, with at least one gene on each major chromosome arm. In other groups of Drosophila, epistatic interactions between genes on the X and second chromosome alter the relative amounts of two pheromones between D. pseudoobscura and D. persimilis. In the virilis group of Drosophila, genetic mapping of D. virilis, D. novamexicana, and D. lummei female pheromonal differences has identified several chromosomal regions on the autosomes.

The European corn borer, Ostrinia nubilalis, provides another striking example of the role of pheromones in sexual isolation. A difference in pheromone blend isolates two forms of this species. Just two loci account for both the difference in pheromone production and the preference for strain-specific pheromone blend seen in these corn borer populations.

Divergence in color pattern can also cause behavioral isolation. In cichlids found in Lake Victoria, females prefer male color that contrasts with the surrounding light environment. (The brightness and color of light varies with water depth, among other factors.) Female vision has adapted to local light conditions, and in response, males have evolved color patterns that stand out against their local environments. Differences in female vision reflect changes in opsin proteins that allow females to distinguish colors. One of the genes that encode opsin proteins, long-wavelength-sensitive opsin, is the most variable opsin gene in cichlids and shows strong signs of divergent natural selection among populations. (Genetic differentiation at other genes that are thought to be neutral confirms that populations are reproductively isolated.) As human activity has caused the water in Lake Victoria to become cloudy, species diversity has fallen. It appears that increased turbidity prevents females from distinguishing between males belonging to the same versus different species, leading to a collapse of prezygotic reproductive isolation. Taken together, this evidence suggests that female preference for male color is a key isolating barrier in Lake Victoria cichlids.

Another example of sexual isolation that involves color pattern differences occurs among Solomon Island flycatchers. Plumage color pattern acts as a species recognition signal among populations. In particular, Makira Island flycatchers have a chestnut belly, whereas flycatchers on nearby islands have black bellies. Albert Uy and colleagues hypothesized that the MC1R gene, which has been shown to explain color differences among a number of animal species, might underlie differences in plumage color. After sequencing MC1R from individuals from each population, Uy and colleagues identified a change in a single amino acid between the Makira Island and Santa Ana Island populations that shows perfect association with the color pattern difference, suggesting a role for MC1R in sexual isolation between those islands. Interestingly, a black-bellied population that resides on Ugi Island does not show the same association between amino acid change and plumage difference, which suggests that different genes may cause plumage changes in different populations of Solomon Island flycatchers.

Songs often represent an important component of courtship rituals. Courtship songs in animals such as crickets, fruit flies, frogs, and birds can diverge, giving rise to sexual isolation. In Laupala, a genus of Hawaiian crickets, differences in courtship song between the species L. paranigra and L. kohalensis are caused by differences in several chromosomal regions. Each of these chromosomal regions appears to explain a small amount of the total species difference in courtship song, which suggests that differences in song in this system involve many genes of small effect.

Gametic Isolation

Finally, prezygotic isolation can occur after mating/spawning but before formation of a hybrid zygote, so-called postmating prezygotic isolation. One type of postmating prezygotic isolation involves isolation between the gametes of two species. Examples of such gametic isolation occur in abalones and sea urchins, both of which release their gametes into the water. The abalone and sea urchin systems have been thoroughly studied genetically. In abalones, fertilization requires the sperm protein lysin to interact successfully with the egg protein VERL. Lysin shows high levels of divergence between species; both lysin and VERL show little polymorphism within species. The combination of high levels of divergence between species and low levels of polymorphism within species suggests that positive selection has driven divergence of these proteins between species. Similarly, in sea urchins, fertilization requires the bindin protein of sperm to interact successfully with a surface receptor on eggs. The bindin protein has also been shaped by positive selection, particularly between species that are found together geographically.

2. GENETICS OF POSTZYGOTIC ISOLATION

Serious discussion of postzygotic isolation, like so much else in evolutionary biology, began with Darwin, who devoted a chapter of The Origin to hybrid sterility. Darwin was primarily concerned with the problem of how something as obviously maladaptive as hybrid sterility could evolve under natural selection (this problem is sometimes called “Darwin’s dilemma”). Later, during the Modern Synthesis of the 1930s and 1940s, progress was made on both theoretical and empirical aspects of the genetics of postzygotic isolation. The theoretical progress included a plausible solution to Darwin’s dilemma, and the empirical progress included work on the genetic basis of hybrid sterility and inviability, in both animals and plants. Although the genetics of postzygotic isolation was largely neglected in the decades following the Modern Synthesis, a new burst of work began during the 1980s, particularly in model systems like the fruit fly Drosophila. This work has continued into the genomics era, with the identification and characterization of individual genes that cause the sterility or inviability of hybrids.

Before summarizing our current understanding of the genetics of postzygotic isolation, it is important to distinguish between the two forms that it can assume: extrinsic and intrinsic. In extrinsic postzygotic isolation, hybrids suffer low fitness not because of any inherent defect in development but because they fall between the ecological niches occupied by the parental species. In intrinsic postzygotic isolation, hybrids suffer low fitness because they suffer from defects in development.

Extrinsic Postzygotic Isolation

If different populations or closely related species are adapted to different ecological conditions—as they surely are—extrinsic isolation might often appear among their hybrids. Surprisingly, however, we know little about the genetics of this kind of reproductive isolation. Perhaps the best-studied example in animals involves a small fish, the threespine stickleback (Gasterosteus aculeatus). Two different forms or “morphs” of these sticklebacks are sometimes found in freshwater lakes along the west coast of North America, e.g., Paxton Lake on Texada Island. The limnetic morph is adapted to open waters and has a narrow morphology; the benthic morph is adapted to the littoral zone and has a broader, deeper morphology. Morphological differences include traits plausibly involved in predation, feeding, and adaptation to different habitats, e.g., jaw morphology and number of gill rakers.

Hybrids between the two morphs occur rarely in nature and can be produced readily in the laboratory. For many morphological characters, hybrids are intermediate between the benthic and limnetic parental forms. Studies by Schluter, Rundle, and colleagues have shown that these hybrids, although intrinsically healthy, suffer low fitness when placed in either the benthic or limnetic habitats. In short, these hybrids, with their intermediate morphologies, appear to fall between the morphologies required to succeed in either parental niche (benthic or limnetic). Genetic analysis reveals that some of the relevant morphological differences reflect divergence at many loci, while others reflect divergence at a modest number of genes.

Other examples of extrinsic postzygotic isolation—some involving intermediate behavior in hybrids (e.g., hybrid birds that migrate in an incorrect direction)—are well known. Unfortunately, most such cases have not yet been rigorously genetically analyzed.

Intrinsic Postzygotic Isolation

Much more is known about the genetics of intrinsic than extrinsic postzygotic isolation. It is clear that a variety of genetic mechanisms can, and sometimes do, cause developmental problems in hybrids. It has long been known, for instance, that polyploidy—the sudden doubling of chromosome number—plays a part in hybrid sterility in many plants (see chapter VI.9). Chromosomal arrangements that differ between species also sometimes cause fertility problems in species hybrids. Some species of mice, for example, feature a number of different chromosome rearrangements; when present in heterozygous form in hybrids, these rearrangement differences disrupt meiosis, thereby lowering fertility. At least in animals, however, intrinsic postzygotic isolation appears often to result from incompatibilities between genes in hybrids.

Early in the twentieth century, Bateson, Dobzhansky, and Muller each showed how such genic incompatibilities could evolve unopposed by natural selection, thus resolving Darwin’s dilemma. The key to the Dobzhansky-Muller model (Bateson’s precedent was appreciated only much later) is that new alleles at two or more genes might have beneficial fitness effects (or no fitness effects at all) within species but cause sterility or inviability when brought together in species hybrids. For example, consider a simple two-locus example in which two geographically separated species begin with the genotype aabb. In one species, an A mutation appears and spreads, yielding AAbb individuals; in the other species, a B mutation appears and spreads, yielding aaBB individuals. Both mutations are fit on their usual species genetic background. But if the two species later meet and cross, there is no guarantee that the resulting AaBb hybrids will be fertile and viable. The reason is that the A and B alleles have never been “tested” together in a common genome. The Dobzhansky-Muller model thus emphasizes the role of epistasis in speciation: genes may interact in unpredictable ways within hybrids, possibly causing inviability or sterility. (It should also be noted that the Dobzhansky-Muller model can cause hybrids to suffer a loss of fitness that is both post- and prezygotic: if two populations independently evolve different mating behaviors, the combination of relevant genes in hybrids may yield individuals that are sexually unattractive and so suffer low mating success.)

A considerable body of genetic data now supports the Dobzhansky-Muller model. During the 1980s and 1990s, geneticists mapped the loci that cause hybrid sterility or inviability in many different pairs of species. These studies nearly always revealed that incompatibilities are the result of between-locus interactions: sterility or inviability arises when some chromosomal region from one species is brought together in hybrids with other chromosomal regions from the other species, as predicted by the model. In some cases, hybrid sterility or inviability appears to involve a single Dobzhansky-Muller incompatibility; that is, hybrid sterility or inviability is caused solely by the interaction in hybrids between two (or perhaps three or four) genes. In other cases, species appear to be separated by many Dobzhansky-Muller incompatibilities; that is, many different combinations of genes from two species independently cause developmental problems and lower hybrid fitness. Indeed, hybrids between species that have diverged for a long period of time can suffer a kind of “overkill”: many different Dobzhansky-Muller incompatibilities may each be capable of killing or sterilizing hybrids. The rapid accumulation of genes that cause sterility or inviability of hybrids, leading to this kind of genetic overkill, is expected on theoretical grounds and has been dubbed the “snowball effect.”

Surprisingly, genetic analyses during the 1980s and 1990s revealed another pattern: the genes involved in these hybrid incompatibilities are often on the X chromosome. This so-called large-X effect is connected closely to another pattern that characterizes postzygotic isolation, at least in animals: Haldane’s rule, which states that when only one hybrid sex is sterile or inviable, it is the “heterogametic” sex, that is, the sex that carries both an X and a Y chromosome. A flurry of genetic studies, mostly in the fruit fly Drosophila, showed that Haldane’s rule has two likely causes: (1) the alleles involved in Dobzhansky-Muller incompatibilities are mostly recessive in their effects on hybrid fitness (and thus are fully expressed when they reside on the X chromosome of XY hybrids); and (2) the X chromosome has an especially high concentration of genes involved in Dobzhansky-Muller incompatibilities. The reasons for this recessivity and high concentration of postzygotic isolation genes on the X chromosome remain somewhat uncertain and are the focus of much current work.

In the last decade, evolutionary geneticists have devoted much attention to the molecular identification and characterization of the genes that cause intrinsic postzygotic isolation. This effort has proven difficult. The reason is simple: the attempt to identify “speciation genes” is the attempt to do genetics where it is, by definition, nearly impossible to do—between species, that is, between taxa that do not readily exchange genes. Several genetic and molecular techniques have been used to overcome this problem, and as a result, approximately a dozen genes that cause some hybrid sterility or inviability have been identified at the DNA sequence level. (Most of these studies were performed in Drosophila, though others were performed in vertebrates.)

Although this sample of genes is small, several patterns have already emerged from these studies. First, the genes that cause intrinsic postzygotic isolation have many different biological functions: some encode DNA-binding proteins, some encode enzymes, and yet others encode structural proteins. Second, comparison of DNA sequences between two species that produce sterile or inviable hybrids shows that the genes that cause these hybrid problems are often rapidly evolving. Third, this rapid evolution is often caused by positive selection. (This can be shown via several molecular population genetic tests—for example, the McDonald-Kreitman test—that use DNA sequence data from the two species that produce sterile or inviable hybrids.) The precise nature of the selection involved remains somewhat unclear, however. Geneticists are currently investigating two possibilities: that rapid evolution reflects (1) adaptation to the external ecological environment, or (2) “genetic conflict,” that is, adaptation to the “selfish” effects of other genes in the genome. Some evidence supports this second possibility—for example, some genes involved in hybrid sterility also appear to be involved in forms of genetic conflict. But more data are required before confident conclusions can be drawn about the frequency with which ecological adaptation versus genetic conflict drives the evolution of the genes causing intrinsic postzygotic isolation.

3. SUMMARY

In conclusion, evolutionary biologists now know a good deal about the genetic basis of postzygotic reproductive isolation and a growing amount about the genetic basis of prezygotic reproductive isolation. It seems clear that both forms of isolation typically involve a history of selection, whether natural or sexual. It is also clear that genes that have large effects on reproductive isolation exist, although reproductive isolation sometimes seems to result from the divergence of many genes, each of smaller effect.