CHAPTER ELEVEN

PROOF: THE EVIDENCE FOR EVIDENCE

The basic purpose of a trial is the determination of truth.…

Justice Potter Stewart¹

Is Minneapolis “on the edge of the Rockies?”² Google Maps says “no.” A glance out of any Twin Cities window says “no.” Jean Baudrillard said “yes.” Then again, Baudrillard also contended that America was a “simulation” whose only reality existed in television and movies.³ Baudrillard took a lot of flak for his pronouncements thirty years ago, though in hindsight, perhaps his real sin was radical understatement. Television, with its parade of infotainment and press briefings, has become more Walt Disney than Walt Cronkite. Cinema, after The Matrix debuted in 1999, also relinquished any remaining claims on reality. (Only an unembarrassed simulation would risk Keanu Reeves hauling around a copy of Baudrillard, though the movie winks at us by hollowing out the book so that Keanu/Neo can use it as a tote bag.) Virtual realities, alternative facts, creative accounting—anything goes in the era of post-truth simulation, with one exception. We still expect law to maintain some relationship with the truth—visitation rights, if not full custody.

Facts, truth, knowledge, and the relationships among them are complicated, and philosophers have developed a sophisticated discourse on these subjects. But lawyers are not philosophers, they’re advocates, and epistemology isn’t part of the curriculum—evidence is. Evidence is a wobbly blend of empiricism and armchair philosophy that converts factual claims into a legal truth: the verdict. Legal evidence is not about the facts, pure and entire; legal evidence is only what attorneys can convince courts to accept as fact. Baudrillard was partly right; we live in a legal Matrix.

Sometimes, law outright declines to take reality into consideration. For example, if police find a kilo of cocaine bearing a dozen clean prints from the defendant and damning DNA evidence, and obtain a willing confession, it’s quite likely that the defendant is guilty of possession as a factual matter. But if the police seized the kilo in a warrantless raid prompted solely by racial animus, the kilo ceases to exist, and if the kilo is the prosecutor’s only evidence, the legal truth switches from “guilty” to “not guilty.” Many people applaud this, though if the exclusion of tainted evidence is morally good or socially useful, it does confirm that law openly manipulates reality.

Indeed, law must manipulate facts. By themselves, facts are inert, and require mechanisms to give them social weight and meaning—and in law, those mechanisms are called the rules of evidence. To achieve a satisfying social truth, law uses filters, but to filter sensibly, law must first understand the basic facts. Before applying a single rule of evidence, judges must understand that an atlas is a more reliable guide than Baudrillard, at least when it comes to the location of the Rocky Mountains.

Every case in an adversarial system must tease out reality from competing narratives, but law’s history of locating truth is not reassuring. Although lawyers have had at least 4,000 years to develop good techniques to parse witness testimony (the oldest surviving legal text, the Code of Ur-Nammu, references perjury), good solutions eluded law. For many centuries, torture was the standard technique to locate truth. In ancient Rome, slaves often served as key witnesses in disputes between their owners. Roman law assumed slaves would lie to protect their masters, so the dungeon became an instrument of corroboration. Ancien régime France also allowed torture, with refinements—torment could last eighty-five minutes tops, with no leading questions allowed.⁴ Continental jurists also attempted to mathematically grade the credibility of different types of witnesses. Testimony by nobles, clerics, and men of property counted for more than that of their inferiors, testimony by older men carried more weight than that of younger ones, and testimony by parties and their relatives was excluded as inescapably biased.⁵ Today, science provides us with better means to sort evidence than the rack or the coat of arms, though various legal actors still make status-based assessments, believing or disbelieving cops because they’re cops, and so on. Law’s evidentiary process remains impressionistic, and has not kept pace with science or society, and a major problem is the law itself. Courts say they disdain tainted evidence, but the laws of evidence are themselves tainted.

Prove It

Each part of the U.S. legal system uses evidence differently, though only police, courts, and some agencies are bound by rules governing the collection, use, and analysis of data. Curiously, the institutions with the greatest powers to collect evidence also have the fewest rules governing its employment. Presidents head the largest evidence-gathering enterprise in human history, encompassing everything from the NSA to the CIA, though presidents are functionally unbounded by evidentiary limits when formulating policy, issuing executive orders, or giving commands for quasi-juridical detentions and assassinations. Presidents can and have given credence to evidence that would be inadmissible in court, that comes from unreliable sources, or that has been collected in violation of international and domestic law—as happened during the war on terror.

Congress also has considerable evidentiary resources, including research services, hearings, and reports furnished by private parties and other branches of government. It can also subpoena documents and testimony, conduct investigations, and impeach and try high officials. Though these are quasi-judicial powers, Congress is not subject to the procedural rules that obtain in normal courts; its operations lie largely beyond judicial review, and Senate impeachment trials eschew normal court procedure.⁶ The relationship between evidence and the legislature and executive can be fairly described as “it’s complicated.” This creates special difficulties as questionable evidence seeps into the general legal system, including the laws of evidence that are codified by Congress. Law emerges from an epistemic morass, but there’s nothing the rest of the legal system can do but play along.

Play along it does, with greater and lesser degrees of coherence and sincerity. For example, when agencies make rules, they have been required to consider the best available science; when agencies conduct in-house hearings, they employ a streamlined version of normal court rules. Police also have their own procedures governing evidence. Many of these procedures are driven by the potential for future litigation, because work premised on faulty or improperly collected evidence can be undone. Thus, while litigation represents only a small portion of legal activity, courts’ evidentiary practices shape attitudes across much of the legal universe, making courts the natural focus for any discussion of evidence. But are court practices actually good at distinguishing fact from fiction?

Objection, Relevance?

In litigation, relevance is the master evidentiary rule. The rule is deceptively simple: irrelevant evidence is generally excluded, while relevant evidence is presumptively fair game.⁷ And what’s “relevant”? Per the Federal Rules of Evidence (FRE), information is relevant when “(a) it has any tendency to make a fact more or less probable than it would be without the evidence; and (b) the fact is of consequence in determining the action.”⁸ With that definition, the question might be better framed as: What isn’t relevant? All the FRE require is “any tendency,” and there are few situations in which another data point does not, in some tiny way, change the odds. The big data revolution makes this abundantly clear. If an Amazon shopper buys a Diet Coke today after a lifetime of drinking Pepsi, the shift might indicate incipient diabetes, psychotic meltdown, or a change in domestic partner. This data point has tiny weight, but it still carries some information, and number crunchers will update their models accordingly. Should courts follow the same algorithmic procedures when they consider evidence? Again, the relevance standard is “any tendency” to make a fact more or less probable, so in the right context, there’s no logical reason why the switch to Diet Coke couldn’t be germane. The rules before trial are even more expansive, allowing parties to request information “relevant” to the dispute, even if that information would not itself be admissible.⁹ During police investigations and so on, “reasonable suspicion” alone can support a search for evidence (even if the suspicion is based on a factual or legal mistake).¹⁰ The evidentiary dragnet trawls up a lot of information, whose yield courts must sift.

The relevance rule doesn’t provide much help. Bodiless algorithms benefit from more information, but humans do not. Courts must therefore depart from a strict reading of the relevance rule to cut off what would otherwise be an infinite chain of inquiry. The question isn’t whether evidence is relevant, but whether it changes probabilities enough to be worth the time.¹¹ Relevance decisions are informed by experience, wisdom, and other judicial virtues, but in a technical sense, decisions to admit evidence are guesses, because until evidence is introduced, there’s no way to know how much new evidence changes probabilities. And the act of guessing itself changes the equation of a case, as the decision to admit or exclude evidence reflects judicial models of how a case will develop. This may not be “prejudicial” in the legal sense, but it’s definitely a pre-judgment. The consequences are usually immaterial, but appellate courts rarely second-guess trial judges’ evidentiary decisions, and biased judges can exclude relevant evidence to favor their own version of the case, with minority defendants the historical losers.^*

The relevance rule embodies many of the quandaries that afflict the laws of evidence. First, the most important evidentiary rules are unworkable at face value, and to function, law must allow judges the discretion to apply the rules. Law also provides a giant list of codified exceptions. For example, the rule against hearsay contains almost thirty escape-hatches, and the hearsay rule is itself an exception to the master relevance rule; indeed, it’s reasonable to describe almost all the other evidentiary rules as exceptions to the relevance rule. Second, evidentiary rules, like the standards of proof, are fundamentally probabilistic, and humans lack a strong intuition about probability. Finally, as we’ve seen, evidentiary decisions are generally difficult to have reviewed and reversed, removing a safety net from one of law’s most error-prone endeavors. Although evidentiary decisions are incredibly consequential, they are often made on the fly, argued by attorneys operating in real time before judges who must decide on the spot. The superstructure for processing evidence is a wobbly mess. Then again, so is most evidence.

Papers, People, and Problems

Documents and witnesses are staples of the litigator’s diet. Documents are exactly what they seem: records of all types, paper, electronic, and otherwise. (But not red groupers, as far as the Sarbanes-Oxley Act goes, as we saw in the Preface.¹²) The essential problem with documentary evidence is incomprehensible volume: life leaves a vast sediment of receipts, contracts, emails, spreadsheets, and text messages. Computers are thickening that sediment at a tremendous pace: by 2020, there will be over 5,000 gigabytes of digital information per human, plus reams of paper documents.¹³ Parsing documents has become exhausting, and it’s easy to overlook crucial information. Many cases are therefore decided without all the facts.

The huge amount of documentary evidence creates not only epistemic issues, but equality problems. Parties with greater resources have greater evidentiary arsenals. This is especially worrisome in criminal matters, in which prosecutorial resources often greatly outstrip those of the defense. The Court relieved some of the pressure via the Brady doctrine, which requires prosecutors to share exculpatory evidence with the defense.¹⁴ However, Brady has proved difficult to enforce and prosecutors sometimes fail to share. This is a problem with documents generally; one side often has exclusive control. The advent of metadata and backups has made it harder to tamper with, or deny the existence of, documents, but the problem remains.

Documents are, however, pretty reliable as evidence goes. Most documents are created long before litigation is even suspected, and are largely untainted by knowing cant. The cant comes later, when litigators quote documents selectively. While the rules allow rebuttal using the full document, doing so is cumbersome. Accidental misinterpretation is another problem, as documents rarely arrive with context. An email stating that “this corporation won’t hire people because they’re minorities” could be genuinely damning evidence of discrimination, or a mere typo (“won’t ^fire…”), or something else (an exhortation to hire only on the merits, without reference to personal characteristics). Everyone has experienced the possibility for misinterpretation in text messages. Text lacks prosody, so important nuances get lost, and I’m not aware of any rigorous jurisprudence on the interpretation of clarifying emoticons like smiley faces, excrement piles, and eggplants. Understanding documents requires witnesses, and this is where trouble really starts.

Witnesses, unlike documents, are highly changeable. Witnesses know that legal proceedings are underway, and even if they face no immediate liability, they have agendas and interests that may affect testimony. Witnesses lie, because that’s part of human nature. Indeed, researchers have noted that most people lie on a daily basis, and the most common reasons for lying are to conceal personal transgressions and to achieve personal gain, matters which are, of course, at the heart of litigation.¹⁵ In more religious ages, the prophylactic against lying involved oaths, one hand up and the other on a Bible. To say this feudal ritual has lost some of its power is gross understatement. Many countries now abjure oath-taking as useless, consigned to the same rubbish bin as witness torture.

The secular supplement to oath-taking is the perjury prosecution. In 1968, a presidential commission concluded that perjury was widespread, and while not much follow-up work has been done, the general sentiment is that perjury remains common, though to an unknown degree.¹⁶ (“It happens all the time,” per one federal judge.¹⁷) We know that witnesses lie, a conclusion reachable by common sense, and because courts regularly overturn convictions because police give misleading testimony—indeed, police perjury happens frequently enough that it’s been nicknamed “testilying.” But overturning a conviction is different from holding witnesses accountable, and law seems reluctant to chase perjurers. In part, perjury is difficult to prove: perjury prosecutions come after the fact (when memories have degraded further, and parties have lost interest), and law requires that a misstatement be both material and intentional, and “confusion, mistake, [and] faulty memory” obviate the required intent.¹⁸ All we can say definitively is that perjury is not a vigorous safeguard; no more than 185 people were arrested for federal perjury in 2008–2009 (0.1 percent of federal arrests) and many of them walked free.¹⁹ Similar figures likely obtain in state courts. Lying is a low-risk enterprise.

Even witnesses who do not intend to lie are not bracingly reliable. The basic rule is that any idiot is “competent” to testify, so a witness’s stupidity, drunkenness, or bias will not preclude him from proffering evidence.²⁰ Witnesses can also be manipulated, and this is a real problem in suspect identifications. Police lineups, for example, have long been plagued by manipulation. Law now forbids grossly biased lineups, so police cannot stock the tank with wildly implausible decoys (e.g., five white decoys and one black suspect) or overtly suggest which suspect a witness should pick out. But police can still manipulate in subtler ways (“take your time and be really sure”), and might do so unknowingly (by sending unconscious signals). Although researchers have suggested various improvements, such as requiring lineups to be conducted by officers unrelated to the case, police have been slow to reform.

Even with change, error rates will likely remain high, given that memory is fallible, especially in the stressful circumstances that often make people eyewitnesses in the first place. Another factor that’s important, given racial disparities in arrests, is the cross-race effect: witnesses have greater difficulty in correctly identifying faces from outside their own racial group.²¹ Adding these effects to other cognitive biases, varying incentives to lie, and the stress of testifying leaves eyewitness testimony much less reliable than jurors commonly believe. The Innocence Project conducted a review of convictions overturned on DNA evidence, and concluded that more than 70 percent were based on faulty eyewitness testimony.²² That’s only a sample, and one based on convictions known to be flawed. However, scholars have repeatedly observed memory’s capacity to fail in research settings, and have even managed to create false memories in test subjects, many of whom insist that their implanted memories are true even after the trick was revealed.²³ As many witnesses are coached or otherwise prepared for trial, this creates unsettling questions of reliability.

Eyewitnesses remain central to litigation, and there are few protections against misleading testimony. In Perry v. New Hampshire, the Court considered the propriety of admitting an identification made shortly after the crime, where the witness identified a black man on site (who was, suggestively, standing next to an officer near the scene), but was unable to repeat the identification in a subsequent photo lineup.²⁴ Nevertheless, the Court held, 8–1, that due process rights were not violated because there was no indication of improper police conduct and because cross-examination and other “safeguards” granted the defendant adequate protection against dubious testimony. Perhaps, but only if cross-examination and other safeguards are rigorous and sensible.

In fact, many of the rules governing witnesses’ testimony are concerned more with ritual than substance, and of these, the hearsay rule is perhaps the most important and troubling. “Hearsay” is testimony to establish a fact, given by a witness without direct knowledge of that fact.^* For example, if a witness saw the defendant’s blue Ford run over a person, and says as much in court, that’s “direct testimony.” But if someone simply tells the witness those details, and the witness testifies accordingly, to prove that the defendant ran over a man with a blue Ford, that’s hearsay. By default, hearsay is forbidden, being deemed too attenuated and unreliable to serve as evidence.²⁵ But hearsay is admitted all the time, pursuant to numerous exceptions. Some hearsay exceptions are sensible. For example, a county-recorded deed or NASDAQ’s records may be admitted under the property and business records exceptions to prove ownership of a house or the closing price of Intel. NASDAQ is not in the business of recording false stock prices to exonerate small-town brokers, so there’s no reason to believe such information is shoddy.

Unfortunately, most hearsay rules are not as sensible. There’s a lot of historical detritus in hearsay law, much of it based on assumptions that have been shown to be dubious, or even false. For example, “excited utterances” are exempt from hearsay restrictions, so a person can testify to details relating to “startling events,” presumably because the law believes that statements made “while the declarant was under the stress of excitement” are particularly credible.²⁶ Of course, recall of many types of information is compromised by stressful circumstances, but we’re stuck with the exception, because some ancient judge thought it a good idea.²⁷ On the subject of ancient, “ancient documents” are also exempt. This exception is an unhelpful marriage of law’s love of the past with the shady ethics of the small-town antiques dealer, as “ancient” is defined as at least twenty years old.²⁸ What’s so special about twenty years—is it the case that people planning to plant misleading evidence only do so 239 months in advance, but not 241? The list of moldy treasures continues. A particular gem is the statement made “under the belief of imminent death.”²⁹ This hearsay exception is limited to homicide prosecutions and civil cases to ascertain the cause or circumstances of death, and only when the original declarant is no longer available (presumably, because the “belief” of imminent death was accurate). Obviously, these special limitations implicitly admit that imminent-death hearsay might be particularly shaky, so it’s not clear why law allows its use in the most consequential species of case, the murder trial. But the “imminent death” exception hardly brims with reason in any event.

Budding mystery writers could make a career out of the hearsay exceptions alone: the buried “ancient document,” hearsay regarding “facts of personal or family history,” reputations about “boundaries of land,” statements made for “medical diagnosis or treatment” (who’s ever lied to his shrink?), “excited utterances,” and so on. And the aspiring Agatha Christie needn’t worry if she accidentally includes “double hearsay” or bungles the details of a particular exception. The rules provide a catchall option, allowing hearsay’s admission where the interests of justice require such and there are “equivalent circumstantial guarantees of trustworthiness.”³⁰ As the original guarantee is near worthless, ginning up “equivalent” guarantees of trustworthiness isn’t hard. Of course, one cannot spin a rich mystery out of hearsay doctrine alone. The mystery writer should therefore have her villain plant hair fibers, spent rounds, and fake tire treads. These might seem to be the kind of gratuitous flourishes that bring down a villain, because forensic science, unlike witnesses and hearsay, is supposed to be iron-clad. And so it can be in novels and movies. But in real life, forensics is anything but reliable. Perhaps Baudrillard is right: all is simulation, as fake as CSI, a show aptly situated in Las Vegas, the most unreal city in America.

Forensic Evidence

Forensics, loosely construed, has been around for a long time. The Bible refers to speech pattern analysis (in the Book of Judges, the Gileadites identify enemy Ephraimites based on their mispronunciation of “shibboleth”); Hippocrates commended physicians to study poisons to identify foul play; Romans autopsied Julius Caesar (conclusion: death by stab wound; stab #2 of 23 being deemed the fatal blow), and their lawyers analyzed bloody palm prints (to exonerate a murderer) and handwriting (to validate wills and contracts); and ancient Indian physicians assisted courts with matters concerning childbirth (finding: pregnancy lasts about ten lunar months or forty calendar weeks). Today, we look at ancient forensics and see a mess. At best, it confirmed the painfully obvious. More usually, it was false, tending to supply whatever conclusion was required by its sponsors: a mercenary and unscientific charade.

The rise of modern forensics in the nineteenth century supposedly changed all that, but the concurrent development of forensic science and modern police departments should be a clue that forensic science is, for the most part, not “science” at all. Proper science has no master and no supervening interest in a particular result; it is an independent witness. With the major exception of DNA evidence, however, almost all forensic science was created for, and remains sponsored and conducted by, law enforcement; forensic science is a witness with an agenda.³¹ This important detail goes overlooked in many trials, as do the considerable inaccuracies of forensics generally. As a result, courts and juries can be misled, especially as technology has become an article of faith in the search for truth.

One particular challenge with scientific and forensic evidence is that each depends on probability and statistics, branches of math that are not intuitive, and in which lawyers have little training.^* Two sadistic staples of college math departments suffice to show where intuition and math diverge. The first is the Birthday Problem, which asks how many people need to be in a room before there’s a 50/50 chance that two share the same birthday. It’s easy to intuit that this is a trick question, and that the number must be lower than, say, 183 (365 days, divided by two people, rounded up), but the number is much lower: just 23. The second example is the Monty Hall Problem, which involves a prize hidden behind one of three doors. You choose Door #3, but the host reveals that Door #1 hides nothing; is there any point to changing your choice given this new information? The answer is yes. Many people flatly refuse to believe this, and even after being forced to prove it in three different college classes, my certainty is more abstract than intuitive. The endnotes link to a proof, but it’s sufficient here to note two things.³² First, our mathematical intuitions are often wrong. Second, our prior assumptions should be updated in light of new data. In fact, people perform versions of this constantly, including by adjusting first guesses as soon as it’s clear there’s a trick question involved.

Lack of mathematical fluency creates endless problems for law, often when attorneys use math to “quantify” certainty or doubt to meet a standard of proof (itself usually framed in probabilistic terms, such as “more likely than not”). In several cases, prosecutors have argued that the chances of innocence are infinitesimally small by multiplying a series of numbers, a strategy that only works if each event is “independent” (i.e., unrelated). If 50 percent of a town’s residents own a car, 20 percent of cars are blue, and 10 percent of cars are Fords, then a prosecutor might argue that it’s highly unlikely that the victim, run down by a blue Ford, was hit by anyone other than the defendant (who owns a blue Ford). The population of owners of such rarities seems to be a mere 1 percent (i.e., 50% x 20% x 10%), close to proof beyond a reasonable doubt. But if Ford only makes blue cars, then a blue Ford isn’t quite so unusual—and neither is the suspect. (Roughly speaking.) A California case from the 1960s made precisely this sort of error.³³ Although that case became semi-infamous across Anglophone law, the error persists. In Britain, courts imprisoned several parents for the sudden deaths of their children, based on the same mistake, a series of injustices that prompted a rebuke to the Lord Chancellor by the Royal Statistical Society.³⁴

Math errors routinely affect criminal cases, civil cases (employment discrimination), quasi-Constitutional cases (affirmative action and districting), and regulatory enforcement (antitrust and securities fraud). Lawyers can string together numbers all day long, but unless the variables are independent (and in both science and law, independence is a fact and therefore must be proved, not assumed), calculations will be wrong. This presents special problems outside courts, as when gut intuition dominates decisions to stop or arrest; the overrepresentation of minorities among convicts can make racial profiling seem valid, but as we saw in Chapter 10, even though more black drivers were stopped and arrested by the SFPD, they were less likely to have contraband.

Forensic conclusions always involve math, and this makes them prone to misinterpretation regardless of other factors. For example, experts often testify about whether evidence, like a fingerprint, is a “match.” A “match” seems definitive, except that there’s no such thing as a definitive forensic match, only a range of odds. (For this reason, prosecutors faced with proving matters beyond a reasonable doubt hate to get into the mathematical weeds because there’s always mathematical “doubt.”) Rigorous experts can only offer, based on various assumptions, a range of “confidence” subject to error rates. Even when those ranges are accurate, they’re still prone to misinterpretation. An expert might be 99.999% confident that only 0.1% of the population exhibits a particular characteristic, so the prosecutor might summarily conclude that there’s a 99.9% chance that the defendant committed the crime (or a 0.1% chance that he didn’t). But this may not constitute proof beyond a reasonable doubt. Without any additional information, all that can be said is that in large cities, many people could share this characteristic; in Manhattan, at least 1,600 people might and when the island is swollen with workday commuters, up to 4,000, of which the defendant is only one.³⁵ Depending on other evidence, figures as low as ~0% or as high as 99+% could be derived.

All forensics operates on a spectrum of uncertainty, and of all forensics’ many techniques, only well-conducted DNA analysis operates at the comfortable end of the continuum. Scientists developed DNA analysis independent of law enforcement, and the core principles have been well studied, peer-reviewed, and pose few conceptual problems. Applying DNA analysis is the tricky part. There are several testing regimes of varying quality, and none of them test the entire DNA molecule. In theory, comparing the full sequence of two DNA molecules could produce near-infallible identifications, but doing so is impracticable, so forensics labs test a set of snippets. The fewer the snippets, the greater the possibility of ambiguity or error. The most common test, used by the FBI, tests twenty fragments by default (increased from thirteen, after 2016), though DNA samples degrade over time and often, labs can test fewer than twenty fragments, raising the possibility of misidentification.³⁶ Compounding the difficulty, results must be interpreted by lab staff and by courts, and this is a problem because of conceptions about what DNA can prove. Genetic material can all but disprove a match—to take an extreme case, if a genetic sample has XY chromosomes and no matching markers, then a female suspect can be freed as innocent.^* This is one reason why DNA evidence has been so useful in exonerations. “Matches,” however, merely describe ranges of possibility, and given testing’s incompleteness, a certain number of false positives are expected. (Worse, with lower-quality DNA samples, labs must use amplification techniques, vaguely akin to blowing up a photo, with losses in fidelity.) DNA identification can be quite impressionistic, and different technicians sometimes draw wildly different conclusions from the same sample. In one study, only 6 percent of labs were able to correctly exclude a match in a mixed DNA sample, and when called upon to describe the odds of matches, labs varied wildly—in the words of one researcher, the results were as different as two fertility labs setting the odds of pregnancy at 50/50 or 1 billion-to-one.³⁷

DNA analysis can provide useful and accurate information so long as the samples are good, the technicians reliable, and the results interpreted with integrity. Unfortunately, quality problems plague DNA analysis. Many samples are compromised, either through natural decay or by mistakes in collection. Most forensics labs are run by law enforcement, creating at least the appearance of bias. Training for technicians is often poor, the workload too great to allow for necessary care, and quality controls can be lax. Cross-contamination is not infrequent, where DNA from the suspect is introduced into samples from the victim (or vice versa), resulting in false matches that prove nothing more than that a suspect’s DNA, unremarkably, matches his own. In the 2000s, audits of Houston’s DNA lab revealed serious deficiencies in training and practice; audits also showed that Virginia’s state lab had done questionable work, and other labs have been similarly afflicted.³⁸ Even the FBI has been affected. Following a lab technician scandal involving falsified procedures and inadequate testing (a disaster discovered only by accident), the Office of the Inspector General noted serious deficiencies in FBI lab work, including inadequate protocols that prevented reproducibility and lackluster internal controls. The Inspector General also found that the FBI, despite knowing of seriously compromised analyses, failed to properly inform almost half of the affected parties.³⁹ Although problems at crime labs have been well documented for many years, protocols have not been uniformly improved.

While poor methodology taints DNA analysis, genetic testing is at least sound in theory; most forensic science cannot claim that much. Fingerprint analysis has been widely used for over a century, developed by Sir William Herschel (son of the astronomer) to authenticate documents in British Bengal, and was considerably elaborated by Sir Francis Galton, providing police with new techniques.^{* 40} While fingerprint analysis performs adequately as a security feature in door locks and iPhones, it’s probably inadequate on its own to prove anything “beyond a reasonable doubt.” For one thing, it cannot be proved that fingerprints are unique; all experts can do is describe the chances that a print did not come from a suspect.

With clean samples and a large body of general print data as reference, it’s at least possible to draw useful inferences. However, not many crime scenes offer clean prints, only a mix of incomplete, overlapping, smudged, or otherwise imperfect samples, which are subjected to equally imperfect analysis. The customary analysis involves three levels of detail: general shape (the loops, whorls, etc., formed by skin ridges), “Galton points” (the details of the ridges themselves), and microscopic variation (siting of sweat pores, and so on). The loops and whorls aren’t overly revealing and the microscopic details are often compromised in crime scene prints, so analysis tends to focus on Galton points, of which most humans have 75–175. Some countries require 24 or 30 points to correspond before declaring a match, but the U.S. does not, allowing investigators to make holistic judgments based on all three levels of analysis.⁴¹ In other words, print analysis is pretty subjective, and technicians can and do disagree about the same set of prints. Technicians have even disagreed with their own prior analyses; in one study, technicians (unwittingly) reviewed matches they had made previously, and for ~10 percent of samples, came to different conclusions in the second round.⁴²

The weakness of print analysis has brought a flurry of legal challenges, and the FBI felt compelled to defend law enforcement’s cherished tool in the 2000s. To that end, the Bureau commissioned a large study, and in 2009 (and to no one’s surprise), the study concluded that good prints could lead to highly certain identifications. Due to its flaws, the study was widely condemned. The controversy continued, and fingerprinting came under sustained attack. Science groups raised serious doubts about print methodologies and customary testimonial glosses (“the print is unique and we are confident that the fingerprint came from the defendant”), a practice that the AAAS described as “indefensible,” noting that there is “no scientific basis for determining when the pool of possible sources is limited to a single person [i.e., the suspect].”⁴³ Naturally, defenders of the indefensible rose to the bait, and then-Senator Jeff Sessions, a member of the Judiciary Committee, waved away the brewing controversies, stating that “I don’t think we should suggest that [print and other forensic analyses] that we have been using for decades are somehow uncertain.”⁴⁴ Well, those methods are uncertain, and tradition doesn’t change that, any more than Ptolemaic tradition made the sun revolve around the earth.

Sessions may think tradition suffices, but law requires expert scientific testimony to be credible. There are two basic tests, Frye and Daubert. The Frye test (from a case in 1923) asks if experts reached their conclusions using “generally accepted” techniques, though this is less an inquiry about accuracy than consensus.⁴⁵ (Galileo would not have survived Frye.) Some states still use a modified Frye test, but many state, and all federal, courts have moved on to the Daubert test, articulated in 1993. Daubert’s central inquiry is whether a method is “reliable,” though this immediately causes conceptual problems, because in science “reliability” just means the ability to produce the same result over and over.⁴⁶ An otherwise perfect watch, set five minutes fast, is “reliable” and wrong. Daubert’s use of “reliable” irritates some scientists, but the rest of the opinion shows that what the Court was really interested in was validity/accuracy.⁴⁷ To that end, it suggested various indicia including testability, peer review, quality control, error rate, and the old Frye standard of general acceptance. Lower courts generally content themselves to apply these criteria mechanically, while missing Daubert’s spirit, which is: Could a given method be wrong? That requires independent thought every time, taking new research into account, but once a method has passed Daubert in one court, it tends to be grandfathered in without too much attention paid to subsequent developments. The cult of history and stare decisis strikes again, and Daubert turns into Frye, but with general acceptance determined by legal precedent, not science.

What kinds of forensic evidence, beyond fingerprinting, might fail a rigorous application of Daubert? Quite a lot. Bullet lead analysis, despite its known faults, was used for over a decade after Daubert, though it was finally abandoned in 2004. But other questionable methods persist: hair and fiber analysis, forensic dentistry, some blood tests, tread imprints, paint tests, etc. All these staples of TV and court have all fallen into serious doubt due to unacceptable error rates, poor quality control, lack of rigorous peer review, and the difficulty (or impossibility) of replicating results. Essentially all information, as we’ve seen, is relevant, so these methods can add something (especially at the detection stage), but because they carry the patina of rigorous science, they may unduly sway a jury—and judges should consider excluding them on grounds of prejudice.⁴⁸

Safeguards: Cross-Examination, Confrontation, and Judicial Wisdom

The adversarial system is the core protection against dubious testimony, supplemented by judges’ occasional and unprompted interventions. Adversaries can cross-examine witnesses, including the expert witnesses who present forensic evidence, to undermine credibility. Parties can also present their own witnesses to offer rebuttal, though many criminal and non-corporate litigants cannot afford to retain experts who command the same aura as lab techs from Quantico. The natural solution, advocated by ex–federal appellate judge Richard Posner, is for courts to exercise their existing powers to appoint neutral experts. Doing so would partly alleviate considerations of expense and also mute questions about bought testimony. (After all, parties hire experts to supply helpful conclusions, which are usually forthcoming.)

Parties can also employ the FRE to exclude evidence. The rules of hearsay, relevance, competence, and so on, all provide mechanisms to combat testimony, though as we’ve seen, these rules are so broad that they are often useless and, anyway, it’s difficult to unring the bell if objections arrive after live testimony has been offered. Lawyers may also ask for evidence to be excluded when its “probative value is substantially outweighed” by dangers that evidence is too confusing, repetitive, dilatory, could “mislead[] the jury” (lacking practical options, the rule presumes the judge is immune to being misled), or if the evidence is “unfair[ly]” prejudicial.⁴⁹ There’s a certain reticence to invoke the prejudice rule outside a few well-trod areas, though again, judges face little risk of being reversed.⁵⁰

The prejudice rule is subjective and not a perfect protection. For a start, the rule allows some prejudice (“mere” prejudice is not enough), and perhaps even a lot of prejudice, given that the rule weighs importance against effect. That requires two determinations, of evidence’s value and its potential to taint, determinations that occur in a partial vacuum, as there’s no way to know before the evidence is introduced. Again, judges’ mental models shape cases, potentially taking the effective decision out of the jury’s hands.^* The law here is spotty, though the general thinking is that if judges believe evidence could overwhelm a jury’s rational faculties, they should seriously consider exclusion. Especially vivid crime scene photos, evidence of gang membership, or one party’s extraordinary wealth, have sometimes been excluded on grounds of prejudice.

Prejudice involves more than just vivid personal characteristics. Forensics and experts might seem to be steps toward the objective (and thus anti-prejudicial), but they seriously implicate the evidentiary rules’ screen for confusing, prejudicial, and otherwise misleading evidence. Science has considerable mystique, which experts practicing quasi- or even junk science can borrow. The law here is a mess, with various evidentiary rules (and Daubert) blending together, but there’s a strong argument that, aside from good DNA evidence, a lot of forensic testimony should be spiked by judges before a jury hears any of it.

Curiously, one type of expert testimony which probably should be permissible is often excluded: testimony by social scientists and psychologists about the unreliability of witnesses. Many federal courts are hostile to experts-on-witnesses, and the Eleventh Circuit bans them outright, on the belief that jurors understand the frailties of human testimony as matter of “common sense” and do not require expert help. That’s ridiculous, because while jurors understand that witnesses make mistakes, the frequency of mistakes and the circumstances that contribute to them are hardly common knowledge.⁵¹

Functionally, the decision to admit or exclude almost all evidence rests with the trial judge, though the FRE override discretion for certain types of evidence in the interests of the public good. One important example is evidence of remediation. Law does not want to deter people from obtaining insurance, making repairs, or offering to settle cases; therefore, evidence of these events is presumptively excluded for some purposes. In a trip-and-fall, it may be highly relevant that a shopkeeper repaired a sinkhole and offered a giant settlement, but the law draws a partial veil, potentially sacrificing the interests of specific litigants to promote the welfare of society at large.⁵² This might be sound policy, but it’s not motivated by fidelity to fact. And excluding evidence of remediation is not the only policy judgment law makes.

At Play in the Poisonous Orchard

Legal, social, and evidentiary principles become highly entangled when evidence in criminal cases has been obtained by unconstitutional means. In 1914, the Court held that evidence gathered in violation of the Fourth Amendment could not be introduced, establishing a strong “exclusionary rule,” and subsequent cases applied that rule to violations of the Fifth and Sixth Amendments and required the states, originally unbound by the federal rule, to do the same.⁵³ The “fruit of the poisonous tree” doctrine extends the exclusionary rule by forbidding evidence that derives from unlawfully obtained evidence.⁵⁴ These rulings provoked fierce protests, because they freed criminals notwithstanding evidence of guilt, but were considered integral to policing the police and maintaining the integrity of the legal system.

The exclusionary rule has holes, which conservative majorities on the Court have been widening for several decades. The Court has granted police “good faith” exceptions in various circumstances, such as reasonable mistakes.⁵⁵ Mistakes are easy to fake, and officers have good reasons to fake them, as the “good faith” exception comes into play precisely because there’s been a defective search (making bad faith a plausible assumption). The exclusionary rule also does not apply in civil trials, grand jury proceedings, and probation hearings. Furthermore, while tainted evidence can’t be used in a criminal trial to prove guilt, it can be employed to impeach witnesses before a jury, opening a considerable back door.

The Court keeps tearing new holes in the exclusionary rule, claiming that it is a joy-killing prophylactic from a bygone era of widespread police abuses and limited civil remedies. The Court offered no support for its police-are-better-now sociology (which, in a different context, would fail the Daubert test). As for the expansion of civil remedies, the Court misreads its own work: we’ve repeatedly seen that the Court has been expanding immunity doctrines, weakening the very sanctions that the Court says provide deterrence and redress.

Today, the Court doesn’t concern itself as much with the absolute rights and wrongs of a police search, but proffers a pseudo-pragmatic balancing test, asking whether “police conduct [is] sufficiently deliberate that exclusion can meaningfully deter it, and sufficiently culpable that such deterrence is worth the price paid by the justice system.”⁵⁶ These exclusionary conditions might be satisfied, the Court suggested, if police acted deliberately, recklessly, or with gross negligence, something rather different than the old “good faith” test. It now appears that police can harvest poisonous fruits all day long, so long as their acts are merely negligent, and/or the products of unfixable incompetence or genuine disinterest. Worse, even if one enforcement agency collects evidence by means sufficient to trigger exclusion, it appears other law enforcement agencies can leverage that same information to build their own cases, creating yet another complexity.⁵⁷

The exclusionary rule has never been solely about vindicating the Constitutional rights of defendants; it also protects the broader integrities of the Constitutional system, the judiciary, and the police. Perhaps the Court is willing to barter some of these values, but the Court should ask if it’s selling legal integrity too cheaply. First, the Court cannot go lower than the floor set by the Constitution—nothing can be sold for less than the price of due process, a right implicated by defective searches. From there, it’s an upward negotiation. When police ignore the Court’s own jurisprudence on the Fourth, Fifth, and Sixth Amendments, they undermine the judiciary—and the Court should charge a premium for that. A strong exclusionary rule is important, not just for one defendant’s sake, but for the judiciary’s.

The Right to Confront

The Constitution provides another protection against questionable evidence: the Sixth Amendment’s promise that a criminal defendant has the right to “be confronted with the witnesses against him.”⁵⁸ The Confrontation Clause exists to allow cross-examination, which is supposed to be an essential advantage of the common law over the inquisitorial system. This should be one right that a proud common-law system defends to the hilt.

Unfortunately, the word “witness” is not simple—as we saw with hearsay, there are clearly cases in which the legal witness, the person on the stand, is merely a mouthpiece for the true witness, the person who saw what was going on. Direct witnesses are often unavailable, meaning that a strict interpretation of the Sixth Amendment might allow defendants to get away, or even create incentives to murder witnesses. So in 1980, the Court held that unconfronted/indirect testimony can be admitted if it comes with sufficient “indicia of reliability,” such as falling within established hearsay exceptions (which, as we’ve seen, are frequently rubbish as indicators of reliability) or if there are other guarantees of “reliability.”⁵⁹ Don’t worry if that judicial logic seems murky; it was.

In 2004–2006, the Court tried to clean up its mess. To do this required understanding what a Sixth Amendment “witness” really is—or more precisely, was. So, the Court fired up the TARDIS for another of its Dr. Who adventures, and it’s worth following the story, which illustrates how the Court arrives at many of its more ridiculous conclusions. In one witness-confrontation case, the majority, led by Scalia, tumbled out of the TARDIS to retrieve Webster’s Dictionary of 1828, but of course the right date is 1791, when the Sixth Amendment came into legal being. But, you know, close enough. And what Scalia’s majority concluded was that a “witness” is someone who offers “testimony,” and “testimony” is information offered in solemn conditions that are, or seem, preparatory to legal proceedings, such as a formal interrogation initiated by police—but not, say, statements made while the police are responding to an emergency call. In other words, you have a right to confront whoever is on the stand (the officer), but not necessarily the person who actually possessed the information in the first place (the victim, say).⁶⁰ Now, this understanding of “witness” is generally consistent with the Dictionary’s fifth entry, though the third definition says that a “witness” is just someone who saw something, and Scalia’s gloss is arguably in tension with the second entry, which just speaks to “that which furnishes evidence or proof.”⁶¹ In other words, the Dictionary doesn’t really support Scalia’s conclusion, it’s a prop. And if you’ve seen Dr. Who, you know the props are cheap and unconvincing.

The Court sweated none of this, declaring that statements not produced with full legal ceremony—oaths, the looming clouds of litigation, police custody, whatever—aren’t quite testimony.⁶² Ergo, witnesses aren’t always really “witnesses,” and thus there’s no Sixth Amendment problem. Hence, the Court condoned the admission of a recording of a 911 call that identified the suspect, noting that the recording was made during an ongoing emergency, not a police interrogation.⁶³ In fact, the call in question was not to 911, but from 911; it was a follow-up inquiry about the identity of the suspect, and thus at least a bit like an investigation.⁶⁴ This opened the gate to many potential abuses, though the Court breezily asserted that we needn’t be bothered by unconfronted, “non-testimonial” hearsay because the Framers weren’t either. But early Americans did worry about hearsay’s frailties. Indeed one part of the hugely famous, hearsay-driven trial of Walter Raleigh (involving non-testimonial hearsay, a detail Scalia overlooked) was a major motivation for confrontation protections.⁶⁵

Enough with Dr. Who jurisprudence. The essential point of the Confrontation Clause is that the accused must be able to test his accusers wherever possible; our rights here should be clear and robust. They are not, and this is a real problem in the age of forensic evidence, most of which is presented by testifying experts. But an expert is an intermediary, operating as a sort of Vanna White by turning letters to reveal information that other people have produced—and surely the accused should be able to test those other people, his true accusers. The Court has struggled with such cases, throwing trial judges into disarray. As Justice Kagan put it, the state of the law here “is—to be frank—who knows what.”⁶⁶ We might or might not have a right to confront. It just depends on which Justice you ask, what dictionary he has, and what entry he flips to.

Hodgepodge Exceptions

The evidentiary rules are filled with many other gates and back doors, but only two need detain us. The first includes testimony about character, criminal history, and plea arrangements. The general rule is that these matters are too inflammatory and biased, and forbids them as direct evidence, but there are many ways to sneak in this kind of information, through witness impeachment and so on. The particular danger is that merely entering a plea discussion opens doors that cannot be easily closed, and given overcharging and prosecutorial browbeating, plea negotiations by uninformed defendants can create damning evidence for later use.⁶⁷ As prosecutors strongly prefer defendants to discuss and plead, this leaves defendants seriously compromised. For the most part, actual convictions can be excluded, save for sex crimes. Still, what law purports to exclude can often be smuggled into a criminal prosecution via other means. And in civil cases, the rules are even more porous.⁶⁸

The second category of exclusion is based on privileges. The most important of these is attorney-client privilege, but it’s not a blanket protection. The client owns the privilege, and only communications seeking legal advice are protected; simply cc’ing a lawyer does not provide a shield (though addressing “any thoughts?” to the cc’d lawyer might). However, soliciting advice about how to commit a crime or fraud falls outside the privilege, so a client cannot ask a lawyer how to carry out a murder or ask for help covering up a past crime. The bounds can be blurry, as substantively similar matters can fall into, or out of, the privilege depending on phrasing: if a landlord wants to evict a tenant, asking a lawyer how to do it lawfully or even asking if extreme harassment is lawful will be privileged, but asking for practical tips to evict a tenant “by any means” treads well into a gray area. Normally, careful phrasing can keep clients within the bounds of the privilege, though recent cases, involving certain famous lawyers and clients, show that privilege isn’t foolproof. The privilege also opens up somewhat in fee disputes between attorneys and clients, so it’s good to keep current on legal bills. Federal law also enshrines the marital privilege, and one spouse cannot be forced to testify against another, but this privilege can be waived by the witness-spouse; it’s not absolute.⁶⁹

Federal law generally does not recognize other privileges.⁷⁰ Other federal privileges exist mainly in popular imagination—journalists and their sources enjoy no absolute, evidentiary privilege, nor do priests and penitents. Some states offer additional privileges, for example shielding doctor-patient discussion, though often subject to considerable limitations. As for executive privilege, courts avoid that issue whenever they can, but executive privilege is not absolute, as every president since Nixon has been reminded.

Juries and Inquisitors

A major theoretical purpose of evidentiary rules is to ensure that that questions of liability and culpability are decided only on proper evidence. With almost all cases decided before trial, it may not seem like the rules of evidence have a real legal life, with a few exceptions like the exclusionary rule. But the rules do matter; litigation is always a possibility, and pretrial proceedings, which involve evidentiary disputes under the rules, influence many settlements. Losing a major pretrial evidentiary motion may by itself prompt a settlement. The rules adjust probabilities, and even if the rules embody no coherent theory, their effects are real enough.

By now, we can take it as given that the laws of evidence are largely detached from pure fact, so the final questions are whether we can or should dispense with most evidentiary rules. As a society, we’d clearly prefer to retain attorney-client privilege and Constitutional protections, but these protections were imported into the rules of evidence and would survive even if the FRE vanished. But what about hearsay and the rest? One temptation is to dump the rules entirely, with the standard rhetoric pointing to the civil law tradition, blissfully unencumbered by a complex, incoherent evidentiary apparatus.^* The two systems have actually converged in recent years, in part because of European integration and legal reform, but civil and common law still retain different approaches to law, as their structures require. For example, in the civil tradition, when serious crimes occur, judicial police investigate, not regular police; the exclusionary rule has less practical importance. Depending on the civil law jurisdiction, a prosecutor might be assigned to advance the case, but generally, an investigating judge oversees the inquiry, interviews witnesses, visits crime scenes, and performs other tasks unthinkable to American judges. The investigating judge then compiles a dossier, turned over to the “sitting judges,” who control the calling of witnesses and experts and generally run the show. The parties do have lawyers, but they’re generally passive, and cannot choose, communicate with, coach, or directly question witnesses in the same freewheeling way that characterizes common-law practice.

Civil-law judges also try to resolve cases on the narrowest grounds, so rather than the common law’s Tolstoyan survey of life stories, proceedings focus on whatever is most germane to a verdict. Contrary to general belief, some civil-law countries do have lay jurors as part of the judging panel, but there’s always a professional judge to keep everyone on track. Judges will exclude certain evidence on constitutional and other grounds, especially now that EU treaties have been incorporated into members’ legal regimes, and some countries have strict rules about witness testimony (parties are frequently deemed irretrievably biased and therefore excluded as witnesses). The overall expectation, though, is that the judges will flip through the dossier, ask questions, and weigh evidence appropriately—without lawyers dickering over weird matters like excited utterances. The whole thing is so technocratic and placid that it defies even televisual fantasy. The French, lacking a system conducive to courtroom drama, had to import American TV shows, which distorted French imaginations rather badly (apparently, people started addressing judges as “votre honneur” instead of the correct “madame la présidente/le juge”) and the Ministry of Justice begged for home-grown dramas—the success of which you can test with a Netflix search.^* The civil system reposes substantial and coherent faith in judges to parse evidence wisely, which does not mean that judges grant all evidence equal (or sometimes any) weight, only that with judges in control, elaborate rules become less necessary.

Evidence in common-law countries reflects a less coherent view about judges and fact-finders. American law presumes a lone judge can rise above any prejudice—it perhaps has greater faith here than the civil systems, in which core investigation is carried out by one set of judges and decisions are rendered by a panel (usually a different one). By contrast, American judges see whatever the parties drag before them, and can only will themselves to unknow forbidden evidence. The American attitude toward juries is somewhat contradictory, with lay juries viewed as a bulwark against tyranny, yet too childlike to handle raw evidence. This logical divide may be irrelevant, as the most critical matters, including the exclusionary rule, are handled away from the jury. For evidentiary rulings made in court, it seems that juries can’t quite forget what they’ve heard. The trial proceeds anyway, unless the prejudice is “gross.”

Almost all evidence is relevant, but not all of it is credible, and American law picks and chooses so strangely that the whole thing seems ad hoc, the product of historical accident. That’s precisely what it is; for the most part, the FRE embody common-law doctrines developed centuries ago, often across the sea. The common law was supposed to be wise and case-specific; statutes are meant to be clear and broadly applicable. The FRE, and its state equivalents, are an unpersuasive hybrid. And so it is on an uncertain basis that we are judged, and in losing cases, punished.