4.
Blind Bias
At a surface level, we are all aware that our preconceived biases affect how we view the world. In 2000, I remember being with a good friend, who happened to be on the opposite end of the political spectrum, while we watched the presidential debate between Al Gore and George W. Bush. I thought that, without a doubt, my candidate appeared more presidential, intelligent, and honest, whereas it was obvious that my friend’s candidate was being evasive. By the end, I knew that even my friend would have to admit that his candidate had hardly answered a single question, and when he did, it was with a jargon-filled catchphrase meant to appeal to his base. There was no substance there.
But when we discussed the debate afterward, my friend fervently thought the opposite. My candidate, he said, hadn’t answered a single question all night; whereas his candidate addressed the problems head-on, gave details and specifics, and appeared more presidential. Our contrasting reactions reflected national surveys. Democrats thought Gore appeared more presidential and honest, and gave more specific, concrete answers, while Republicans thought the same about Bush. The American public, my friend and I included, saw what we wanted to see. We had preconceived biases, and we focused on only those pieces of evidence that confirmed them. Evidence that contradicted our biases did not register.
This phenomenon is known as “confirmation bias.” The psychological definition of confirmation bias is “the seeking or interpreting of evidence in ways that are partial to existing beliefs, expectations, or a hypothesis in hand.”1
Human susceptibility to confirmation bias is pervasive in our lives and society, and has been demonstrated in studies time and time again since the concept was first introduced into the psychological literature more than fifty years ago. In one study, for example, subjects watching a videotape of a child taking an exam who were able to see the answers provided by the child were far more likely to rate the child’s answers as “above grade level” when told beforehand that the child was from a high socioeconomic background, and “below grade level” when told the child was from a low socioeconomic background.2
In another study, therapists watched a videotape of a man responding to personal questions asked by another person. When the therapists were told that the man was a job applicant who was responding to these questions as part of a job interview, they tended to label him as “well-adjusted.” But when told he was a “patient,” thus implying he was seeking psychological help, they were more likely to label him as “disturbed.” Even though the two groups watched the same videotape, those who believed they were observing a job applicant described him with terms such as “responsible,” “bright,” “pleasant,” and “attractive,” while those who believed they were watching a “patient” described him with adjectives such as “defensive,” “passive-aggressive,” “repressed,” and trying “to seem brighter than he is.”3
Studies show that when people are given evidence that is contrary to a theory they support, they tend to stick to their beliefs despite the evidence.4 Interestingly, when they are later questioned about the contradictory evidence, they fail to remember it, or they remember it in a distorted manner that merely confirms their preexisting beliefs.5 Another study showed that jurors tend to make up their minds about a case fairly early on in a trial, and then, after trial, remember only those pieces of evidence that confirmed their initial instincts. They are unable to recount the conflicting pieces of evidence, or they remember the evidence in an inaccurate manner that simply confirms their initial beliefs.6
In a nutshell, studies reveal that the human mind is wired to confirm rather than disprove its initial suspicions. One of the first clinical studies designed to explore confirmation bias, known as the Wason Rule Discovery Test, after its creator, Dr. Peter Wason, asked subjects to identify the rule that applied to a series of three numbers.7 They were given the series 2-4-6 as the starting point. To decipher the rule and test their hypothesis, subjects could suggest other number sets; they were told “yes” or “no” as to whether their number sets followed or violated the rule. When the subjects were confident that they had discovered the rule, they were told to stop and announce it.
Given the numbers 2-4-6, most subjects came up with the same initial hypothesis: that the rule was a series of increasing even numbers. Thus, the vast majority of subjects suggested triplets of numbers that complied with that rule, such as 4-8-10 or 2-6-8, and were told that those suggestions complied with the rule. After a few attempts to confirm their initial hypothesis, and after receiving positive feedback, most subjects announced that they were confident they had discovered the rule—a series of increasing even numbers. But they were wrong. The rule was simply “a series of increasing numbers.” Thus, odd-numbered series, like 3-5-7, or sets mixed with increasing odd and even numbers, would have also satisfied the rule. But very few subjects attempted to disprove their first hypothesis. Odd numbers did not occur to them, given their initial hypothesis. They set out to confirm the hypothesis, and thus obvious and simple number sets like 3-5-7 that might disprove their hypothesis simply did not register as possibilities.
Confirmed in various contexts, Wason’s Rule Discovery Test stands today for the widely accepted psychological phenomenon that the human mind works only to confirm its initial suspicions. It does not set out to falsify its own theories and thus does not explore contradictory avenues that, with hindsight, should have been fairly obvious.
The mind’s vulnerability to confirmation bias has been so widely studied in a variety of contexts that even the Wikipedia definition of the term references nearly 150 studies confirming its existence in many different contexts. The prominent psychologist Raymond Nickerson of Tufts University, founder of the Journal of Experimental Psychology, has written that confirmation bias is so “sufficiently strong and pervasive that one is led to wonder whether the bias, by itself, might account for a significant fraction of disputes, altercations, and misunderstandings that occur among individuals, groups and nations.”8 He writes:
Can anyone doubt that whenever one finds oneself engaged in a verbal dispute [confirmation bias] becomes very strong indeed? In the heat of an argument people are seldom motivated to consider objectively whatever evidence can be brought to bear on the issue under contention. One’s aim is to win and the way to do that is to make the strongest possible case for one’s own position while countering, discounting, or simply ignoring any evidence that might be brought against it. And what is true of one disputant is generally true of the other, which is why so few disputes are clearly won or lost. The more likely outcome is the claim of victory by each party and an accusation of recalcitrance on the part of one’s opponent.9
Nickerson goes on to discuss how confirmation bias infiltrates not just our personal lives but also various fields, including medicine, science, and law. It has contributed to societal problems ranging from the wholesale rejection of compelling new ideas (like Galileo’s theories or groundbreaking concepts like continental drift) to great tragedies like the Salem witch trials and the Holocaust.
• • •
This book cannot tackle all the ways that confirmation bias affects us unnoticed in our daily lives. But by demonstrating how confirmation bias contaminates the criminal justice system in ways unknown to the general public—and how these biases result in great injustices—one can begin to understand how this phenomenon might wreak havoc in other parts of our lives as well.
Indeed, the criminal justice system is saturated through and through with confirmation bias. For example, detectives tend to believe witnesses who tell them stories that confirm their own beliefs about a case. Witnesses who give contradictory information are seen as lying or mistaken and are often pushed to the wayside.
But perhaps the best way to understand how confirmation bias affects outcomes in the criminal justice system is to study how it operates in the forensic sciences. Crime scene investigation (CSI) experts in their white lab coats are held up by our society as bastions of neutrality. Theoretically at least, they have no stake in whether a criminal defendant is convicted or acquitted. They often have fancy degrees and operate in a world of cold, hard, objective science. But once we understand how easily even forensic scientists succumb to confirmation bias, we can begin to see how police officers, prosecutors, defense attorneys, judges, and even witnesses succumb as well, skewing the facts and bringing about grave injustices.
• • •
The discovery of hundreds of wrongful convictions by the innocence movement in the past two decades has exposed deep problems in our forensic sciences. In fact, inaccurate forensic science testimony was a contributing factor in 154 of the first 325 DNA exonerations in the United States, second only to mistaken eyewitness identifications. In other words, 49 percent—nearly half—of those convictions involved bad forensics.10
With this knowledge, social scientists began asking: Why are we getting all these wrongful convictions based on inaccurate scientific testimony? Why did the forensic expert in this case testify that he microscopically compared the defendant’s pubic hair to one left by the rapist at the crime scene and concluded that they matched, when DNA testing of the hairs has now told us that they’re not a match? Why do we keep seeing flawed scientific testimony in case after case?
When an innocent person is wrongfully convicted as a result of flawed forensic testimony, one of two possible phenomena, or both, may be at play. First, some of our forensic sciences are now known to be nothing more than theories or speculation without sufficient grounding in actual science or scientific method. In other words, the method used to analyze the evidence and reach a conclusion is untested with no known accuracy rate. When this problem contaminates a case, it was not necessarily the expert who testified that was the problem; rather, it was the weaknesses in the method she was trained to use. The second phenomenon that can lead to flawed scientific testimony is the confirmation bias of the expert who analyzed the evidence. This bias can skew the results in a case even if the scientific principles underlying the expert’s conclusions are generally sound.
Contrary to popular belief, many of the “CSI sciences” that experts describe in court are not based on objective, hard facts immune to manipulation or misinterpretation. Instead, experts typically analyze patterns, or compare images, to determine if the pattern or image matches or is “consistent with” something else. Does the defendant’s tire tread match the tire tread left in the mud when the perpetrator peeled out and fled from the robbery? Do the microscopic grooves on the bullet found at the crime scene match imperfections in the barrel of the suspect’s gun? Were the drops of blood found on the suspect’s clothing consistent with examples of high-velocity blood spatter? Does the handwriting on the demand note used in a bank robbery match samples of the defendant’s handwriting? Do any of the suspect’s fingerprints match the fingerprint found on the bloody knife? Does the bite mark on the victim’s body match the plaster impression of the suspect’s teeth?
In other words, many forensic sciences are subjective processes of comparing and matching rather than purely objective exercises. That is where human error can enter the equation. And in the wake of the innocence movement, with the scads of cases uncovered in which experts have given inaccurate courtroom testimony that led to a wrongful conviction, social scientists have performed a variety of experiments to see if and how confirmation bias of the expert might skew results and lead to these injustices.
Dr. Itiel Dror, a leading researcher from the University College London, has been at the forefront of this research. In 2005, Dror and his colleagues performed an experiment using five experienced fingerprint experts.11 They asked the experts to make a comparison between two prints, one of which belonged to a suspect and the other of which came from a crime scene. They informed the experts that the prints were from a case in which many now believed that the fingerprint expert had made a mistake and incorrectly concluded that the prints were a match, leading to a wrongful conviction. Thus, Dror provided the fingerprint experts with background and context in an attempt to create a preexisting bias that the fingerprints likely would not match. Dror told the experts, however, to ignore this contextual information and to perform an independent examination to determine for themselves whether the fingerprints matched or not.
Unbeknownst to the fingerprint examiners, Dror and his colleagues did not actually supply the experts with a set of fingerprints from another case where an error had likely occurred, but instead provided each with a set of fingerprints that that particular expert had previously concluded was a match in a prior case. (Fingerprint experts look at thousands of fingerprints each year, and likely cannot remember a set of fingerprints they examined even a week before.) Although each of the experts was unknowingly examining a set of prints that he or she had personally called a match in a prior case, four of the five reversed their conclusions and now gave a different answer. Three of the five determined that the fingerprints did not match, while one said that the answer was inconclusive. Only one fingerprint expert stuck to his original conclusion of a match. Thus, 80 percent of the fingerprint experts changed their answer as a result of being given a context that created a preexisting belief about what the right answer should be.
A follow-up study involved forty-eight different trial samples.12 It demonstrated once again that fingerprint experts were highly susceptible to suggestion of either a match or a nonmatch depending on the context provided (for example, “he confessed to the crime” vs. “he has a strong alibi”). And confirmation bias swayed the results even in fairly obvious cases where there were strong similarities or discrepancies between the fingerprints.
This is shocking from a forensic science that has long held itself up as “infallible,”13 and boasted that errors are “virtually impossible.”14 These studies show how subjective the process really is and how easily fingerprint experts can produce flawed results that are skewed by their preexisting beliefs about what the “right” answer is supposed to be. But Dr. Dror and his colleagues have gone beyond the world of fingerprints, revealing the presence of confirmation bias in areas such as anthropology (determining the gender of skeletal bones),15 handwriting analysis,16 and DNA test result analysis,17 among others.
• • •
The high-profile case of Brandon Mayfield provides a perfect example of how this confirmation bias plays out in the criminal justice system.18 In 2004, a series of bombs exploded in four commuter trains in Madrid, Spain. The authorities immediately suspected a terrorist attack, perhaps by Al Qaeda, which spurred an international investigation, including heavy involvement by the FBI. A fingerprint was found on a plastic bag that had contained explosives used in the attack. The FBI ran the print in the AFIS database (which includes fingerprints taken from arrestees in the United States), and the computer kicked back several fingerprints from different Americans as possible matches. The FBI then examined those fingerprints more closely to see if one of them was a perfect match to the one retrieved from the bomb packaging. One of the individuals connected to a fingerprint on the list immediately raised serious suspicions. He was Brandon Mayfield, a lawyer from Oregon, who was on an FBI terrorism watch list in part because he was married to a Muslim Egyptian national and had recently converted to Islam himself. With the belief that Mayfield was by far the best suspect of the group, a series of FBI fingerprint examiners independently compared Mayfield’s print to the print from the bombing and concluded that it was a definitive match. Mayfield’s fingerprint was on the bomb package. No question. So Mayfield was arrested.
After his arrest, Mayfield asserted his innocence and had a defense-hired expert examine the fingerprints, too. Mayfield’s expert confirmed what the FBI already knew—it was without question Mayfield’s print on the bomb package.
Two weeks later, Spanish authorities linked the fingerprint in question to an Algerian national named Ouhnane Daoud. The FBI ultimately conceded that the fingerprint did not match Mayfield. He was released from jail, awarded $2 million in compensation from the government, and received a formal, written apology.
Later, the FBI acknowledged that confirmation bias played a role in the misidentifications. Indeed, three different FBI fingerprint examiners—and Mayfield’s own defense expert—may have succumbed to the bias and erroneously declared a match as a result. This happened despite clear discrepancies between Mayfield’s print and the bomber’s, ones obvious to the Spanish experts.19 The biases of the American experts did not allow them to see those discrepancies. The experts saw only what they expected to see, and their minds did not register anything else.
Please take about ten seconds to complete the following test (without looking at the answer below).
How many letter Fs are in the following text?
FINISHED FILES ARE THE RE
SULT OF YEARS OF SCIENTI
FIC STUDY COMBINED WITH
THE EXPERIENCE OF YEARS . . .
If you answered six, you are in the minority who got it right.20 Typically, about 75 percent of people in this experiment answer three, leaving out the Fs that appear in the three uses of the word “of.” Why do most answer three? Because the mind is so accomplished at reading that it no longer needs to analyze or register the word “of.” The mind has deemed it a waste of time, so it creates a shortcut, causing most readers to miss those three Fs.
By analogy, when experts—or anyone for that matter—expect a particular result, their minds naturally create shortcuts to help them see only what they need to see. There were many discrepancies between Mayfield’s fingerprint and the bomber’s, but the experts did not see them or register them because of confirmation bias. Likewise, Dror’s experiments demonstrate how common it is for expert witnesses to miss discrepancies and call two fingerprints a match when confirmation bias tells their minds not to see the differences.
• • •
But fingerprint comparisons, unscientific though they are, are actually far less subjective and open to interpretation than many of our other forensic sciences. Bite mark comparisons (comparing a bite in the victim’s skin left by the perpetrator to the suspect’s dental impression), microscopic hair comparisons, tire tread comparisons, ballistic fingerprinting, blood spatter analysis, and a host of other forensic disciplines are equally, if not more, subject to skewed results from confirmation bias.
The problem is compounded by the fact that, when making such determinations, experts are usually not working with clear, easy-to-read images. While a plaster cast of the suspect’s tire tread made in a police impound lot may be fairly clear, rarely is a tread left by the perpetrator’s car when peeling out and away from a crime scene clean and easy to decipher. It may be filled with gravel, distorted from the sideways spinning of the car, or left in wet mud that immediately began to change shape from gravity and moisture. While a suspect leaves clear fingerprint images in a police station booking room, where an officer dips his fingers in ink and carefully rolls them on a fingerprint card sitting on a stable surface such as a desk, murderers rarely dip their fingers evenly in blood and carefully roll each finger across the knife they just used to murder someone.
The plaster dental impression made of a suspect’s teeth by a forensic odontologist is often an accurate reconstruction of the suspect’s teeth. But a bite mark on a victim’s body, often made through layers of clothing, is created while significant movement and sliding are likely occurring during a struggle, and on a surface—skin—that has varying degrees of elasticity and is overall a poor substance on which to record and retain an accurate image.
To demonstrate this point, perform the following experiment. Turn your arm so that the inside is facing you (so you can see the blue veins in your wrist). With your other hand, place your thumb and index finger in pinching position, one of them on either side of your arm just below the elbow (the side of the elbow toward your hand). Now squeeze the skin toward the middle of your arm and examine the amount of skin you are able to pinch. Next, attempt to gather skin from your arm in the opposite direction. Place your index finger toward your hand and your thumb toward your elbow, and attempt to pinch inward. You will see that it is nearly impossible to grasp much skin when pinching in this direction. This is because skin, in different parts of your body, is elastic in some directions but not in others.
Thus, parts of a curved bite might impact skin that is quite elastic and moves from that pressure when the bite takes place, while another part of the same bite may impact skin that holds firm. The result is a distorted image that does not accurately reflect the position of the biter’s teeth. This is just one factor, in addition to movement and sliding from the likely struggle and the layers of clothing in between the teeth and the skin, that may cause a bite mark on a body to differ significantly from the shape of the biter’s teeth.
In sum, experts often make comparisons by examining one clean image—the suspect’s image taken at a police department under controlled conditions—against a distorted or smeared image left at the crime scene. This opens the door even more to human interpretation, confirmation bias, and error. Evidence examined by forensic scientists is not cold, hard data immune to human interpretation. Rather, it is somewhat like a Rorschach inkblot. Different people see the evidence in different ways, and what they see in an image can be skewed by what they expect to see.
• • •
In my days as a prosecutor, we routinely told experts examining our evidence what outcome we needed. That may seem shocking to some readers, but it was run-of-the-mill for us. If it were a ballistics test, we might say, “Confirm that the bullets came from the defendant’s gun,” or, “We believe these are from the defendant’s gun, so please run the test to confirm.” When we used the word “defendant,” it meant that he was already arrested and we were trying to solidify our case. Even if the word “suspect” was used, it was, in hindsight, a loaded term that told the expert that we had gathered enough evidence to move forward, and that he should expect a match.
Typically, we even supplied the experts with a likely trial date and corresponding deadline before trial by which we needed their written report. This sent the message, of course, that we thought this defendant did it (and that we had enough evidence already to have made an arrest and gotten an indictment from the grand jury), but that we needed more evidence to nail him.
Some experts developed reputations in my office for being “good,” meaning that they gave you the results you wanted. I prosecuted many white-collar crime cases, which usually involved a lot of documents, and thus I had a frequent need for a handwriting expert. The first time I needed one, I did what everyone in the office did—I asked around the office for recommendations. I received several. One prosecutor gave me the name and number of an expert and told me, “He always gets back to you quickly and he’s not a wimp—he’s not afraid to call a match a match.”
I used this particular handwriting expert numerous times in my prosecutorial career. On one occasion, I needed droves of documents that I believed the defendant had signed to be analyzed in a very short period of time. The trial was starting, but I did not receive these documents until the eve of trial. I shipped the documents to the expert overnight and asked him if he could give me an answer on them as quickly as possible, because my trial was starting. I also sent the defendant’s handwriting exemplars (samples of his signature that were not disputed), so the expert could make the appropriate comparisons.
Two days later, I got a package back in the mail from him. He confirmed that all of the signatures had been made by my defendant. Given that it would have taken him overnight to ship the package back to me, it’s clear that he had analyzed all of the signatures—perhaps hundreds of them—in one day.
In my prosecutor’s office, we not only routinely told forensic scientists what answer we sought from their analysis and the time by which we needed it, but in some instances indicated that it was important because a “match” would be crucial to getting a conviction. This no doubt put great pressure on the forensic scientists to confirm our theories of the case, and created strong confirmation biases. In the example above, I told the handwriting expert that I needed him to confirm that the signatures on all the documents matched the defendant’s, and I made clear that this was important because trial was starting and I needed the answers ASAP.
Experts who were too technical, and too worried about nuance rather than the big picture (the big picture of course being that a match existed between the defendant and the crime scene evidence because the defendant was clearly guilty), were considered an aggravation, too academic, and prone to miss the forest for the trees.
At the time, I, and I believe others in my office, felt that there was nothing wrong with this practice. I was completely unaware of the effects of confirmation bias, as most prosecutors and police officers remain equally ignorant today. And we all believed that our defendants were 100 percent guilty. We knew that the signatures were made by the defendant even before we had an expert examine them. And in a ballistics case we believed that of course the bullet came from the defendant’s gun. Perhaps in many cases we believed this simply because we were correct. But perhaps we also had our own ingrained confirmation biases and suffered from tunnel vision. My bias explained why I did not question how the handwriting expert could have possibly been careful when he analyzed hundreds of signatures in a single day. It was a formality. Experts were there to confirm what we already knew, and to provide a form of evidence that would convince the jury.
But it was not just my prosecutor’s office that created confirmation bias in expert witnesses. Figure 2 shows a actual submission form sent in 2008 from a detective to an expert witness requesting forensic analysis.21
FIGURE 2. Form submitted by detective to forensic expert in 2008 for forensic testing. Received by author from Dr. Itiel Dror, which he obtained from a case file during his research. Reproduced with permission of Dr. Dror.
Note that in the “Summary of Case” in the middle of the document, the detective effectively told the expert that he should expect to find the suspect’s fingerprint because the suspect “reportedly drove” the car prior to the shooting and others in the car had already been eliminated as sources of the fingerprints. In the “Remarks” section toward the bottom, the detective tells the expert that the suspect is a murderer (he “pulled the trigger”), then puts great pressure on the expert, who is a fellow state employee, by saying that she is “making every effort to place [the suspect] in the truck,” and that a fingerprint match from the expert is crucial because the “one witness riding in the truck was too drunk to make an identification.” In other words, the detective is telling the expert that he should expect to find a match and that this is very important for nailing a murderer. This intake form is not an aberration. It is typical of how we communicated with experts in my office, and I believe typical of how it is done today all across the country.22 One-third of forensic scientists in the UK who responded to a 2012 survey from New Scientist magazine admitted that they have felt pressured by law enforcement to reach results helpful to the prosecution in their cases.23 In my opinion, if one-third admit to being pressured by law enforcement, the actual percentage who have actually been pressured is likely higher, as this is something that many would be loath to admit. And of course, short of outright pressure, confirmation bias is still created whenever a forensic scientist is told the expected “right” answer before he or she begins their analysis.
In the Netflix docu-series Making a Murderer, which America binged-watched in 2015 and 2016, several examples of this phenomenon were clearly depicted. In that case, Steven Avery had been charged with the murder of Teresa Halbach. The police believed that Avery killed her in his garage, where they found a bullet on the garage floor. On cross-examination, it was revealed that when the police sent the bullet to a state lab for DNA testing they told the lab, “We need you to place Teresa Halbach in Steven Avery’s garage.” In other words, the DNA lab technicians were told even before commencing their testing that the “right” answer would be an opinion that Teresa Halbach’s DNA was on the bullet. Further cross-examination revealed that even though the results of the test should have been ruled inconclusive under the lab’s own protocol, the lab technicians went against that protocol to render an opinion that Halbach’s DNA was on the bullet.
Another dispute in the case was whether Halbach’s bones, found near Avery’s home, had been moved to that location after they were burned. If so, it would support Avery’s theory that someone had framed him. The prosecution put a forensic anthropologist on the stand who, it became clear, had been told by the prosecution before she began her assessment what answer she was supposed to give—that the bones hadn’t been moved. And, of course, that was the opinion she ultimately gave in her testimony. However, on cross-examination, the defense was able to show not only that her opinion differed from that of the defense’s expert witness, but that her opinion had no basis in science and, even worse, contradicted common sense. The forensic anthropologist ultimately seemed to back off from her opinion on cross-examination, admitting, once its fallacies were exposed, that she couldn’t really give an opinion either way as to whether the bones had been moved.
In Making a Murderer, the way the forensic experts had been preconditioned by the prosecution to give answers that confirmed the prosecution’s theory of the case brought to the public’s consciousness what those of us in the innocence movement had known for years: that confirmation bias is a problem in our criminal justice system, and that it affects even supposedly neutral scientists.
In some jurisdictions, it gets even worse—lab technicians are not only told the “right” answer before they begin, but they have a financial incentive to provide that answer to law enforcement. Indeed, some jurisdictions provide the lab with an additional fee—taken out of the defendant’s court costs!—which kicks in only if the defendant is convicted.24 In others, forensic scientists receive formal “kudos” within their department for helping the police and prosecutors get convictions, as reflected in the minutes of crime lab meetings in Houston.25
A lab technician in Ohio, whose work may have led to several wrongful convictions, including the conviction of my client James Parsons, favored the prosecution so much that her personnel file even contained comments to that effect written by her supervisors. In my opinion, if a lab supervisor actually puts in writing that a lab technician goes too far to help the prosecution, it has to be bad. Really bad. Indeed, she was even overheard on the phone asking cops, “What do you need the evidence to say?”26 Cops and prosecutors loved her, though. One thank-you note in her personnel file, among many others of equal enthusiasm, said, “If you were here right now, I would give you a heartfelt hug and kiss for Valentine’s Day. . . . You helped me more than you will ever know.”27
• • •
In 2009, the National Academy of Sciences (NAS) issued a report entitled Strengthening Forensic Science in the United States: A Path Forward. The NAS is a private, nonprofit agency of scholars “dedicated to the furtherance of science and technology.” Congress chartered it during the Lincoln administration, with a mandate to advise the federal government on scientific and technical matters. Strengthening Forensic Science was initiated by a 2005 act of Congress, which required the NAS to examine the state of forensic science in the United States and make recommendations for improvement.
The NAS held hearings throughout 2008, taking testimony from federal agency officials; academics, and research scholars; private consultants; federal, state, and local law enforcement officials; scientists; medical examiners; a coroner; crime laboratory officials from the public and private sectors; independent investigators; defense attorneys; forensic science practitioners; and leadership of professional and standard-setting organizations. Between hearings, the NAS reviewed the literature on forensic science—reports, studies, and so on—to aid its inquiry.
The NAS also considered what the innocence movement had taught about the state of forensic sciences in this country. It invited Peter Neufeld, cofounder of the Innocence Project (based at Cardozo Law School in Manhattan), to testify. The work of the innocence movement was noted in the NAS report.
Strengthening Forensic Science was released on February 18, 2009. In the introduction, the NAS declared, based on what it had heard from those who testified: “The forensic science system, encompassing both research and practice, has serious problems that can only be addressed by a national commitment to overhaul the current structure that supports the forensic science community in this country. This can only be done with effective leadership at the highest levels of both federal and state governments, pursuant to national standards, and with a significant infusion of federal funds.”28 Strengthening Forensic Science reveals the problematic state of forensics in this country. Ironically, at a time when the public watches CSI and believes that forensic scientists are performing scientific miracles to catch bad guys, the report demonstrates that reality is far from this Hollywood myth.
Strengthening Forensic Science explicitly acknowledges the role that confirmation bias plays in causing error in the forensic sciences. Indeed, Dr. Dror testified before the committee, and his groundbreaking work was cited throughout the report. The NAS recognized that the prevalence of confirmation bias in the forensic sciences is partly a result of the close relationships between forensic scientists and law enforcement. It made two significant recommendations. First, it advised that Congress create a National Institute of Forensic Science (NIFS) to oversee forensic scientists in order to “remov[e] all public forensic laboratories and facilities from the administrative control of law enforcement agencies or prosecutors’ offices.” Second, it recommended that the NIFS “develop standard operating procedures . . . to minimize, to the greatest extent reasonably possible, potential bias and sources of human error in forensic practice. These standard operating procedures should apply to all forensic analyses.”29
Strengthening Forensic Science was welcomed by all in the innocence movement. The report not only recognized that confirmation bias distorted expert conclusions and testimony, but recommended that forensic scientists be entirely independent of prosecutors and police. Leaders in the movement who had been trying to raise awareness of this problem for many years, in an effort to push through reforms, got little to no traction. Finally, a prestigious independent agency that advises Congress had recognized the depth of the problem and urged Congress to make sweeping changes to improve the system.
Yet, more than eight years later, little to no reform has taken place. I frequently speak about the innocence movement across the state of Ohio and around the country. Prosecutors and judges are often in the audiences. From talking to them and answering their questions, I am under the impression that, even today, very few know anything about Strengthening Forensic Science. Many have not even heard of it.
As a result, many of the forensic disciplines that were identified as problematic in the report are still being used without any significant modifications or improvements in courtrooms across America. Indeed, in 2016, the President’s Council of Advisors of Science and Technology issued a report echoing Strengthening Forensic Science, concluding that many forensic disciplines still in use are based on “scant scientific underpinnings” and need additional support to be considered valid and reliable. In response, U.S. Attorney General Loretta Lynch issued a statement saying her department “believes that the current legal standards regarding the admissibility of forensic evidence are based on sound science and sound legal reasoning.” She concluded by adding, “the department will not be adopting the recommendations related to the admissibility of forensic science evidence.”30
One recommendation from Strengthening Forensic Science that was adopted, however, was the Department of Justice’s creation of the National Commission on Forensic Science to implement some of the suggestions from the report. Shortly after it began its work, however, Donald Trump was elected president, and his administration promptly killed it.31
The beat goes on.
• • •
Cases where the confirmation bias of a forensic scientist contributed to a wrongful conviction are plentiful. Ray Krone’s case in Arizona is a notorious example.32 On the morning of December 29, 1991, the body of a thirty-six-year-old female bartender was found in the men’s bathroom of the bar where she worked. Krone, a regular patron, became a suspect because he had helped the victim close up the night before. Because the victim had been bitten during the attack, and bite marks were found on her body, the police had Krone bite into a Styrofoam cup to take an “impression” of his teeth. Several experts for the state then compared the impression to the bite marks on the victim’s body and declared them a definitive match.
Krone’s teeth were particularly crooked, and he became known in the media as the “Snaggletooth Killer.” At his trial, the forensic experts told the jury that Krone’s distinctive teeth—and only Krone’s teeth—could have made the bite marks on the victim’s body. The prosecution played up the power of this CSI-type evidence in its closing arguments. And, of course, with such seemingly ironclad evidence, the jury convicted. Krone was given the death penalty and sent to death row.33
Ten years later, Krone became the hundredth person exonerated and freed from death row in America. DNA testing of the saliva from the bite marks and blood left by the perpetrator proved that Ray Krone was not the killer. Rather, a man named Kenneth Phillips, who had been convicted of a sexual offense and lived near the bar, was identified as the true killer. Krone later received a $4.4 million settlement from the state of Arizona, and is “snaggle-toothed” no more after getting his teeth reconstructed on the TV show Extreme Makeover.
Krone’s case begs the question: How could several forensic odontologists have taken the stand, looked the jury in the eye, and told them that Ray Krone’s unique teeth—and only Ray Krone’s teeth—could have made the bite marks on the victim’s body? Were they intentionally lying to frame an innocent man? Of course not. The state’s scientists were given a job to do—match Krone’s teeth to the bite marks on the victim’s body. Before they even started their analysis, they believed Krone’s teeth would match because he was the person held up by the police as “the guy who did it.” With a task as subjective as comparing a bite mark in malleable skin to a bite impression on a Styrofoam cup, it is easy for an expert to see what she expects to see. Simply put, Krone was convicted not on the grounds of cold, hard, neutral science, as the jury believed, but as a result of confirmation bias.
The Larry Pat Souter case out of Michigan is one of many that highlight how pressure from prosecutors and police to solve a crime and help nail their prime suspect can lead forensic scientists to view evidence in a skewed way.34 In August 1979, Kristy Ringler was found unconscious on a highway in White Cloud, Michigan. She died shortly thereafter. An autopsy concluded that she died from two blows to her head that could have been caused either by homicide or by being struck by a car.
When police arrived on the scene where Ringler’s body had been spotted, a crowd of people had already started to gather. The group told the police that Ringler had been with them at a nearby party earlier that night, and that they had come down the street to see what had happened. Larry Pat Souter was at the scene, and he told police that he had met Ringler at a bar earlier that night and that she had accompanied him to the party. Souter admitted to police that at one point during the party he and Ringler had gone outside together, and that she had said she was going home and then began walking along the highway. Souter claimed that he had followed her a short distance trying to convince her to get a ride rather than walk, but then gave up after a few minutes and went back to the party.
The next day, police found a whiskey bottle near the spot where Ringler’s body had been found. A trace of blood was found on the label. A lab determined that it was the same blood type as both Ringler’s and Souter’s. Souter admitted that the bottle was his, explaining that he had discarded it on the way to the scene. But he insisted that he had nothing to do with Ringler’s death. He told police that the blood on the bottle had come from a cut on his finger from earlier that night.
Police also recovered particles of glass from Ringler’s clothes that the lab determined were not from a car’s headlights. Nevertheless, a forensic pathologist consulted by the police told them that, in his opinion, Ringler’s injuries were a result of being struck by a car. At that point, the local county prosecutor decided that there was not enough evidence to charge Souter or anyone else.
The investigating police officer on the case, however, had an ongoing suspicion that Ringler had been murdered. He speculated that Ringler could not have been hit by a car because her body appeared to have been placed in the road, and no blood or debris was found on her clothing. A few years later, he presented the case to the county medical examiner, who concluded that the whiskey bottle belonging to Souter could have caused Ringler’s injuries. But the local prosecutor again decided that there was not enough evidence to charge Souter.
More than a decade after Ringler died, a newly elected sheriff, who had run on a promise of solving old homicide cases, including Ringler’s, took over the investigation. With renewed vigor, his deputies reinterviewed many of the old witnesses from the case in an attempt to jump-start the investigation, but they discovered no new leads. In this pressure-filled atmosphere, the sheriff asked the medical examiner to review the evidence again, and this time—miraculously—the medical examiner concluded that Ringler’s injuries were caused by a blow from Souter’s whiskey bottle. In 1991, more than twelve years after Ringler’s death, Souter was arrested for her murder.
At Souter’s trial, in 1992, several forensic pathologists working for the state testified that Ringler’s injuries were consistent with being struck by the whiskey bottle. Souter claimed innocence, and explained his blood on the whiskey bottle by presenting the testimony of the owner of the house where the party took place, who told the jury that Souter had asked her for a bandage that evening because he had cut his finger on a broken door handle. Another witness testified that when Souter returned to the party, he was not acting abnormally, sweating, or breathing hard, and that there was no blood visible on his clothing. Still, Souter was convicted and sentenced to twenty to sixty years in prison.
In prison, Souter continued proclaiming his innocence. Eventually he and his supporters were able to amass substantial new evidence demonstrating his innocence. The most compelling was a statement from a woman who had read about Souter’s possible wrongful conviction, and came forward and provided police with evidence that her father had been involved in a hit-and-run on the same night and highway where Ringler was killed. Ultimately, a federal appellate court found that Souter had presented compelling evidence of innocence, and a federal district court reviewed that evidence and tossed out Souter’s conviction. He was released in December 2005 after serving thirteen years for a crime he didn’t commit.
Notice that in Souter’s case the changing opinions of the state’s forensic scientists depended on how fervently the state believed at the time that Souter had killed Ringler and how much pressure the authorities were under to obtain a conviction. When the medical examiner got the message that the authorities believed Souter did it—and wanted his conviction—suddenly, after more than ten years had elapsed, the whiskey bottle with Souter’s blood on it matched the injuries on Ringler’s head and became “the murder weapon.”
Some may say that the fix was in on this case, and that the forensic scientists knew there was no match between the whiskey bottle and the injuries, but simply rubber-stamped the sheriff’s request. But, after seeing this phenomenon in case after case, I believe the problem is subtler. In cases where scientists provide slanted reports to assist the prosecution, I think they usually deeply believe their flawed conclusions. Because scientists are human, and humans tend to see what they want to see when the pressure is on, most of these scientists simply convince themselves that the wrong answer is correct. In other words, rather than trying to shoot an arrow at a bullseye on the side of a barn, they shoot the arrow first and then paint a bullseye around wherever it lands. What they come to believe as the “right” answer causes them to see things, honestly see things, they didn’t see before.
Take the Robert Lee Stinson case in Wisconsin. Confirmation bias caused two forensic odontologists to call Stinson’s teeth an “identical” match to a bite mark on the victim “with no margin of error,” even though Stinson was missing a tooth where the perpetrator clearly had a tooth.35 DNA evidence later conclusively proved Stinson innocent, and a panel of odontologists who reviewed the case easily identified this obvious discrepancy that the state’s experts at trial had overlooked. Stinson served two decades in prison for a murder he didn’t commit, before he was released in 2009.
In another case, in Mississippi, a forensic odontologist provided testimony for the state that bite marks matched a number of different suspects, resulting in a series of possible wrongful convictions. And in at least two of the cases where DNA testing later cleared the defendants after years in prison, it was determined that the marks on the victims’ bodies weren’t even bite marks at all but had been made by insects and other animals after the victims had been killed and left outdoors.36
A particularly extreme example of confirmation bias skewing a result—so extreme that it may approach outright forensic fraud—comes from the Missouri case of George Allen. Allen was convicted in 1983 of murdering and raping a woman in her apartment. The crime occurred before there was DNA testing, and a forensic serologist testified that blood typing of the semen found on the victim’s clothes could not exclude Allen as the perpetrator.37 In closing arguments at trial, the prosecutor hammered home this evidence, stating that if Allen had been excluded as a source of the semen, “We wouldn’t be here. We’d know that he couldn’t have [done it]. But it’s consistent.”
Years after he was convicted and imprisoned, Allen’s attorneys found notes from the forensic serologist, which had not been previously disclosed, indicating that he had initially concluded that the semen from the crime scene contained an antigen that excluded Allen as the source. But the serologist had crossed out that finding and changed it to a finding consistent with Allen’s culpability after learning of Allen’s blood type. When he later typed up his formal report for trial, he revealed only that the semen was consistent with Allen. Years later, after all of this came to light with Allen’s exoneration, the serologist explained in depositions that after he learned from the authorities that Allen was the culprit, he assumed that his initial result excluding Allen must have been a result of lab error. So he just changed it.
Allen was freed in 2012 after serving thirty years in prison for a crime he didn’t commit. Again, the “right” answer—the one that the scientist deemed “must” be true given the state’s theory of the case—rather than objective science, dictated the result. The scientist in Allen’s case believed so strongly in the state’s evidence against Allen that it caused him to conclude that his initial finding exonerating Allen must have been a mistake.
Lest the reader think that confirmation bias leading to skewed expert results has only occurred in a few random cases around the country, let me say directly: that would be a serious misimpression. Any attorney who, like me, does postconviction innocence work in any state in this country can tell story after story about cases where forensic experts provided tenuous conclusions that did nothing but confirm the preexisting beliefs they had developed from their friends in the police department or prosecutor’s office. It is a very common occurrence. And one does not have to dig very deeply into the files of any organization like mine, the Ohio Innocence Project, to see plenty of examples. It’s like shooting fish in a barrel.
My office represented Walter Zimmer, who with his co-defendant, Thomas Siller, was exonerated and received a six-figure settlement from the city of Cleveland after spending thirteen years in prison for a murder he didn’t commit.38 The true killer, Jason Smith, had initially been caught red-handed because his fingerprints were all over the home of the victim, an elderly woman who had been tied to a chair and beaten to death in her living room. Smith cut a deal with prosecutors, however, receiving what a court later called a “breathtakingly favorable deal” to testify against Zimmer and Siller, whose fingerprints were also present in the victim’s home. Zimmer and Siller were contractors who had done extensive work there.
At trial, Smith testified that he was present when Zimmer and Siller beat the victim to death, but witnessed the crime from a distance inside the house. Zimmer and Siller asserted their innocence. They argued that Smith committed the crime alone and implicated them simply to save his own skin. Pants worn by Smith on the night of the murder were analyzed by the state’s forensic scientist, who testified initially that he did not find a single drop of blood on them, thus corroborating Smith’s story that he was not the killer. Later, the scientist modified his conclusion, admitting that he found one drop of the victim’s blood on Smith’s pants. Although Smith’s pants had numerous small brownish stains on them, the scientist said that he had not performed tests on those spots to determine if they too were the victim’s blood because he could tell from their appearance that the stains were not blood. The prosecutor maintained in closing arguments that the single drop of the victim’s blood likely got on Smith’s pants when he rubbed up against Zimmer or Siller—who would have been soaked in the victim’s blood if they’d committed the murder—and that the absence of a large amount of blood on Smith’s pants corroborated Smith’s story.
But DNA testing performed after Zimmer and Siller had already served years in prison demonstrated that Smith’s pants were, in fact, covered with the victim’s blood. Those brownish spots that the scientist had concluded could not be blood were indeed blood, and not just any blood, but blood that DNA testing confirmed belonged to the victim. Zimmer and Siller were exonerated, and Smith got five years in prison for perjuring himself against the pair to reduce his own culpability. In the expert’s mind, the spots weren’t blood since he believed that Smith was not the killer, and hence that’s what he saw. Zimmer and Siller spent a combined twenty-six years in prison as a result. They were declared innocent and compensated by the state of Ohio.
In the Ohio murder case of Bob Gondor and Randy Resh, the authorities theorized that the two men killed the female victim and then put her body in the back of Gondor’s pickup truck so that they could dispose of it in a nearby pond.39 The state’s experts analyzed marks and cuts on the victim’s face and body, and concluded that they were made from the body bouncing on the pieces of wood found in the back of Gondor’s truck, and from gravel near the pond, as the defendants had allegedly dragged the body into the pond where it was later found. Years later, after substantial evidence of their innocence was amassed, an expert from Minnesota re-analyzed the marks. The expert, hailing from a state full of ponds and lakes, was very familiar with bite marks left by turtles on dead bodies found in water, and she testified that the marks on the victim’s face and body were textbook turtle bites. Gondor and Resh were released and declared innocent by the state of Ohio after spending sixteen years in prison for a crime they didn’t commit. The state’s expert had analyzed the body under a belief that Gondor and Resh had carried it in a truck full of wood and then dragged it through gravel, and—presto—saw wood and gravel marks on it. This conclusion was presented at trial as a high-tech CSI revelation, when in reality it simply confirmed a preexisting belief.
In the Ohio Innocence Project case of James Parsons, the authorities believed that he had killed his wife with a large tool that had the brand Craftsman embossed on the side in raised letters.40 The supposed expert who testified for the state claimed that she had analyzed the sheet that was over the victim when she was attacked, and, from her extensive experience, was able to see some of the letters from the word “Craftsman” in the blood-stains on the sheet. She claimed that she could see the letters only under special conditions in her lab that could not be reproduced at trial. And she failed to photograph the incriminating images that she alleged she had been able to produce in her lab. Later, when the Ohio Innocence Project took the case back to court to try to exonerate Parsons, and exposed this junk science in the process, OIP was able to obtain this lab technician’s personnel file. It contained notations indicating that she often slanted her lab results in favor of prosecutors.41 Parsons’ conviction was overturned after this junk science was exposed in postconviction litigation—but only after he had spent twenty-three years in prison.
In another Ohio Innocence Project case, one that is no less disturbing, two victims had been murdered in a bar, and the perpetrator had then used a crowbar to rip open the cigarette machine to take the coins inside.42 When Ed Emerick later became a suspect, a forensic expert analyzed Emerick’s crowbar and testified that it—to the exclusion of every other crowbar in the world—had made the scratches on the side of the cigarette machine. Emerick’s attorney, to his credit, questioned the state’s expert during trial about the possible effects of confirmation bias. He got the expert to admit on the stand that he knew what the “right” answer was before he even began his analysis. The expert countered by arguing that his supervisor had performed an independent examination of the crowbar after he reached his conclusion and reached the same conclusion. The defense attorney then got the expert to admit, however, that the supervisor was only asked to “confirm” the results of an initial analysis if a match had been made by the first expert. Thus, the supervisor had the same preexisting bias as the first expert and knew that his underling had already called Emerick’s crowbar a match. To this day, Emerick remains in prison.
If all that isn’t scary enough, consider the Ohio case of Ryan Widmer, which I helped litigate. The state theorized that Widmer drowned his wife in their bathtub, while Widmer maintained that his wife suffered from a medical condition that caused her to lose consciousness in the tub and drown.43 The police found large body marks on the sides of the tub. Such marks can be left by parts of a human body, such as arms and legs, touching the tub and leaving an oil mark much as when a fingerprint is left. The state’s expert claimed to be able to identify certain marks on the tub as a “male forearm” and a “female hand.” Moreover, his identification of the gender and body part of each mark corroborated precisely the state’s theory as to how Widmer pushed his wife into the tub to drown her.
In the Parsons, Emerick, and Widmer cases, the state’s expert testimony was not grounded in any sort of actual science. There is no accepted scientific principle that allows an expert to conclusively identify letters from a tool in a bloody sheet that no one else can see. There is no scientific principle that allows an expert to determine that Emerick’s crowbar—to the exclusion of every other crowbar in the world—left the scratches on the cigarette machine. There is no scientific principle that allows an expert to identify body-oil smudges on the side of a tub as a male forearm or a female leg. Rather, in each of these three cases, a forensic scientist simply saw what he expected to see as a result of confirmation bias, and then presented that conclusion to the jury under the guise of scientific infallibility.
The cases I’ve recounted here are from my own case files. Other similar cases from my files are too numerous to even mention. Again, any attorney doing postconviction innocence work in this country can literally tell story after story of individuals being convicted based on unscientific speculation by experts stemming from a desire to confirm the prosecution’s theory of the case. This goes on every day.