Speaking of tests that aren't all they're cracked up to be, let's look at DNA testing. This is supposed to be the absolute silver bullet of criminal justice, an incontrovertible way to pin guilt on someone. After all, the chances of a mismatch are one in a billion, a quadrillion, a jillion! Some experts have testified under oath that a false match is literally impossible.
Not quite. As he did with HIV testing, risk scholar Gerd Gigerenzer of the Max Planck Institute punches a hole in the matching of genetic material:
In the first blind test reported in the literature, three major commercial laboratories were each sent 50 DNA samples. Two of the three declared one false match; in a second test one year later, one of the same three laboratories declared a false match. From external tests conducted by the California Association of Crime Laboratory Directors, the Collaborative Testing Services, and other agencies, the psychologist Jonathan Koehler and his colleagues estimated the false positive rate of DNA fingerprinting to be on the order of 1 in 100. Cellmark Diagnostics, one of the laboratories that found matches between O.J. Simpson's DNA and DNA extracted from a recovered blood stain at the murder scene, reported its own false positive rate to the Simpson defense as roughly 1 in 200.
It gets even worse. In 1999, the College of American Pathologists performed its own secret tests of 135 labs. Each lab was sent a DNA sample from the “victim,” some semen from the “suspect,” and a fake vaginal swab containing DNA from both parties. They were also sent a strand of the “victim's” hair. The object was to see how many of the labs would make the matches (ie, match the two sperm samples of the man, and match the hair and DNA sample of the woman). But something unexpected happened: Three of the labs reported that the DNA from the suspect matched the victim's DNA! Obviously, they had mixed up the samples. Only fourteen labs tested the hair, but out of those, one screwed it up by declaring a match to the “suspect.”
These kind of switches don't happen only during artificial situations designed to gauge a lab's accuracy (which are usually performed under ideal conditions). During a 1995 rape trial, a lab reversed the labels on the DNA samples from the victim and the defendant. Their testing then revealed a match between the defendant's alleged DNA (which was actually the victim's) and the DNA on the vaginal swab, which didn't contain any semen from the rapist. Luckily, this boneheaded move was caught during the trial, but not everyone is so lucky.
The Journal of Forensic Science has reported an error that was discovered only after an innocent man had been convicted of raping an 11-year-old girl and sentenced to prison, where he was undoubtedly brutalized in ways that would give you nightmares for the rest of your life, were you to hear them described in detail. After four years, he was released because the lab hadn't completely separated the real rapist's DNA (extracted from his semen) from the victim's DNA. When the two were swirled together, they somehow matched that of the poor bastard whose eleven alibi witnesses failed to sway the jury. But when the semen DNA was checked properly, it was beyond doubt that a match didn't exist.
While most false matches are the result of human error, other factors do come into play. Some testing techniques are more definitive than others. In the case of one innocent man — Josiah Sutton, found guilty of rape based primarily on DNA evidence — criminology professor William C. Thompson said: “If police picked any two black men off the street, the chances that one of them would have a DNA profile that ‘matched’ the semen sample as well as Sutton's profile is better than one in eight.” Also, we mustn't forget about corruption. In some known cases, DNA analysts have misrepresented (ie, lied about) their findings in order to obtain convictions.