5

THE END OF ANONYMITY

“The history of surveillance is the history of powerful surveillance systems being abused.”

ON DECEMBER 17, 2004, PRESIDENT GEORGE W. BUSH SIGNED THE Intelligence Reform and Terrorism Prevention Act into law. It was a sweeping set of reforms most notable for creating the director of national intelligence (DNI), a cabinet-level official who oversees America’s intelligence community.

The bill also gave $20 million to the Transportation Security Administration (TSA) “for research and development of advanced biometric technology applications to aviation security, including mass identification technology.” “Biometric” is one of those words that sounds complex but refers to something quite simple: unique physical or behavioral characteristics that can be used to identify someone digitally. They include fingerprints, retinal scans, voices, gaits, or—what we’re interested in for the purposes of this chapter—faces.

That act also laid the groundwork for a biometric entry and exit system at airports, arguing that “completing a biometric entry and exit data system as expeditiously as possible is an essential investment in efforts to protect the United States by preventing the entry of terrorists.” Of course, “as expeditiously as possible,” when referring to government work, should not be mistaken for actual speed. Customs officials began fingerprinting all foreign nationals between the ages of fourteen and seventy-nine upon arrival after the act passed, but not until December 2018 did America get its first biometric airport terminal, in Atlanta.

Since then, as I discussed in the prologue, passengers have been able to check in to their flights and go through the security line by having their pictures taken. The picture-taking replaces the traditional ID and ticket check; the picture is matched to the passenger’s passport photo and to the biographical information that the airline and Customs and Border Protection (CBP) have on file.

Passengers can opt out—and, except for the experiment I describe below, I always do. Whether you want to do the same is up to you. But, as I mentioned earlier, opting out can be difficult. Facial-recognition screening is quick; the process is set up for it. Opting out slows everything down. Maybe you can feel, or imagine you can feel, the people behind you in line glaring and growing impatient. But to voluntarily use facial recognition in this way is to normalize it, and to signal to the airlines and government that you approve of it. Most importantly, bear in mind that the more you choose to use facial recognition, the more it will be used in ways and places that you do not choose. Your assent helps this technology spread.

CBP keeps American citizens’ pictures for twelve hours—enough time to get them on the plane and into the air. They already have our passport photos and biographical data. They keep foreign nationals’ pictures for seventy-five years, but they then fill the same function as Americans’ passport pictures—on future visits, foreigners’ faces are compared to the images that CBP already has on file.¹

I flew out of Atlanta’s biometric terminal to Quito, Ecuador, in early June 2019. Many people have no direct experience with facial recognition in their daily lives; I wanted to see how it works in practice. I had already checked in using my phone, but I was curious to see how good the facial-recognition cameras would be. In my passport picture, I am bearded and deeply tanned; at the time of my flight, I was clean shaven and, alas, much paler (that’s what happens when your beat changes from roaming around Southeast Asia to a desk in Washington, DC).

At the same time, I had, and still have, tremendous misgivings about facial recognition. It worries me that around half of all Americans—the majority noncriminal, of course—have their pictures stored in police-accessible databases (the pictures are mainly passport and driver’s license images; state laws vary in whether they allow police to access the latter). It also worries me that the technology has a reputation for bias against women and nonwhite faces. Although, in this case, facial recognition just replaces the existing practice of manually checking a passport picture, I worry about the acclimatizing effects. Airlines want to expand the practice to domestic flights, and I suspect that as people grow accustomed to checking in seamlessly with their faces, the technology will start showing up in more and more places—which means more and more databases will contain our faces. How well guarded are those databases? Who gets to see them?

Facial recognition presents a greater potential for abuse—for subjecting innocent people to permanent suspicionless surveillance and tracking—than any other technology in this book, even automatic license plate readers and cellphone tracking. After all, you can always take the bus or leave your phone at home. You can’t shed your face so easily. And regulations governing its use are at best ad hoc, at worst nonexistent, and rarely include penalties for breaking them.

So after I dropped off my bags, I wasn’t sure what I wanted to do: have my picture taken and go through the system for the sake of journalistic documentation, or opt out as a stand against an unreliable and potentially dangerous technology. In the end, I did not have to make a choice.

By the time I made it near the front of the line, the system was wonky. Both the punctiliously polite TSA agent and the well-dressed, middle-aged blonde woman in front of me were starting to lose their cool as she stood staring at a camera, repeatedly not having her picture taken. He ended up screening her, me, and who knows how many others behind me the old-fashioned way. He put the ticket’s bar code on the scanner, compared my face to the one in my passport, and waved me through. If this were a novel, the whole exchange would have been a little too on the nose. Reality has no such narrative constraints.

Of course, just because one airport line’s facial-recognition system failed to function does not mean that facial recognition is a failure. Just the opposite: it is improving constantly. Still, some might argue that my experience should allay people’s fears. They might assert that the fevered imaginations of privacy activists have turned facial recognition into a perpetual-surveillance bogeyman, but in practice, facial recognition is clunky and only somewhat effective in a limited number of use cases. Police mainly use it to compare pictures taken under controlled circumstances to other pictures taken under controlled circumstances. So calm down, argue facial-recognition boosters; all it does is enhance and speed up what law enforcement already does.

This idea is complacent and wrong. Facial recognition may not be able to track us everywhere we go right now. But the technology is not quite in its infancy; it’s in mid-adolescence, its big-paws-on-a-puppy stage. Amazon’s Rekognition, for instance, is a plug-and-play system that uses machine learning to constantly improve. Facial recognition will only get better, cheaper, and—absent strong public outcry—more commonly used, risking the end of our ability to be anonymous in public.

Right now, and only right now, concern about its potential harms can be mobilized to outweigh its ability to actually do harm. This is not the time to laugh off its clunkiness. This is the time to consider carefully what facial recognition can do, what we want it to do, where we want it used, and, most importantly, where we do not want it used, and what we do not want it to do—and to take action and craft policy accordingly.

At its core, the phrase “facial recognition,” as ordinary people most often use and understand it, refers to a series of algorithms applied to a digital image designed to detect, recognize, and identify a human face. There are numerous specific, proprietary algorithms—at least one hundred available on the US market—that do each of these things. Parsing and evaluating the difference between them is beyond the scope of this book; instead, I’m going to try to give a very basic, general explanation of how this process tends to work.

Neurologists, programmers, and others who understand the more intricate scientific and technological aspects of facial recognition better than I do may be familiar with the description that follows. They may even find it simplistic. I don’t apologize for that: I’m not a scientist, and this book is for the lay reader. But instead of skipping ahead or gnashing their teeth, I hope they keep reading; I hope they find the next section a helpful guide to explaining the technology’s dangers to the general public.

The first step in facial recognition is face detection, and the first step toward both is training. Programmers show software numerous images of faces, telling the computer that these things are faces. The software can then deduce that in future images, these roundish things with two roundish sunken spaces that might reflect light above a protrusion, with other protrusions on either side of the big roundish object, and a line or open space below the central protrusion, probably constitute a face (they are probably eyes, a nose, ears, a mouth).

With algorithms that use deep learning or neural networks, as the best ones now do, precisely how they reach this conclusion—or indeed, any conclusion—remains something of a mystery, even to the algorithms’ creators. This is not entirely surprising: how human neural networks (also known as “brains”) function at a granular,

decision-making level is similarly mysterious. These algorithms are, in a sense, black boxes, and are judged by their output, not the routes they take to get there.

These sorts of opaque algorithms may be brilliant and efficient, but they also contravene the right to confront one’s accuser, which is central to the concept and jurisprudence of due process. If that accuser is a complex neural network whose precise functioning cannot be explained—and which, unlike people, who use a complex neural network between their ears, cannot be confronted and questioned—then that right is violated.

Your phone’s camera detects faces in real time. When you prepare to take a picture, boxes or circles probably appear around the faces in the image. When that happens, the camera’s software is saying, in essence, that there is a high probability that this set of features—“features” here referring not directly to facial features, but to specific patterns: gradations of color relative to other gradations of color in the image, such as pools of white around a dark core (eyes), above a protrusion that catches or shadows light (nose), and so forth—constitute what it understands to be, based on previous experience, a human face.

Algorithms then determine whether the image of a face is occluded, say, or in profile. Such images are usually sufficient for detection as faces—a program can usually detect whether a face in profile, or in poor lighting, or turned downward, away from the camera, is indeed a face. But an image clear enough to be identified based on comparison to a picture in a database, usually taken under cooperative conditions (meaning when the subject is posed and looking directly at the cameras), requires rotating the image so that it is as close to a straight-ahead shot as possible, which most software can do quickly.

Next the algorithm will usually translate a color image of a face into grayscale. By getting rid of colors, the algorithms are left only with gradations of brightness. It will then search for unique points of reference on the image that collectively compose a unique face. Precisely which ones vary by algorithm, but they tend to include distances: between the eyes, from one side of the chin to the other, from the bridge to the tip of the nose, and so on—dozens of distance markers. The algorithm compares these unique features to those found on people’s faces in its database and determines how likely it is that a match exists.

The introduction of “convolutional neural networks,” as I will discuss later in this chapter, has improved facial recognition’s accuracy well beyond the old method, which relied solely on comparing linear measurements between points on faces on file. Still, as far as I know, most facial-recognition systems have not yet reached the gold standard of “one to one in the wild,” meaning a system that can reliably identify a person as they walk past a camera at their usual pace, occluded or poorly lit. But they will.

Nor will this recognition capacity stop at faces. In Israel I saw a system being trained to recognize gaits and bodies, so it can identify someone from behind or without a clear view of the face. Chinese systems recognize ages and genders; they are being trained to recognize emotions. And systems that use machine learning and neural networks, as the best do today, improve on their own; they can learn, just as we do.

Yet even the best facial-recognition systems determine probabilities, not certainties, and operators of these systems need to determine what level of probability they are comfortable with. Set the bar too high, and the system will produce a lot of false negatives: it will miss people who are in the database. Set it too low, and it will produce a lot of false positives: it will see likely matches where none exist. Or, in policing terms, set it too high, and the suspect will not be identified; set it too low, and innocent people will be bothered and possibly arrested.

This problem is not exclusive to facial recognition. Television shows like CSI paint biometrics and forensic science as practically infallible. These TV detectives talk of finding a fingerprint or other biometric “match,” when in the real world such evidence is far less certain: it usually points to probable matches, at best, rather than toward certainty. And sometimes biometrics fail.

Consider, for instance, the aftermath of the Madrid train bombings in 2004. On March 11 of that year, terrorists detonated several bombs on commuter trains that killed around 200 people and injured another 1,400. Eight days later, the FBI identified a fingerprint on a recovered bag of detonators as Brandon Mayfield’s. Mayfield was a lawyer in Oregon; investigators found no links between him and the attacks. Nonetheless, Mayfield was put under covert surveillance, and his home and office were searched. He was arrested on May 6 and detained as a material witness for two weeks—despite his lack of connection to the attacks, and, even more alarmingly, despite the Spanish National Police having told the FBI nearly a month earlier that the fingerprint did not in fact belong to Mayfield. Not until the Spanish police publicly revealed that the fingerprint belonged to an Algerian national was Mayfield released. He ultimately won a public apology and $2 million from the federal government.²

Or consider also the well-earned discrediting of other quasi-biometric sources of data, such as bite-mark analysis. Often presented at trials as forensically accurate, in fact bite-mark analysis is wholly unscientific: it’s just a person making an observed judgment. Bite marks shift and change when left on skin: they can be distorted by the skin’s elasticity, or shift over time as the victim’s skin swells, heals, or decays. Yet it has put people in prison, often incorrectly: bad bite-mark evidence has led to multiple exonerations.³

But where does facial recognition fall on the biometrics spectrum, which stretches from DNA analysis (fairly reliable, given an adequate sample and the proper interpretation) to bite-mark analysis (wholly subjective and without scientific backing)? How good is facial recognition at, well, recognizing the right faces?

Answering that question is difficult, not least because every algorithm is different. In that sense, asking whether facial recognition is accurate is a bit like asking whether cars are fast. Some are and some aren’t. Some have higher top speeds than others. Some run smoothly at high speeds, and some rattle and shake on highways. A Lamborghini can reach higher speeds, and reach moderate speeds faster, than my trusty old Subaru. Some cars are generally faster than others, but almost all of them will get you where you want to go faster than if you walked, just as all facial-recognition algorithms are intended to recognize faces.

In testimony to the House Oversight Committee on March 22, 2017, Kimberly Del Greco, deputy director of the FBI’s Criminal Justice Information Services Division, said that the system her agents used at that time “returns the correct candidate a minimum of 85 percent of the time within the top 50 candidates.”⁴ That’s not very good.

It raises a question for trials: if a person is arrested based on a lead generated by a facial-recognition algorithm that ranked him the forty-fifth most likely match of fifty, how should a defense attorney use—and how should a judge and jury weigh—that information? Generally, police say that facial recognition is used to generate leads, not proof. Presumably, police and prosecutors will say that in this case, the lead that came from the forty-fifth person panned out, while the first forty-four did not, and it shouldn’t matter where someone ranked as long as the police arrested the right person. But that contradicts testaments to the technology’s effectiveness.

In July 2018, the ACLU of Northern California built a facial-recognition database using publicly available mugshots and Rekognition, Amazon’s deep-learning facial-recognition software. Used by police departments and large corporations, it is part of Amazon Web Services’ push into the surveillance market (Amazon also owns Ring). The ACLU, as mentioned in Chapter 1, ran every current member of Congress through the database, using Amazon’s default settings. The database misidentified twenty-eight members of Congress as arrestees, including a disproportionate share (40 percent of wrong matches) of nonwhite representatives, including John Lewis, a hero of the civil rights movement who represented part of Atlanta from 1987 until his death in July 2020, and Luis Gutierrez, who represented parts of Chicago for twenty-six years.

Matt Wood, Amazon’s vice president of AI, pushed back on the ACLU’s experiment, claiming that it used Rekognition’s “default confidence threshold” of 80 percent, not the 99 percent confidence level it recommends for law enforcement use. Moreover, Rekognition, he said, is “constantly improving.” Twice he pointed out, in one brief post, that Rekognition has been used to fight human trafficking (though he never says precisely how).⁵ But anything can be harmful in one case and helpful in others. Surface-to-air missiles have helped win wars; that doesn’t mean that my neighbor needs one. They would be far less productive in solving minor lawn-care disputes.

The ACLU’s is just one of many studies and experiments that have found racial and gender bias in facial-recognition algorithms. Sometimes those biases emerge without studies: in 2015, Google’s consumer-facing facial-recognition app misidentified two African Americans as “gorillas.”

In early 2018, Joy Buolamwini, a doctoral student at the Massachusetts Institute of Technology, and Timnit Gebru, an AI research scientist at Google, published a paper analyzing how well three leading facial-recognition algorithms—IBM’s, Microsoft’s, and Megvii’s, which is more commonly known as Face ++—identified genders across races. They found that the error rates of all three were roughly 2.5 times as high when identifying women than men, and nearly twice as high for darker-skinned than for lighter-skinned faces. All performed at their worst when trying to identify the faces of darker-skinned women.⁶

Precisely why these algorithms perform worse at these tasks is not clear, but it may have something to do with the datasets on which they were trained. If these sets contain more men than women, and more white people than nonwhite people, then the algorithm may perform best with the most familiar characteristics.

This suggests that facial-recognition race and gender bias might be ameliorated by training algorithms using a more diverse set of faces (which only means the algorithms would perform equally badly for all genders and ethnicities). IBM, at least, seems to believe that theory may be true: in early 2019, it released a dataset of one million diverse faces, annotated in ten different ways to give facial-recognition algorithms a stronger starting point.⁷ In response to Buolamwini’s finding, Microsoft, according to the New York Times, said it was “investing in research to ‘recognize, understand and remove bias.’”⁸ (Microsoft, like Amazon and IBM, has called on the government to regulate facial recognition, although, with Congress as dysfunctional as it is, that more or less amounts to virtue signaling.)

In aggregate, the algorithms appear to be improving. The National Institute of Standards and Technology (NIST), a nonregulatory government standards agency in the Department of Commerce, has performed regular tests on commercially available facial-recognition algorithms. In 2018, it evaluated 127 algorithms from 39 different developers, finding that they were collectively 20 times more accurate than in 2014.⁹

The report attributes the improvement to the use of convolutional neural networks. Facial recognition, it says, “has undergone an industrial revolution, with algorithms increasingly tolerant of poor quality images”—in other words, as facial recognition improves, so does its ability to accurately identify people from imperfect photos. The days of us having to stand still and stare directly into a camera in order to be recognized may soon seem as outdated as having to remain perfectly still for a daguerreotype.

But aggregate improvements can still mask microlevel differences. Just because facial recognition has grown more accurate does not mean that all algorithms perform equally well on all races and genders. An evaluation that NIST released in December 2019 found that, across 189 commercially available algorithms from 99 firms, systems fared notably worse at identifying African and East Asian people than they did at identifying Europeans. (That was not the case for algorithms developed in China, which displayed low false-positive rates for East Asians—suggesting that training networks on a broad array of faces could help ameliorate the racial false-identification gap.)¹⁰

Some systems will perform better than others. Still, without adequate benchmarks and policy guardrails, nothing compels police forces to choose a less biased algorithm if, for instance, it is more expensive than one that performs worse.

And even as facial-recognition algorithms grow more accurate across all demographics, they can still be used in a biased way—by being deployed, for instance, as some agencies currently deploy patrol forces: disproportionately in nonwhite neighborhoods. An algorithm that successfully identifies 99.8 percent of all the faces it sees, but is mostly used to identify black and brown people, becomes just another tool to perpetuate racial bias. As with any other technology that is used the wrong way, facial recognition risks calcifying, rather than ameliorating, society’s biases. That statement raises a crucial question: What is the right way to use it?

Perhaps the most targeted, limited way that police use facial recognition is to generate investigative leads. The sheriff’s department in Washington County, Oregon, was the first law enforcement agency in America to use Rekognition, Amazon’s facial-recognition tool. According to both its website and a presentation by Chris Adzima, a senior information analyst with the Washington County Sheriff’s Office, at a police-tech conference in May 2019, they do not use facial recognition to surveil people going about their daily business.

The department wrote a usage policy specifically vowing that their employees would “not employ this technology without a lawful justification that is based upon a criminal nexus,” that it would “not employ this technology to conduct mass surveillance,” and that facial recognition could “not be used to conduct surveillance of persons or groups based solely on their religious, political or other constitutionally protected activities.” The policy also states that “facial recognition search results are potential leads that require follow-up investigation,” and that those results alone do not constitute probable cause.¹¹

If a criminal suspect is caught on camera while engaged in what appears to be criminal activity, an officer, with approval from a commanding officer, compares that image to a database of county mugshot photos dating back to 2001. The picture that starts the investigation might be through any of a variety of means: through surveillance footage, for instance; or because someone took a picture of an assailant with a cellphone camera; or because a suspect consented to have his picture taken by a police officer, or has been arrested and had a mugshot taken, but refuses to provide identification. Only trained officers are allowed to run the searches.

The usage policy contains penalties for violating it—sort of (“discipline as appropriate”). All requests to run searches are subject to audit, and photographs that do not turn up a match are kept for a year and then deleted from the server. As facial-recognition usage policies go, this is about as good as it gets, but it’s really not all that good. The penalties are vague, and however benevolent the aims and limited the uses, the county is still building a database that could be used for broader and less benevolent purposes later on.

At the same conference where I saw Adzima’s presentation, I met Daniel Steeves, the chief information officer for the Ottawa Police Service. He is everything you would expect a senior Canadian police officer to be: mild, bearded, reasonable, and wry. His department piloted a program similar to Washington County’s: twenty-five officers in a robbery investigation unit received training and were then allowed to run images of people captured by surveillance cameras during what appeared to be criminal activity—Steeves mentioned stealing from liquor stores and gas stations—through a mugshot database.

“Without facial recognition,” Steeves explained, “you’d have an officer sending out a broadcast email saying, ‘Does anyone recognize this guy?’ You’d have two thousand people looking at an image that maybe three people should have seen.” He points out the privacy concerns surrounding that common practice and noted that a manual search of their mugshot database, which comprises fifty thousand images, could take weeks. The facial-recognition program returned possible matches in seconds.

“That acceleration of identifying a suspect created a different problem,” though, he added. “Now they have to assign an officer to apprehend the suspect or deal with the issue. With such quick identification, they found that they didn’t have enough officers on their team to deal with that pace—to go out and apprehend the suspect. We found we’re identifying so quickly we don’t have enough officers to deal with the dispatch.” He also said that facial recognition let them quickly identify a number of serial offenders, getting them off the streets before they committed more crimes.

In a New York Times op-ed published on June 9, 2019, James O’Neill, commissioner of the New York Police Department, detailed how his department used facial recognition, and it was in much the same way as the Washington sheriffs and the Ottawa police. They treat possible matches as a lead, “comparable to tips to our Crime Stoppers hotline—no matter how compelling, they must be verified to provide probable cause for an arrest.” Similarly, a guide for NYPD patrol officers released the following March said “the facial recognition process does not by itself establish probable cause to arrest or obtain a search warrant, but it may generate investigative leads through a combination of automated biometric comparisons and human analysis.” In 2018, said O’Neill, detectives asked officers from the Facial Identification Section to run 7,024 searches, which led to 1,851 possible matches and 998 arrests.¹²

O’Neill claims that facial-recognition software has also “cleared suspects,” though he provides no examples, only a bit of hopeful logic: facial recognition will make convicting innocent people less likely because—according to the Innocence Project, which seeks to exonerate the wrongly convicted—71 percent of documented false convictions stem from “mistaken witness identifications.”

First of all, facial recognition’s current accuracy rate, especially for nonwhite people, suggests that it will not function quite so perfectly. Second, O’Neill’s portrayal of the technology is contradictory: on one hand, facial recognition is just a lead that must be pursued, no different from someone calling in a tip; but it is also powerful and accurate enough to rule suspects out. This formulation implicitly demands that police be skeptical of supporting evidence, but accepting of evidence that contradicts their conclusions, which isn’t really how human nature works.

When detectives obtain a useful image in the course of an investigation, they provide it to trained officers from that section, who run it through a database of mugshots—“not,” Commissioner O’Neill explained in his op-ed, “photos from the Department of Motor Vehicles, Facebook, traffic cameras or the myriad streams of closed-circuit TV video from around the city.” The NYPD does not compare suspects’ photographs to police sketches, as other departments do. O’Neill believes it would be “an injustice to the people we serve if we policed our 21st-century city without 21st-century technology.”

That’s a fair point. But the NYPD’s rules are just that: self-imposed rules. What are the penalties for deviating from them? He does not say, and the NYPD appears to have no formal usage policy. Nor does O’Neill say what happens to photographs that generate no matches, who gets access to data generated by the system, or how accurate the system is overall: how many false matches it generated, whether any innocent people were questioned or arrested as a result of those matches, and what precisely he means by “match.” If a “match” is, say, someone whom the system identifies with 95 percent or greater certainty, that’s one thing; if it simply throws out a dozen people who might be the suspect, with a certainty of only 30 or 40 percent, and one of those leads pans out, is that better or faster than the old-fashioned method of investigation? That question is especially hard to answer, because nondigital methods of recognizing faces—i.e., leafing through police department mugshot books—do not have known error rates.

As Clare Garvie, a facial-recognition expert with the Georgetown Law Center on Privacy and Technology, pointed out in her report “Garbage In, Garbage Out: Facial Recognition on Flawed Data,” precisely what other evidence officers need to corroborate a match or make an arrest is unclear.¹³

Garvie described a case in which detectives in New York put a suspect in a lineup “solely on the basis of a facial-recognition search.” That suspect was then arrested based on witness identification that would not have happened had he not first been put in the lineup. In Washington, DC, an officer with the Metropolitan Police Department showed a witness a “possible match” photo from MPD’s database, and then obtained an arrest warrant based on the facial-recognition match, witness confirmation, and a vague social media post about a possible birth date, containing only the day and month, as the only evidence.

Strictly speaking, these arrests were not made solely on the basis of the facial-recognition match. But it was pretty close to that. What makes this doubly worrying is the quality of images used. Some agencies permit facial-recognition searches to be run on artists’ sketches, or on faces from partial photographs filled in using 3D modeling software, despite justified doubts about their ability to generate reliable results.

Still, some guidelines are better than none. In early July 2019, the Washington Post reported that federal agents were using facial recognition to trawl databases of state driver’s license photos.¹⁴ Those databases comprise hundreds of millions of pictures of people who have never been convicted of any crime. As the article noted, “Police have long had access to fingerprints, DNA and other ‘biometric data’ taken from criminal suspects. But the DMV records contain the photos of a vast majority of a state’s residents, most of whom have never been charged with a crime.”

Agents of the state searched these databases without the license-holders’ knowledge or consent. Agents with ICE, the FBI, and other agencies executed hundreds of thousands of searches over the past several years. These searches did not require warrants, the consent of elected officials, or any other sort of official policy sanction; they were often executed, according to the Post, “with nothing more formal than an email from a federal agent to a local contact.” Such searches do not just help ICE agents find the undocumented; in states that grant driver’s licenses to undocumented immigrants (Maryland, for example, has issued nearly three hundred thousand since 2013), they lead agents right to their door. And a later article clarified that ICE no longer needed a friendly state contact. They have independent access to driver’s license databases in some states, including Maryland’s—

a solidly liberal state that presumably never would have approved of giving ICE access.¹⁵

Indeed, among the most chilling aspects of this story is that nobody seems to know who granted ICE that access, or when or why—though it seems clear that voters and elected officials had nothing to do with it. The Washington Post quoted an ICE spokesperson saying the agency does not “routinely” use the database for civil immigration enforcement, but that should comfort nobody—it is in fact an admission that they have and do use it for precisely that purpose. ICE’s conduct elicited that rarest of all things in Washington: bipartisan agreement. Both Jim Jordan (R-OH), a stalwart ally of Donald Trump, and the late Elijah Cummings (D-MD), who, as chairman of the House Oversight Committee, led one of the panels involved in investigating the president during impeachment proceedings, blasted ICE for its conduct.

Some twenty-one states, plus Washington, DC, have agreements allowing federal agencies to access their databases of driver’s license photos. Those agreements stipulate that searches must pertain to a criminal investigation, but that is a low bar: it does not say that they can only search for criminal suspects, or with judicial approval, or after obtaining a warrant, only that they have to come up with a minimally plausible connection to a criminal investigation. Entering the country illegally is a criminal offense, which means as long as ICE is trawling for undocumented people, they clear the bar. According to the Post, agents searched not just for suspected criminals, but also for “possible witnesses, victims, bodies, innocent bystanders and other people not charged with crimes.” No policy allows such activity, but neither does any prevent it.

To really understand how invasive this practice is, think back to when you got your driver’s license. Did you assume that by obtaining a license to drive, you were automatically surrendering your face, birth date, address, size, ocular health information, and willingness to be an organ donor to the prying eyes of any federal agent who happened to have a buddy at a state DMV? I certainly didn’t. I assume you didn’t. It’s one thing for a state to retain digital and physical copies of convicted criminals’ biometric and personally identifying information. It’s quite another to do the same for people who have never been convicted. It represents a monstrous expansion of state authority and control. It is the sort of thing that makes people less willing to trust their governments.

Moreover, at least four states whose databases ICE agents trawled (Utah, Vermont, Washington, and Maryland) offer driver’s licenses to undocumented immigrants, reasoning (correctly, in my view) that if undocumented people are going to drive anyway—as many do, because they have to work—then for the safety of everyone else on the road, they might as well take driver’s education classes and obtain auto insurance. To then allow ICE agents to access their personal information is careless and indifferent, at best, if not blatantly duplicitous.

But these sorts of searches are not unusual. According to Clare Garvie, of Georgetown Law, at least one-quarter of America’s roughly twelve thousand police agencies have access to a facial-recognition system. At least forty-four states allow facial-recognition searches to be run on their driver’s license photo databases, and more than thirty let law enforcement directly run or request searches without a warrant.

Add to this the FBI’s database, which contains twenty-five million local and federal mugshots, along with all the local databases to which federal agents have access, and the faces of around half of all Americans are searchable in this way. It’s part of a vast biometric trove overwhelmingly made up of people who haven’t committed any crime, which now can be searched at will by the state’s agents without a warrant.

This is not the only way police use facial recognition. In another report, “The Perpetual Line-Up,” Garvie and her colleagues identified four of the most common police uses: “stop and identify,” when police come across someone who cannot or does not want to identify himself, snap a picture of the person with a tablet or smartphone, and then run it through and possibly add it to a database; “arrest and identify,” when a person is arrested and her picture is then entered in a police database, which may be accessible to federal agents who can run their own searches; “investigate and identify,” when police have surveillance footage of a suspect committing what appears to be criminal activity, and they run the suspect’s face through whatever databases they can access; and “real-time video surveillance,” discussed below.¹⁶

In China, cameras equipped with real-time facial-recognition technology litter much of the country. In December 2017, a BBC reporter, John Sudworth, wanted to test this network’s effectiveness.¹⁷ At that time there were 170 million state-owned closed-circuit television (CCTV) cameras active around China, with plans to roll out another 400 million by the end of 2020, amounting to around one camera for every two people.

Sudworth captured his whole experiment on video. First, we see him submitting his picture to police in Guiyang, the capital of Guizhou in southwestern China. Guiyang’s facial-recognition network has pictures and data on nearly every one of its roughly 4.3 million residents. A tech executive there boasts on the video that the cameras he sells power a network that can match any face with identifying information, including registered cars, friends, and relatives, and trace the person’s movements back one week.

After giving the police his photograph, Sudworth turns up in the city center—he had not told the police precisely where he would start—and just starts wandering. The goal is to see whether the police can find him, and if so, how long it takes. For context: more people live in Guiyang than in any American city except New York. In square miles, Guiyang is around four times the size of Chicago or Madrid. Seven minutes after a car drops him off, cameras catch him entering the bus station, and six policemen—who seem unaware that it’s supposed to be an experiment—surround him. Sudworth is friendly and disarming, but the police are unmoved. “Maybe these guys aren’t in on the joke,” he says on camera. They form a phalanx and walk him out of the station.

The most chilling thing about this experiment is that Chinese police said roughly the same thing American police do. “For ordinary people, we will only extract their data when they need our help,” one policewoman told the BBC. Otherwise, she said, in a nifty little contradiction, “we won’t gather their data.… [I]t only remains in our big database. We only use it when needed.” To Chinese state agents, “when needed” means whenever a higher-up tells them it’s needed. It could be to catch a thief or sexual assailant. But it could just as easily be to track parishioners walking home from an unauthorized church, or to keep tabs on people who have expressed political views that the authorities dislike.

Cameras equipped with real-time facial recognition exist in the West, too. Garvie found that, as of mid-2019, police forces in Detroit and Chicago have them.¹⁸ In March 2020, OneZero, an online journal about tech and science run by a platform called Medium, reported that Wolfcom, a company that has sold bodycams to at least 1,500 law enforcement agencies around America, was beta-testing real-time facial recognition.¹⁹

Detroit’s usage policy allows facial recognition to be run on “any interface that performs live video, including cameras, drone footage, and body-worn cameras”—which seems, at least to me, to mean any camera the police can get their hands on. Police in Detroit can run facial-recognition searches on their smartphones.²⁰ Detroit’s police chief later said they wouldn’t run real-time facial recognition, and they would only use the technology “when officers have reasonable suspicion that a crime has occurred.”²¹ A subsequent directive limited its use to violent crimes and home invasions, and limited image searches to still photographs. But the capacity exists, and it’s just waiting to be turned on.

The array of available cameras in Detroit includes hundreds of city-owned ones at schools, churches, hotels, public housing complexes, reproductive health clinics, and addiction treatment centers. Do we really want the government knowing how often people in its mugshot database go to church, or how often someone with an old conviction takes his kids to school? Do you want the government to track when people visit facilities that provide abortions? Are you really anonymous if you have to pass by a camera equipped with real-time facial recognition on your way to your Alcoholics Anonymous (or Gamblers Anonymous, or Narcotics Anonymous) meetings at your local church?

Detroit’s usage policy says the department vows not to “violate [the] First, Fourth and Fourteenth Amendments.” But the very existence of facial-recognition-equipped cameras at sensitive locations—such as churches and clinics that provide abortions—risks at least chilling First Amendment–protected activity. And the Fourth Amendment prohibits the government from running general, warrantless searches. Of course, one could argue that Fourth Amendment protections attach to people and their possessions, not to photographs and information that they have already submitted to a government agency (i.e., mugshots or driver’s license photos).

But, as noted above, I would wager that people submit their information to state DMVs with the belief that it will be used for limited identification purposes by the specific entity to which it was submitted, not suspecting that it might become part of an immense database searchable at will by multiple agencies from multiple levels of government. Though mugshots are less voluntary, their subjects still have constitutional rights.

How—and even whether—those rights and privacy expectations will be protected, in America and elsewhere, is not yet clear: there is too little jurisprudence. In 2015, three residents of Illinois filed a class-action suit against Facebook, claiming that the company had violated the Illinois Biometric Information Privacy Act by storing biometric identifiers—in this case, their faces—and subjecting them to facial-recognition technology without their express written consent and without a retention schedule. (The Illinois law requires companies that collect biometric information to obtain a subject’s express permission, to notify people that their biometric data is being stored and how it is being stored, and to let them know why and for how long. Absent a court order, it cannot be stored indefinitely.) Facebook tried to get the case dismissed, arguing that the three plaintiffs had failed to prove any concrete harm resulting from Facebook’s actions; the plaintiffs argued that Facebook’s violation of their privacy rights was itself a substantive harm.²²

The court agreed with the plaintiffs. Facebook appealed to the Supreme Court, which declined to hear the case. Facebook then settled with the plaintiffs for $550 million. Illinois’s Biometric Information Privacy Act is among the more stringent such state laws. But Facebook’s settlement shows that such laws need not be national to be effective. In much the same way that California’s emission standards provide a floor for carmakers across America, perhaps stringent, thoughtful state privacy laws can do the same for tech firms.

Conversely, the High Court of Justice for England and Wales dismissed a case brought by a Welsh civil liberties campaigner who contended that use of real-time facial recognition by the police violated his civil rights. He sued over the collection and real-time processing of images gathered through CCTV cameras, arguing that the practice violated his human rights as well as Britain’s data protection laws. The court disagreed, holding that a “clear and sufficient legal framework” governed the technology’s use and that the police were using the technology in a fair, legal, and nonarbitrary manner. This ruling suggests that police use of facial recognition in England and Wales faces no substantive challenge, and that people who live in those countries will need to accommodate themselves to being constantly tracked and monitored in public.²³

The US Supreme Court’s ruling in Carpenter v. United States, decided in June 2018, gives Americans slightly more protection.²⁴ Seven years earlier, the FBI had arrested Timothy Carpenter based on evidence it obtained by tracking his cellphone. One of his partners in crime gave the police Carpenter’s phone number, and the police obtained a magistrate’s order—not a judicial warrant—that let them track Carpenter’s location and call history.

The Supreme Court held that the government’s warrantless acquisition of a suspect’s cellphone records, which revealed his location and movements over a long period of time, violated his Fourth Amendment rights. A narrow 5–4 majority held that the Fourth Amendment protects not merely a person and that person’s possessions, but also that person’s expectations of privacy. That is a heartening decision, one that could form the basis of a new understanding of privacy rights and expectations in the digital age.

Though it has not yet been applied to real-time facial recognition, it could. Real-time facial recognition contravenes our expectation that the state will not constantly track (and possibly record and store) our public movements, especially when we have not been suspected of a crime. But the sooner someone forces courts to answer whether Carpenter applies to real-time facial recognition, the better: police are not standing still. Officials in both Chicago and Detroit insist that they do not engage in real-time facial-recognition surveillance, but the capacity exists, and public opposition may be the only thing holding police back from using it more widely.

The ACLU of Northern California obtained documents showing that police in Orlando, Florida, had bought—and, at least for a time, used—real-time facial-recognition services from Amazon.²⁵ Amazon has said it will suspend the accounts of any customers found to be violating its terms of service. That vow is welcome as far as it goes, but it is ultimately hollow. There are no laws regarding how police can use real-time facial recognition, including disclosure. The only reason we know about the systems in Detroit, Chicago, and Orlando is that Clare Garvie and the ACLU attorneys took the time to file Freedom of Information Act requests, and then publicized the results.

To its credit, Amazon has endorsed what it believes to be an appropriate set of guidelines for police use of facial recognition. It should “always be used in accordance with the law, including laws that protect civil rights,” wrote Michael Punke, the company’s vice president of global public policy, in a blog post.²⁶ Actual humans should review any results before police decide to interview or detain a suspect, and police should only accept identifications for which the algorithm is at least 99 percent confident. Moreover, police should be transparent in their use of facial recognition, and citizens should know when, where, and how the technology is being used. These are all eminently sensible ideas, but in a free society, citizens’ civil liberties should not depend on companies deciding to develop sensible policies, and police departments deciding they would like to follow them. Legislatures, composed of people elected by and accountable to the public, should debate and pass clear regulations, with swift and enforceable penalties for failing to adhere to them.

Axon, the leading provider of bodycams to police, decided in June 2019 against equipping its cameras with facial recognition. Its AI Ethics Board, an external advisory committee composed of academics, activists, and police officers, felt that the technology “should not be deployed until [it] performs with far greater accuracy and performs equally well across races, ethnicities, genders, and other identity groups.”

The board also declined to answer the question of “whether face recognition on body-worn cameras can ever be ethically justifiable.” Moreover, it said, “no jurisdiction should adopt face recognition technology without going through open, transparent, democratic processes, with adequate opportunity for genuinely representative public analysis, input, and objection.” Again, this is a sensible and principled stance; and again, the civil liberties of citizens in a free society deserve better protection.²⁷

Congress has begun, at last, to agree—or at least to start preparing to agree. In 2019 and 2020, the House Oversight Committee held multiple hearings on facial recognition, and members of both parties seem inclined to regulate the technology. The devil, as ever, is in the details, and Congress will probably not manage to pass substantive legislation before 2021: the presidential election of 2020, the priorities imposed by the Covid-19 pandemic, and the institution’s endemic dysfunction make reaching agreement sadly unlikely. But I would encourage anyone who wants to see sensible federal regulation passed—meaning real restrictions on how long personally identifiable biometric data can be stored, strict limits on who can access facial-recognition databases and why, and real penalties for violating these regulations—to call their congressional representatives early and often.

Some places—such as San Francisco and Somerville,

Massachusetts—have decided that the most reasonable approach to facial-recognition technology is not to regulate it, but to prohibit city agencies from using it at all. I wholeheartedly support these bans, though I don’t think they’re the only way we should combat facial recognition. For one thing, they are unlikely to pass everywhere: most places aren’t as liberal as San Francisco and the Boston area, which both have highly educated, tech-savvy populations.

Matt Cagle, a technology and civil liberties attorney with the ACLU of Northern California who helped spearhead the San Francisco ban, argues that facial recognition is dangerous regardless of its accuracy. Inaccurate facial recognition risks ensnaring the innocent. But accurate facial recognition, when used without viable regulation in real time, risks chilling political expression and association. Some people would be less likely to attend a public protest or political rally—or perhaps to attend services at a mosque, or accompany a friend to an abortion clinic—if the police tracked and recorded them. It should unsettle you, in other words, that Rekognition misidentified a disproportionate number of nonwhite lawmakers. But the breadth and accuracy of Clearview, the facial-recognition app I discussed in the prologue, should also unsettle you.

And even if facial recognition starts out tightly regulated, Cagle contends, it is “vulnerable to mission creep.” He noted that the Wiretap Act has been amended several times to broaden the range of crimes for which the government can intercept communication. As for those who argue in favor of facial recognition in a limited set of circumstances, like the ones approved by the Washington County Sheriff’s Office in Oregon, that still requires building a database, which can easily be used for real-time mass surveillance. “The history of surveillance,” Cagle told me, “is the history of powerful surveillance systems being abused, and the stakes are just higher with facial recognition.”

Perhaps this view strikes you as alarmist. American police do not generally use real-time facial recognition. The technology is improving. Just because Chinese police can find one person in a city of 4.3 million in less time than it takes to drink a cup of coffee does not mean that European or American police will—we don’t do that sort of thing.

But the technology permitting it exists, and abuses often begin as things “we just don’t do”—until we do. For instance, there existed a general understanding that federal agents could not just demand identification from people without probable cause or a warrant. But then ICE agents started boarding Greyhound buses and demanding proof of travelers’ citizenship (technically, they cannot do this without “reasonable suspicion” that someone has broken immigration laws; in practice, it happens). People learned that Border Protection officers have expanded powers within one hundred miles of a border, where two-thirds of Americans live; and they then learned that their “understanding” was really just an expectation.

Without clear regulation, the use of facial recognition could spread in a similar fashion. Those mugshot databases that police currently use just to identify legitimately suspected criminals today could easily fuel real-time facial recognition as soon as police decide they want it. ICE agents could show up at bus and train stations with facial-recognition-equipped tablets looking for undocumented people—and then other federal agents and eventually local police could just as easily do the same, because why should immigration violations receive higher priority than other crimes?

If this is not the world we want, then we need to organize and speak up now. There is no technological solution to these abuses. Those facial-recognition-defeating sunglasses are cool, but not scalable: they may protect you while you’re wearing them, but only when you’re wearing them, and only you—not people you love, or your fellow citizens. There is no rewind button to uninvent things. There is only democracy. We can shape the world we want, rather than just acquiescing to ever-increasing levels of surveillance and decreasing levels of privacy.

Kade Crockford, who directs the Technology for Liberty Program at the ACLU of Massachusetts, called the Somerville and San Francisco facial-recognition bans “a wrench [thrown] into the gears of technological determinism.” She pushes back against “this story that if it’s invented it will be deployed, and you’d better get out of the way,” and says, “It’s up to us to decide whether we think it’s appropriate for governments to use this technology.” This seems exactly right to me. Perhaps more people believe that facial recognition will keep us safe than believe it will imperil us. But whatever we decide, the important thing is that we decide, and do not passively allow our civil liberties to be whittled away more than they already have been in the name of public safety.