As president, I believe that robotics can inspire young people to pursue science and engineering. And I also want to keep an eye on those robots in case they try anything.
US PRESIDENT BARACK OBAMA (OCTOBER 23, 2009), SPEAKING TO STUDENTS IN WASHINGTON, DC, AS PART OF AN EVENT KICKING OFF A SCIENCE EDUCATION INITIATIVE
I, for one, welcome our new robot overlords.
KENNETH JENNINGS (BROADCAST FEBRUARY 14, 2011), USING A REFERENCE FROM THE SIMPSONS TO POKE FUN AT HIS IMMINENT LOSS TO WATSON, AN ARTIFICIAL INTELLIGENCE SYSTEM BUILT BY IBM, IN A SPECIAL TWO-GAME, COMBINED-POINT MATCH ON THE TELEVISION GAME SHOW JEOPARDY
There is little question that Jennings and President Obama had their tongues firmly planted in their cheeks when they made these remarks. The two jokes play upon a time-honored cultural meme: the more intelligent—or perhaps “human”—the machine, the more likely it is to threaten its creators. In the past, these types of references were largely restricted to what we might call “geek culture.” Today, such references are becoming more and more mainstream, reflecting how our day-to-day lives are becoming increasingly intertwined with various forms of artificial intelligence (AI). From the “smart” algorithms that power Internet search engines, to the unmanned drone aircraft deployed by the military, to the GPS devices that direct us to destinations through vocal and graphic commands, more and more specialized tasks are performed by machines or through humans partnering with “smart machines.”
DOI: 10.5876/9781607321705.c03
And though there are no signs we have to worry about a “robot apocalypse”—at least not any time soon—there is a need to better understand the broader social implications of our increasing reliance upon AIs. This chapter explores approaches for studying and theorizing the complex and shifting relationships formed between humans and AI systems. I begin with a brief discussion of why many past efforts have focused on maintaining clear boundaries between human and machines. I then discuss how a “cyborg” approach, which blurs those boundaries, enables a different perspective on human/AI interactions. Using data collected in an ethnographic study of chatbots, I show how using the framework of “cyborg anthropology” helps us to uncover how humans and AIs work together to reinforce various cultural ideologies. Finally, I consider the potential limitations of a cyborg approach and how Donna Haraway’s concepts of “companion species” and “encounter value” provide exciting possibilities for new understandings of human/AI interactions.
Historically speaking, popular and academic writings1 about the cultural implications of AIs have primarily dealt with definitions of the boundary between human and machine. Historian of science Jessica Riskin argues that, in the West, these two cultural categories have long stood in opposition to each other, with each defining and being defined by the limitations of the other (Riskin 2003). For a quality or action to be human, it cannot be replicated by a machine. Major technical advances in the capability of machines to replicate something considered human, in turn, shift our understandings of what it means to be human. For example, it was long believed that because the game of chess involved both mastery of logic and the ability to psychologically engage an opponent, it represented a pinnacle of human intelligence.2 Once AIs began to beat grandmasters, chess was redefined as game in which success “can be reduced to a matter of brute force computation” (Hafner 2002).3 Today’s AI developers now agree that because of the role intuition plays in the game, creating “a strong Go program will teach us more about making computers think like people than writing a strong chess program”4 (ibid.).
Although this shift from defining human intelligence in terms of the ability to use logic to defining it by the possession of intuition might seem trivial, it has important cultural implications. First, it demonstrates how Western cultures work to maintain clear boundaries between our cultural understandings of human and machine. A significant amount of science fiction is built on the profound discomfort that arises when one can no longer tell the difference between the two. Sigmund Freud argued that the root of this feeling of dread, which he termed “the uncanny,”5 runs far deeper than simply being confronted by something one cannot be sure is human. He argued that in that moment, our culturally established “clean” understanding of the difference between man and machine is destroyed and must be re-created and recalibrated. In that moment of breakdown one can become “unsure of his true self” (Freud 2003, 142) and begin to question has machine become man, and am I no more than a socially programmed machine?
In addition to existential implications, there have been policy rationales for clearly demarcating man and machine. For example, by the 1980s an increasing number of academics and industry researchers had become concerned with the considerable amount of funding and attention being dedicated to the reproduction of human intelligence. These individuals successfully argued that most AI projects failed to understand the complexities of human intelligence (Suchman 2007; Winograd and Flores 1986). As a result of these efforts, research moved away from attempts to directly reproduce human intelligence and toward focusing on more algorithmic tasks such as data retrieval and organization.
These examples demonstrate the importance of being attentive to the boundaries between humans and machines. However, there are limits to this approach. Consider one of the best-known arguments against thinking computers: “the Chinese Room” thought experiment created by John Searle. Imagine a person sitting in a room filled with instructions and pieces of paper with seemingly random marks—these marks are actually Chinese phrases, but the occupant cannot read them. When a note with different unreadable markings—actually a phrase in English—is passed into the room, the instructions tell the occupant which piece of (Chinese) paper to pass back to their unseen interaction partner. Even though the room appears to be “fluent” in English and Chinese to someone interacting with it, the operator inside the room is simply following a set of specific instructions and never understands the content of what is being translated. Searle concluded that an AI, like the Chinese Room, could only follow directions but never think, that is, reflexively understand what it was doing.
Searle’s argument demonstrates how attempts to define hard boundaries can gloss over the rich complexities of interactions between humans and machines. In part, this is an issue of shifting one’s perspective. To make his argument, Searle uses what Haraway terms a “view from nowhere” (Haraway 1991b, 188–90)—an omniscient position, which allows him to simultaneously see and know everything going on within the encounter. In the real world, we can never actually occupy this position. Like the person interacting with the Chinese Room, we are situated within the interaction, knowing only what we have learned in the past and what we gain through the interaction. From Searle’s view from nowhere, we see that the room is only following instructions. But the interaction partner, being within the interaction and thus having a limited knowledge position, leaves believing that the Chinese Room accomplished the translation because it understood the contents of the message. Although it is true that the Chinese Room does not think like a human, it is also true that, based on the interaction, the person believes that it does think like a human. And a belief that the Chinese Room—or any AI—is intelligent can potentially wield as much cultural influence, if not more, than the “truth” that the Chinese Room is not intelligent.
Thus, although we should not dismiss the issue of the boundary between human and machine, there is much to be gained by an approach that transcends it.
A cyborg is a cybernetic organism, a hybrid of machine and organism, a creature of social reality as well as a creature of fiction ... By the late twentieth century, our time, a mythic time, we are all chimeras, theorized and fabricated hybrids of machine and organism; in short, we are cyborgs ... The cyborg is resolutely committed to partiality, irony, intimacy, and perversity. It is oppositional, utopian, and completely without innocence.
DONNA HARAWAY (1991A), “A CYBORG MANIFESTO: SCIENCE, TECHNOLOGY, AND SOCIALIST-FEMINISM IN THE LATE TWENTIETH CENTURY,” PAGES 149–151
In the mid-1980s, Haraway argued that the cyborg could serve as a powerful metaphor for rethinking our understandings of sociocultural relations, transcending oppositional thinking, and subverting oppressive cultural institutions.6 The promise of a cyborg system, which can be an individual or made up of linked individuals,7 is that it sees no contradiction in being literally and figuratively assembled—or, in Haraway’s words, “coupled”—from parts traditionally held in fundamental opposition to each other (e.g., human vs. machine, man vs. woman, human vs. animal). Never complete, cyborgs exist in a state of constant upgrade, pragmatically adding and removing parts as needed. And because all parts exist in dynamic relation to each other, any addition or removal shifts the understanding of every part and the aggregate whole. Full of contradictions and in a state of constant change, the cyborg is always many things at the same time, making it a powerful tool for understanding culture’s messy complexities.
Using cyborgs as an analytical metaphor means focusing on seeing things as systems within systems. For example, despite the fact they can be physically separated, the person and the Chinese Room become part of a cyborg system through their interaction. At the same time, the person and the room are also systems unto themselves, constructed of different physical and social parts and connected to other social actors. However, we should resist—at least temporarily—our desire to break apart and categorize cyborgs. In analyzing, we must think in terms of “and” rather than “or.” Instead of trying to establish pure “truths”—for example, is the Chinese Room intelligent or not—we concentrate on how different and contradictory aspects and understandings of the room can coexist without negating each other.8
In their 1995 essay “Cyborg Anthropology,” Gary Lee Downey, Joseph Dumit, and Sarah Williams “coupled” cyborg analysis to anthropological methods and concepts to create an analytical framework founded on the belief that “human subjects and subjectivity are crucially as much a function of machines, machine relations, and information transfers as they are machine producers and operators” (ibid., 266). Rather than primarily focusing on how humans create machines, a cyborg anthropological approach adds a deep consideration of how technologies are agents in reproducing social life (ibid., 267). This is accomplished through examining the couplings and contradictions that make up the cyborg in question. Once we understand its construction, we then turn to exploring how, through its interactions with others—including other cyborgs—contradictory understandings of our cyborg come to affect and transform cultural and social systems.
In more anthropological terms, we must examine the ideologies that underlie both the creation of cyborg systems like AIs and the ways in which the cyborgs themselves represent and transform ideologies. Ideologies are clusters of interrelated and internalized ideas, beliefs, and values about different social phenomena. Culturally constructed and individually expressed, ideologies condition how we perceive and interact with the world. We consciously and unconsciously learn ideologies through interactions with individuals and cultural institutions and artifacts. Ideologies are continually reshaped (and also reinforced) in the same interactions that they condition. Using ideologies as analytical tools, anthropologists organize beliefs under different labels to uncover and describe differences between specific cultural groups and the flow power within a culture.9 To find ideologies, an anthropologist examines the sites where cultural beliefs and values emerge and are reproduced, from media artifacts (books, magazines, movies, games, and websites) to interactions between individuals and groups. For example, the idea that intuition rather than logic is the correct marker for “true” intelligence represents a shift in what we might term an “intelligence ideology.”
Now let us turn to my first data set, and see what is revealed when one applies cyborg anthropology to the study of chatrooms and sexbots.
I met Az_Tiffany10 in a Yahoo! chatroom while recruiting webcam users for ethnographic interviews during spring 2005.11 Tiffany’s profile described her as a college student who put on live pay-per-view webcam shows through her website. In hopes of getting an interview, I private-messengered (PM’d) Tiffany and, after a bit of negotiation, offered to join her website in exchange for an interview. She agreed and I left to sign up. After paying the one-day membership fee, I discovered that the site was a façade. Attempts to log in redirected me to a different webcam site where there was no mention of Az_Tiffany. Back in the chatroom, I PM’d her for an explanation. She responded but did not address my questions. In fact, she acted like we had never chatted. All at once the embarrassing truth of the situation dawned on me. A quick scan of our archived conversation confirmed my suspicion: Az_Tiffany was not a cam girl; she wasn’t even a person. Az_Tiffany was a chatbot, a computer program designed to appear human (Leonard 1998, 7). A fake woman got me to join to a scam webcam site. I had been “pwn’d” by a machine.
Chatbots, like Az_Tiffany, operate within chatrooms and private messaging systems, engaging other users in conversations. First developed by academics and hobbyists/hackers, the earliest versions of chatbots began to appear in Internet relay chat (IRC) and multi-user dungeons (MUDs) during the late 1980s. Beginning in the 1990s, as websites—in particular, adult ones—began to pay “finder’s fees” for new members (in some cases more than $40 a head), a new, gray-market application for bots arose as entrepreneurs began to see chatrooms as a seemingly limitless source of easy, ongoing income.
At their core, bots and other conversational AIs are made up of two basic components: a program and a script. The program contains all the software commands the AI needs to take action in the world, including sensing and speaking. The script can be thought of as the bot’s self. Authored separately from the program, the script is the AI’s “personality,” a repository of trigger words, responses, and the specific rules that will govern its behavior. A single program can be used to animate any number of personalities qua scripts, although typically only one at a time. On websites and discussion forums where adult webmasters gather to discuss business, the general consensus is that everyone should write their own scripts—as one poster put it, your bots need to be unique and sound “real ... [otherwise they] won’t catch anyone”12 (Edwards 2004).
The bot is an excellent example of how complex AI systems are. Examining it from a cyborg perspective, we see how the bot is assembled from multiple components (program and script) by multiple people (programmers and authors). Drawing on Erving Goffman’s model of the production of speech in his essay “Footing,” the program is the “animator,” literally a “talking machine” (Goffman 1979, 17), and the proverbial peg on which the script-as-self will be hung. And although, in an average conversation, the person who is animating those words is typically their “author” (ibid.), this is a case where these functions fall on different actors. The bot can be thought of as a cyborg system whose “parts” are never all copresent at once, although each has left a trace and can continue to have an effect on the system. The programmer did her work before the author of the script. And when the bot finally has the chance to communicate, like an actor in a movie or play, there is no need for the author to be present for the show to go on.
All of this building work must necessarily take place and be completed long before AIs can engage in a “live” encounter. Faced with the vast range of conversational possibilities, successful authors bound and control the conversation by creating scripts that perform contextually appropriate speech genres. Russian literary theoretician Mikhail Bakhtin wrote that “each sphere of [human interaction] contains an entire repertoire of speech genres that differentiate and grow as the particular sphere develops and becomes more complex” (Bakhtin, Emerson, and Holquist 1986, 60). By deploying the appropriate contextual cues the author can recruit interaction partners who, knowingly or unknowingly, perform and inhabit the participant roles of that genre. Because a speech genre reflects beliefs about how a type of person should speak, it is also an expression of ideologies. Through the selection of specific words and phrases that will make up the script, authors present us with their ideological beliefs about how those cultural categories combine to create a “real” webcam performer13—or the author’s beliefs about the expectations that other chatters have about the performers. Thus, by studying the bot’s profile, the words it uses for a script, the way those words are spelled, and other interaction markers, we begin to see how the bot embodies ideological understandings of gender, sexuality, language, and online behavior.
One limitation that must be addressed during the creation of the script is the bot’s inability to parse and react to individual utterances within the chaos of a chatroom. Like the Chinese Room, the bot does not understand the meaning of anything that is being said in a chatroom. This means that the bot has no way of knowing if it is being addressed by another chatter. If it was allowed to participate in the chatroom conversation, the bot would respond to every post made in the chatroom, regardless of whether it had anything to do with the bot. The result would be an excessive “spamming” of the chatroom, immediate revealing that it is a bot.
To work around this limitation, authors simulate direct interaction with chatters within the shared space by taking advantage of established chat practices. For example, upon entering a new chatroom the bot Codi_1984 always began with the common chat greeting “A/S/L” (age/sex/language) followed by a pause in action. The bot’s author hopes that during that pause an unsuspecting chatter will respond with his A/S/L. Should that happen, her14 next utterance, “Hi, I’m Codi ... 21 and single at USC,” makes it appear as if she were replying in turn. After taking steps to establish the bot’s identity, the well-planned script encourages chatters to initiate a private messaging session with the AI.
Bot workflow.
In the one-to-one encounter, the bot, or its creator, no longer has to worry about whether it is being addressed because there can be only two participants in a private messaging session. The bot can react to its interlocutor, using a workflow similar to what is charted in Figure 3.1. The interaction begins with the human projecting a new message (Step 1). The bot captures the utterance (Step 2) and searches for keywords (Step 3). Recognizing “what up,” it follows the associated rule, randomly choosing a reply from a list of three possible responses and then projecting that response—“The Sky - Lol!”—into the PM conversation (Steps 4, 5, and 6). The pair-set is completed, and the bot waits for the next utterance from its interlocutor.
If the bot appears too automatic or mechanical, the encounter will fail. So although the bot can move through this workflow instantaneously, most have a delay built into their response mechanism, making it appear as if they are “thinking” and “typing.” Bot authors also play with grammar, use Internet slang and txt spellings, and simulate common typing errors, intentionally misspelling words and transposing letters. Each of these decisions is the expression of an ideological understanding of what it means to participate as a human in a chatroom.
If the bot’s script immediately pushes the webcam site, the bot will lose the chatter. However, bots are currently not sophisticated enough to maintain a lengthy convincing conversation. Although they can archive conversations, bots cannot access that archive in the midst of a conversation. When it comes to participating in conversations, chatbots “forget” what they said as soon as they say it. The longer the conversation goes on, the more likely the bot is to start repeating itself or repeatedly ask the same question. The challenge for an author is to create a script that is neither too short nor too long to get the job done.
Looking at the first twelve exchanges from my conversation with Az_Tiffany, one sees how her author created her script in such a way that the different utterances assembled into a narrative flow that moved the conversation toward the ultimate goal of getting me to join her website (see below). After exchanging greetings, she asked if I was interested in viewing webcams (a3 and a4). In a7, she stated that she was going to be going “on cam” soon. Next she informed me that she was getting lots of PMs (a8). Finally, the narrative came together in the twelfth exchange (a12).
(m1) MattB: hey ... we talked earlier today ... I’m a researcher and I’d like to interview you. In return for the interview I’ll join [your] site. If you’re interested contact me at [my e-mail address]
(a1) Az_Tiffany: yah
(m2) MattB: hey
(m3) MattB: didn’t realize you were still online
(a2) Az_Tiffany: hiya
(a3) Az_Tiffany: are you viewing any cams right now?
(m4) MattB: not yet.
(a4) Az_Tiffany: are you viewing any cams right now? [repeat of a3]
(m5) MattB: not at the moment.
(a5) Az_Tiffany: so are you into cams?
(m6) MattB: not really
(a6) Az_Tiffany: ah ok
(m7) MattB: but I’m actually studying webcams for a project
(m8) MattB: actually my thesis
(a7) Az_Tiffany: sorry to change the subject but my break is almost up at this new cam thingy im doing, so like come and check my cam out if you want to
(a8) Az_Tiffany: ooomg im getting so many [IM]’s!!
(m9) MattB: are you interested?
(a9) Az_Tiffany: ok
(m10) MattB: hmmm so here’s the deal ...
(a10) Az_Tiffany: are you viewing any cams right now? [repeat of a3, a4]
(m11) MattB: I’d like to interview you as part of my project
(a11) Az_Tiffany: yah
(m12) MattB: so how do I get to your cam?
(a12) Az_Tiffany: hon im getting bombarded with [PM]’s so im just gonna go on my cam chat network ... if you wanna come chat with me its a good time, just go to [Tiffany’s website URL] make a chat name and sign in ’k?
Whenever the bot was faced with an utterance that she did not understand (m3, m4, m7, m8, m10)—cases in which I wrote something that the bot did not have a specific answer for—the script responded in a way that unobtrusively advanced the conversation toward the eventual “call to action” (a12). Relying on the interlocutor being firmly entrenched in his genre role and caught up within his ideological expectations of the encounter, each of these “move-along” utterances added and reinforced the bot’s narrative and identity. I was so convinced of the reality of my chat partner and so focused on getting an interview with this “girl” that I glossed over a number of flags, including the repetition (a3, a4, a10) of the specific question “are you viewing any cams right now?” It was only upon discovering I could not access the site that my interaction frame shifted. Upon reexamining the archive of the conversation, I saw the seemingly obvious clues in the conversation.
In keeping with the attentions of cyborg anthropology and now understanding how the bot was created and works, we now shift to examining interactions between bots and human chatters. The relative ease of production and the lucrative potential has led to Yahoo! chatrooms being “overrun by bots,” as one chatter described it. It is not unheard of to have upwards of ten bots in a chatroom at once. Multiple bot postings often interrupt the existing flow of conversation within a room, leading to frustrated outbursts from chatters like “damn bots!” or “is there anyone within this room who isn’t a bot?!” Beyond this sense of general frustration, what can be gained from these encounters with chatbots? The obvious answer might be financial gain for the bot author; however, there is little evidence to suggest that most bot authors see much, if any profit, from their creations. Based on interviews and postings made to discussion boards, authors admit that their efforts are driven by the hope of profits rather than the realization of them.15
Beyond any profits they generate, bots reinforce and change the ideological understandings through their interactions with other chatters. If a person chats with a bot and never realizes it, that individual’s beliefs about webcam performers are reinforced through the interaction. In other words, their preexisting ideological framework is confirmed by the encounter. But what happens if the encounter fails? What if, as in my case, the chatter discovers the bot? Take, for example, the following chatroom exchange in which the chatter “fly_me” quickly transitions from trying to engage a “girl” in conversation to warning the room of a bot in their midst:
(g1) goofygal: does anyone here have a mic [for an audio chat]?
(f1) fly_me: sorry goofy ...
(g2) goofygal: i just got a mic and made a sound clip if anyone wants to hear it ?
(f2) fly_me: what format? goofy?
(g3) goofygal: the clip is in my profile
(g4) goofygal: does anyone here think i sound like Jessica Simpson, that what people tell me
(f5) fly_me: dirty trick goofy
(f6) fly_me: goofys clip is a link guys ... shes a bot
Having discovered that he misrecognized a bot as human, fly_me scolds the bot (f5) and then outs it (f6). Unlike my encounter with Az_Tiffany, fly_me made this mistake in front of other chatters. By taking retaliatory action, fly_me attempts to shift the chatroom focus away from his mistake, presenting himself as a conscientious chat citizen outraged by the bot’s presence. His remark “dirty trick goofy” publicly repositions his relationships with the bot, suggesting that the bot was cheating, playing outside of the accepted rules of the chatroom. Together with the warning (f6), fly_me signaled—intentionally or not—that he won’t be fooled again and that he wanted to make sure no one else gets tricked by goofygal.
There was good reason for fly_me’s public performance, because many chatters have little sympathy for anyone caught misidentifying a bot. The following is an excerpt from one chatroom’s reaction to a chatter named “GalaxyCat,” who was caught publicly flirting with a bot:
Jack-0ff: Look at the responces you were getting ... YOU JUST CHATTED UP A BOT !!! Bwahahahaha
GalaxyCat: Like Im supposed to know that its a fucking bot ...
A-Ron_1980: gimme a fucking break ... you got owned by a bot lol
Mariner-Fan: GalaxyCat, you’re so full of shit. Its one thing to fuck with someone, its another to make yourself look like a complete retard.
MC-FlyGrrl: You got nailed by a robot, bitch
Even after GalaxyCat had fled the room, the taunting continued. Eventually, a chatter questioned how anyone could mistake bots for humans. In response, another wrote, “Don’t worry, only dumb shits like GalaxyCat ever fall for bots.” Although it is often said that reactions are bigger on the Internet, the severity of this sort of attack can also be unpacked as having to do with the violation of cultural boundaries.
For the uninitiated, the discovery that bots, disguised as humans, are interacting with users within the chatroom undercuts foundational assumptions about the chatrooms and creates an environment ripe for conflict and feelings of the uncanny. We can read the actions of fly_me and the chatters who attacked GalaxyCat as attempts to reestablish and redraw the boundary between machine and human.16 Likewise, in discussions about why they do not use bots, a number of adult webmasters and performers shared explanations that reinforced that boundary. One performer I interviewed told me that she would “never use a bot ... they just annoy people and make our job [of finding people who’ll pay] harder ... They lack the human touch.” When I asked her what she meant by the “human touch,” she replied, “You know ... listening, imagination.”
Ask a bot if it is a bot, and it will always respond with a denial. Some even make public posts like “I can’t believe there are so many bots in this room!” and “If everyone complains to Yahoo, we’ll get these bots kicked out.” Contained with these various protests is the inherent claim of “humanness.” And embedded within that claim is another one: “humanness” can be mechanically/digitally reproduced. Through its performance as “human,” intended to hide its mechanical self, the bot becomes the personification of that second claim. In this way, these bots—cyborg couplings of programs, scripts, and authors—have a somewhat adversarial relationship with human chatters. When chatters reveal a bot in the present, the seeds are sewn for the next generation of more “human” bots. Many bot authors constantly review the transcripts archived by their bots to revise the scripts, addressing the flaws that gave the bots away. Once again we find human and machine constantly redefining each other, with one side establishing what it means to be—or rather perform—“human” within the chatroom, and the other side working to make their creations meet that definition, only to have it shift again.17
For all the benefits the metaphor of the cyborg offers us, it has some potential pitfalls. We must recognize that not all couplings work. Throughout “The Cyborg Manifesto,” Haraway emphasizes that the cyborg always chooses to couple (Haraway 1991a, 150–1). Although human chatters and sexbots interact and produce various economic, emotional, and ideological results, their adversarial relationship should be taken as a sign that it could be a mistake to join the two as cyborg. To do so would assume that any interaction—be it good, bad, or indifferent—between the two parties represents a coupling. Such a position ignores the issue of choice, expanding the idea of coupling to the point that everything in the world immediately becomes one giant cyborg. Such an argument certainly can be made, but we have to ask what making such a move, from an analytical and political perspective, would accomplish.
When analyzing a cyborg system, we also must remember that cyborgs are often recursive—systems within systems. To that point, we must recognize that few couplings are total. The sexbot is a cyborg made up of a network of actors and parts, but the fact still remains that the bot also functions as an individual actor, separate and separable from its creators. We, therefore, need ways of being attentive to how different parts function within and outside of the cyborg system. In other words, like the cyborg, we need to find ways to upgrade our analytical tools.
Haraway’s recent writings provide an excellent framework for giving the actors who make up the cyborg their due. Of particular value are the concepts of “companion species” and “encounter value,” which she has developed through her writings on animal/human relations.18 Haraway defines “companion species” as groups of beings that have coevolved in relation to each other (Haraway 2003, 6–7). For example, in the process of domesticating livestock, humans altered the development of various species through selective breeding. At the same time, the integration of domesticated livestock into societies fundamentally transformed human culture. We have already discussed how humans and AIs have relationally redefined each other. And, as noted in the introduction, the proliferation of AIs is also transforming how we go about our daily lives.
Haraway notes that despite the familiarity gained through coevolution, we must recognize that we can never completely know a member of a companion species. Although we can communicate across species, things are always lost and gained in translation. This note of caution is also applicable to AIs. The nature of their production creates an illusion that they are fully knowable, but their creators admit this is rarely, if ever, the case. One AI programmer explained to me: “I am always astounded by my [AI’s] ability to surprise me. I mean, I programmed it ... every line of code. I should be able to predict all of its behavior. And then for whatever reason, it does something completely unexpected. And not always wrong. I know its still following its program—but it isn’t what I thought I programmed.”
Building from the concept of companion species, Haraway extended Karl Marx’s categories of use and exchange value to include what she terms “encounter value” (Haraway 2007, 46). Use and exchange value function as abstract mediators, transforming objects and living things into economic commodities. For example, a cow becomes how much it costs to buy and maintain. It is measured by how many gallons of milk a year it produces. That milk, in turn, is understood in terms of how much it can be sold for. When the cow ceases to produce milk/profit, it can be sold for slaughter. Encounter value, Haraway’s alternative, is an attempt to represent the more intangible forms of value that are created through interactions between, and in the entwining of, companion species. Returning to the cow, its encounter value is understood—versus calculated19—to be based on how it contributes, as a living component, to the broad ecosystem of encounters that produces and sustains what we call a “farm.”
AIs are easily reducible to how much they cost to program, maintain, and operate; the value of the hardware that they run on; and how much power and other resources they consume. But, as with the cow, much is lost in an economic analysis of AIs, especially as if we shift our focus from AIs that exist in an adversarial relationship with humans to those that operate in a more cooperative fashion. To highlight how these two concepts can supplement a cyborg anthropology analysis, we move from the chatroom to a project in which AIs were used in encounter therapy.20
In the mid-2000s a team of researchers at Northwestern University’s Articulab, led by Justine Cassell, ran a series of experiments in which embodied AIs called Virtual Peers took part in a therapeutic treatment for children with autism spectrum disorders (ASDs) (Merryman et al. 2008; Tartaro 2007; Tartaro and Cassell 2006, 2008). Initially developed to assist young interlocutors develop literacy skills through collaborative storytelling, the Virtual Peers were cyborg systems that manifested as three-dimensional, life-size, animated children (Cassell 2004; Cassell et al. 2000; Ryokai, Vaucelle, and Cassell 2002).
The particular Virtual Peer avatar used during the autism treatment was Sam (pictured in Figures 3.2 and 3.3). Projected onto a screen, she talked with her interaction partner using prerecorded vocal utterances and prescripted movements.21 Her partner’s responses were monitored in different ways, including via microphones and motion sensors. Sam’s cyborg body also included a dollhouse (Figure 3.2) that sat in front of the screen onto which she was projected. The dollhouse and its contents were literally wired into the system using radio-frequency identification (RFID) tags and sensors, allowing Sam to “know” where each doll was in relation to the house (Cassell et al. 2000; Tartaro and Cassell 2006; Tartaro and Cassell 2007). The RFIDs embedded within the dolls allowed Sam’s head and eyes to follow their movements, making it appear that she was looking at her interlocutor. The RFID also enabled the dolls to “crossover” into the virtual world. When a doll was placed in a “magic” compartment within the roof the dollhouse, a representation of it would appear in Sam’s world for her to play with (Tartaro and Cassell 2006, 3).
Child interacting with Sam (Tartaro 2007, 1678, fig. 1).
Although Sam was theoretically capable of autonomous interactions with human interlocutors, the system could not reliably navigate the complexity and nuance of person-to-person interactions (Cassell and Tartaro 2007, 398). In other words, Sam was not capable of the thick interpretation (Geertz 1977) necessary to discern if a pause was because the child was thinking of what to say next, was frustrated, or simply remaining obstinately silent. Therefore, Sam was typically operated by a human using a computer running what “her creators dubbed the Wizard of Oz” (WOz) interface (ibid.).22
Despite the reliance on a human operator, it is a mistake to label Sam simply a puppet or metaphorical mask for her operator. The operator often sat in a separate room and followed the interaction through Sam’s senses—the system’s monitoring devices. The operator also had to learn to work within Sam’s limited repertoire of responses. Many of Sam’s reactions—such as following the doll with her eyes—were determined by the system, not the operator. And when Sam responded, it was not with the operator’s voice or words but her own, which is to say the words recorded for her by the vocal actor.23 From the perspective of cyborg anthropology, we can say that, in the encounter between Sam and the children, the operator was temporarily folded or coupled into the cyborg body of Sam.
Sam (Tartaro 2007; image released by Northwestern University, Articulab website: http://www.articulab.justinecassell.com/projects/samautism/index.html).
The goal of these experiments was to see if virtual peers could be used to help high-functioning children with ASD develop peer-interaction skills, in particular, the ability to contingently reply to utterances in communications rather than repeating themselves or responding with non sequiturs.24 The rationale for this therapy was based on previous successes using Virtual Peers to assist children in developing storytelling and other early language skills. Cassell and team were also building on evidence that peer-to-peer interaction therapies can develop conversational skills in autistic children (Kalyva and Avramidis 2005) and findings that children with autism prefer interactions with computers to interactions with peers (Goldsmith and LeBlanc 2004).
A therapeutic encounter with Sam began when the child approached the dollhouse. Sam introduced herself and asked the child to do the same. Sam then invited her partner to help act out a co-created story using the dolls. The following is a summation of one such encounter:
At the beginning of the interaction, Sam starts a story about two children playing hide-and-seek. Sam trails off part way into the story, indicating she isn’t sure what happens next. Claire [the child interacting with Sam] does not at first pick up the trail of the story, but when Sam says, “Then what happens[?],” Claire begins listing items, “Then, they saw a tree. (Sam: Uhhuh.) And a fence.” With some prompting from Sam through backchannel and directed questions, Claire begins participating in the interaction, but her utterances are not contingent with the content of Sam’s story. Over the course of the interaction, her responses increasingly make sense within the context of the story. In her last story, she helps Sam tell a story about a boy and a girl baking cookies for the boy’s grandmother who is sick in the hospital. She says, “They got the ingredients ... And the recipe book ... Baked chocolate chip cookies ... Grandma’s cookies.” (Tartaro and Cassell 2008, 5)
For Cassell and Tartaro, the therapeutic potential of Sam and the other Virtual Peers are very much tied to their artificialness. For example, they refer to Sam as “indefatigable” (Cassell and Tartaro 2007, 403; Tartaro 2007, 1679; Tartaro and Cassell 2006; 2008, 383). Unlike a human child, Sam has limitless patience and her attention will not wane. And beyond partnering with Sam to build stories, the child could also use the WOz interface to control Sam in interactions with other children, a therapeutic technique used once the ASD child has demonstrated improvement in contingent conversation. Although Sam was designed to resemble a child, her value was also derived from the ways that she was explicitly not like a normal child. In other words, her “artificialness” was as important as her “humanness” in creating a successful encounter.
At the heart of that encounter between Sam and a child was the notion that the two must meet as peers, as equals working toward common explicit and implicit goals—in this case, the joint telling of a story and the development of new interaction skills. Cassell and Tartaro write about a “rapport” that must develop between the child and the AI over the course of an interaction to achieve successful therapeutic results (Cassell and Tartaro 2007, 403–4; Tartaro and Cassell 2007, 235–6). This notion of rapport brings us back to encounter value. Cassell and Tartaro described that rapport as being expressed through various signals like sustained eye contact, conversational coherence, and expressions from the children such as “[Sam] reacts to us!” Based on those criteria, the sessions were seen as yielding positive results, with children showing improvements in the areas of attention and conversational contingency after interactions with Sam (Merryman et al. 2008; Tartaro and Cassell 2006, 2008).
Those successes represent an overflowing of the encounter between child and virtual peer, the creation of something greater than the individual parts that persists after the encounter has ended. Generally speaking, all parties must be actively engaged within a therapeutic session. The goal of therapy is to achieve a change that could not occur on its own. Success is therefore a coproduction, something that all share responsibility for. And in the end, that success can be seen as being greater than the individual contributions of each party. A positive change in behavior—the manifestation of the encounter’s value—can only be realized when the child and Sam, although they meet as members of different companion species, productively interact with each other as peers.
This chapter discussed how cyborg anthropology can be used to study the ways in which AI and human cultures are becoming increasingly intertwined. This included focusing on how AIs and humans exist in relation to each other, attending to how conversational AIs come to embody certain ideologies and how, in their interactions with humans, bots reinforced, challenged, and transformed cultural beliefs and understandings. I finished by discussing the benefits of joining the concepts of companion species and encounter value in cyborg anthropology to ensure we remain attentive to limits of coupling and the fact that the parts of a cyborg often have the power to act outside of the system that they are joined to.
In the case of the bots in Yahoo! chatrooms, following cyborgs led us back to the familiar juxtaposition of humans and machines and provided another example of how the boundary between the two is constantly contested and shifting. At the same time, we can look to Sam and her human peers as an example of a situation in which that boundary is never in question. Sam’s creators never attempted to portray her as a living, breathing human. And yet, although she was not human, the children who worked with Sam greeted her as a playmate and a peer and referred to her as a “friend.” It is true that children have long made friends with imaginary and inanimate objects, but I cannot help but wonder as I type this if there is something different about a building friendship with an AI like Sam. But I cannot yet say what it is that feels unique about forming such a bond with a digital companion species.
I, however, am sure of one thing. In 2004, when I began this research, I had trouble finding colleagues who knew what a bot was. By the time the first draft of this chapter was submitted, all of my friends were using GPS units instead of maps; robots were beginning to clean waste, deliver food, and dispense drugs in hospitals in the United States, Japan, France, and Scotland; and the New York Times Magazine had just begun an extended series of articles called “Smarter than You Think,” chronicling the sociocultural effects of recent developments in artificial intelligence. Just before I began edits on the final draft, Watson beat the two best Jeopardy players in the world, and the “Terminators” that dominate public discussions are unmanned Predator drone aircrafts. You do not need to be a social scientist to know these signs suggest plenty of opportunities, and an ongoing necessity, to study human/AI encounters for years to come.
1. Examples include everything from historical research into the philosophical foundation of modern AI projects (Riskin 2003), to deconstructing how teams approach the process of planning and developing an AI and how those decisions shape the overall project (Forsythe 1996; Suchman 2007), to explorations of various ideologies and definitions of “intelligence” and “life” (Forsythe 1993; Helmreich 2000; Riskin 2003; Suchman 2007).
2. To appreciate chess’s historic cultural significance as an index of intelligence, one need look no further than media discussions of Cold War matches between American and Russian players and, in particular, the 1972 World Chess Championship match between Bobby Fischer and Boris Spassky. Media in both countries positioned each competitor as playing to prove his nation’s inherent intellectual and ideological superiority.
3. Similarly, following Watson’s win on Jeopardy, many commentators noted that the game’s focus on trivia made winning largely a function of a “mechanical” ability to efficiently organize, store, and retrieve data.
4. An ancient Asian game in which two players—one using black markers, the other white—compete to acquire and defend territory on a game board.
5. Roboticists and designers use a slightly different articulation of “the uncanny” in discussions about how an artificial representation creates a heightened sense of revulsion in those it interacts with as it becomes outwardly more human (Mori 1970). This concept, termed “the Uncanny Valley,” was wonderfully articulated by Judah Friedlander’s character on the television program 30 Rock who pointed out the difference between R2-D2 and C-3PO, the “nice” robots in Star Wars, and the “scary,” dead-eyed characters of the computer-animated children’s film The Polar Express (“Succession,” 30 Rock, 2008).
6. It is important to understand that the cyborg, like most of Haraway’s theoretical contributions, was intended to be a tool applicable to both activism and analysis; Haraway would arguably see no difference between the two. It was developed as a response to contemporary debates circulating among political activists and in academic disciplines including science studies and feminism.
7. In addition to the classical cyborgs of science fiction, like the Terminator, a single body made up of both organic and mechanical materials, Haraway’s cyborgs can include more metaphorical assemblages like a community action group made up of people from all categories of a society who come together to accomplish a specific task (Haraway 1991a, 154–5).
8. In the cyborg, there is no assumed equivalence or naïve relativism. Two contradictory truths might coexist, but that does not mean that they are equal or should be equal. Haraway explicitly states that the cyborg is “never innocent” and must recognize inequalities both outside and within itself (Haraway 1991a, 150–1).
9. Although people who hold a given ideology may think theirs is superior to others, from the analytical perspective of linguistic and media anthropology, ideologies are never inherently moral or immoral, true or false.
10. All usernames, human and bot alike, have been altered for this section. Beyond altering the identities of chatters, all quotes from chatrooms and discussion boards are reproduced “as is,” with no corrections to spelling or grammar.
11. The primary fieldwork for this part of the chapter was conducted in Yahoo! chatrooms during spring 2005 and supplemented with irregular visits to chatrooms over the year that followed. This entire section will be situated in that ethnographic present. Yahoo! was chosen because, at the time, it was the largest chat network where memberships were freely available to any individual with Internet access. Within the chat network, the majority of my time was spent within rooms designated as general discussion areas, where there was no specific restriction or guidance for the subjects discussed. When I encountered a chatter that I believed to be a bot, I engaged it in a PM session and record the ensuing interview for later analysis. Supplemental observations and interviews were also conducted on a number of Internet discussion boards where administrators of adult websites meet to discuss various aspects of the online pornography business.
12. Words like “real” or “true” often signal when someone is making an ideological claim.
13. In this chapter I only use examples of bots performing one specific genre of webcam performer—the “sexy co-ed.” In my fieldwork, I encountered numerous other types of bots, including “male” ones that frequented gay chatrooms. All the bots played a specific genre of sex performance.
14. Rather than using “it” to refer to the gender-neutral technology of the AI, I keep with long-standing cross-cultural linguistic practices that treat gender as a performance (Kulick 1997) and thus typically refer to the AI according to the gender it performs.
15. Generally speaking, it was unlikely for a single bot to bring in more than a few dollars a month. Even with more than ten bots operating at once, the average “successful” bot-master still makes less than $50 a month, or, as one creator put it, “enough to buy beer.”
16. See Riskin (2003) for an extended consideration of the processes by which that boundary has historically been crossed and reestablished through sociocultural interactions with various forms of artificial intelligence and artificial life projects.
17. As with the example of the games of chess and Go, we see with bots how the test that defines the boundary between human and machine also becomes the goal that must be overcome. Phillip K. Dick’s Do Androids Dream of Electric Sheep? and its movie adaptation, Blade Runner, provide an excellent explanation of the drive to constantly surpass the boundary. In the book and the movie, the test used by the police to identify androids is constantly threatened by the company responsible for manufacturing the androids. The rationale the company offers in the book is that they are not intentionally trying to undermine the police. Instead, they are only doing what their customers want: “[Following] the time-honored principle underlying every commercial venture. If our firm hadn’t made these progressively more human types, other firms in the field would have” (Dick, 1996, 54).
18. Haraway’s writings on companion species deal with profoundly complex ethical issues and concerns surrounding the treatment of animals. I want to be clear that my use of these concepts, in particular, the application of encounter value to human/AI interactions, is not an attempt to draw moral or ethical comparisons or equivalencies between machines and animals.
19. Unlike use or exchange value, encounter value fiercely resists translation into an economic formula.
20. The research on the Virtual Peers project was largely archival, supplemented by short e-mail exchanges with Cassell’s collaborator and coauthor, Andrea Tartaro.
21. Sam’s creators refer to her as female in publications, and I follow their practice.
22. This name is a reference to how the visage of the “great and powerful Oz,” in The Wizard of Oz (1939), was a projection, controlled by the far more mundane figure “behind the curtain,” or in Sam’s case, in the adjoining room.
23. Sam’s voice is also cyborg; instead of using a voice synthesizer to enable Sam to speak on the fly, an adult actress lent her voice to Sam. After all the utterances were recorded, they were digitally manipulated to make them sound like a child had spoken them.
24. Note that the belief that certain skills need to be developed to communicate correctly reflects a specific and dominant language ideology. Following that line of thought, Sam was also a tool for reproducing that ideology within these children.
Bakhtin, M., C. Emerson, and M. Holquist. 1986. Speech Genres and Other Late Essays. Austin: University of Texas Press.
Cassell, J. 2004. “Towards a Model of Technology and Literacy Development: Story Listening Systems.” Journal of Applied Developmental Psychology 25 (1): 75–105. http://dx.doi.org/10.1016/j.appdev.2003.11.003.
Cassell, J., M. Ananny, A. Basu, T. Bickmore, P. Chong, D. Mellis, et al. 2000. “Shared Reality: Physical Collaboration with a Virtual Peer.” Proceedings of the Conference on Human Factors in Computing Systems, January 7–11, Boulder, CO.
Cassell, J., and A. Tartaro. 2007. “Intersubjectivity in Human/Agent Interaction.” Interaction Studies: Social Behaviour and Communication in Biological and Artificial Systems 8 (3): 391–410.
Dick, P. K. 1996. Do Androids Dream of Electric Sheep? New York: Ballentine Books.
Downey, G., J. Dumit, and S. Williams. 1995. “Cyborg Anthropology.” Cultural Anthropology 10 (2): 264–9. http://dx.doi.org/10.1525/can.1995.10.2.02a00060.
Edwards, J. 2004. “Chat Room Bots: Part 1.” XBiz.com. http://www.xbiz.com/article_piece.php?cat=45&id=6706.
Forsythe, D.E. 1993. “Engineering Knowledge: The Construction of Knowledge in Artificial Intelligence.” Social Studies of Science 23 (3): 445–77. http://dx.doi.org/10.1177/0306312793023003002.
Forsythe, D.E. 1996. “New Bottles, Old Wine: Hidden Cultural Assumptions in a Computerized Explanation System for Migraine Sufferers.” Medical Anthropology Quarterly 10 (4): 551–74. http://dx.doi.org/10.1525/maq.1996.10.4.02a00100. Medline: 8979239
Freud, S. 2003. The Uncanny. New York: Penguin Classics.
Geertz, C. 1977. “Thick Description: Toward an Interpretive Theory of Culture.” In The Interpretation of Cultures, 3–30. New York: Basic Books.
Goffman, E. 1979. “Footing.” Semiotica 25 (1–2): 1–30. http://dx.doi.org/10.1515/semi.1979.25.1-2.1.
Goldsmith, T., and L. LeBlanc. 2004. “Use of Technology in Interventions for Children with Autism.” Journal of Early and Intensive Behavior Intervention 1 (2): 166–78. Citeseer.
Hafner, Katie. 2002. “In an Ancient Game, Computing’s Future.” New York Times. August 1. http://www.nytimes.com/2002/08/01/technology/in-an-ancient-game-computing-s-future.html.
Haraway, D. 1991a. “A Cyborg Manifesto: Science, Technology, and Socialist-Feminism in the Late Twentieth Century.” In Simians, Cyborgs, and Women: The Reinvention of Nature, 149–81. New York: Routledge..
Haraway, D. 1991b. “Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective.” In Simians, Cyborgs, and Women: The Reinvention of Nature, 183–201. New York: Routledge.
Haraway, D. 2003. The Companion Species Manifesto: Dogs, People, and Significant Otherness. Chicago: Prickly Paradigm Press.
Haraway, D. 2007. When Species Meet. Minneapolis: University of Minnesota Press.
Haraway, D. 2008. “When Species Meet: An Interview with Donna Haraway.” Animal Voices. http://animalvoices.ca/2008/04/22/when-species-meet-an-interview-with-donna-haraway/.
Helmreich, S. 2000. Silicon Second Nature: Culturing Artificial Life in a Digital World. Berkeley: University of California Press.
Kalyva, E., and E. Avramidis. 2005. “Improving Communication between Children with Autism and Their Peers through the ‘Circle of Friends’: A Small-Scale Intervention Study.” Journal of Applied Research in Intellectual Disabilities 18 (3): 253–61. http://dx.doi.org/10.1111/j.1468-3148.2005.00232.x.
Kulick, D. 1997. “The Gender of Brazilian Transgendered Prostitutes.” American Anthropologist 99 (3): 574–85. http://dx.doi.org/10.1525/aa.1997.99.3.574.
Leonard, A. 1998. Bots: The Origin of New Species. New York: Penguin Books Limited.
Merryman, J., A. Tartaro, M. Arie, and J. Cassell. 2008. “Designing Virtual Peers for Assessment and Intervention for Children with Autism.” Proceedings of the 7th International Conference on Interaction Design and Children. ACM. 81–84.
Mori, M. 1970. “The Uncanny Valley.” Energy 7 (4): 33–35.
Riskin, J. 2003. “The Defecating Duck, or, the Ambiguous Origins of Artificial Life.” Critical Inquiry 29 (4): 599–633. http://dx.doi.org/10.1086/377722.
Ryokai, K., C. Vaucelle, and J. Cassell. 2002. “Literacy Learning by Storytelling with a Virtual Peer.” Computer Support for Collaborative Learning 352–360.
Searle, J. R. 1982. “The Myth of the Computer.” New York Review of Books 29 (7): 3–6.
Suchman, L. A. 2007. Human-Machine Reconfigurations. Cambridge: Cambridge University Press.
Tartaro, A. 2007. “Authorable Virtual Peers for Children with Autism.” In CHI ’07 Extended Abstracts on Human Factors in Computing Systems—CHI ’07. New York: ACM Press, April 28.
Tartaro, A., and J. Cassell. 2006. “Authorable Virtual Peers for Autism Spectrum Disorders.” Proceedings of the Combined Workshop on Language-Enabled Educational Technology and Development and Evaluation for Robust Spoken Dialogue Systems at the 17th European Conference on Artificial Intelligence.
Tartaro, A., and J. Cassell. 2007. “Using Virtual Peer Technology for Children with Autism.” In Universal Usability: Designing Computer Interfaces for Diverse User Populations, ed. Jonathan Lazar, 231–62. Hoboken, NJ: John Wiley and Sons.
Tartaro, A., and J. Cassell. 2008. “Playing with Virtual Peers: Bootstrapping Contingent Discourse in Children with Autism.” Proceedings of the 8th International Conference on International Conference for the Learning Sciences 2: 382–389. International Society of the Learning Sciences.
Winograd, Terry, and Fernando Flores. 1986. Understanding Computers and Cognition: A New Foundation for Design. Norwood, NJ: Addison-Wesley.
