6 Cognitive Labor: A Story of Mental Toil

6.1 Introduction

In the contemporary imagination, computers are the epitome of thinking and intelligence. This image of machines is conjured through a combination of engineering marvels, metaphoric constructions, and discursive practices. A diverse group of actors—the technological elite, savvy entrepreneurs, the media, the state, the military, and a host of other enthusiastic practitioners and observers with a fascination toward technology—propagate this image in apparent unison, although often with divergent interests. The field of artificial intelligence has symbolized this phenomenon in a relentless fashion, with repeating cycles of fad, fanfare, and false starts interleaved with partial successes and fatal failures (Ekbia 2008). AI practitioners and enthusiasts have conceived and constructed systems with varying degrees of capability that mimic those of humans in one or another domain of behavior, but never in a way that encompasses overall human capacities. AI has kept alive one of the most ambitious dreams of modernity and beyond—namely, the creation of machines in the image of human beings. In so doing, however, it has also generated a great deal of confusion, obfuscation, and false expectations that have served, by and large, the interests of the embedding capitalist system and its dominant classes.

This chapter explores the relationship between capitalism and computing conceived as a domain that trades in “intelligent” machines. Whereas the previous chapter focused on computing as social practice, here we highlight computing as cognitive practice. Through a set of diverse examples—Amazon Mechanical Turk, the video game League of Legends, and self-service systems—we show how seemingly intelligent machines and automated systems need intense human involvement to operate effectively and deliver their promised functions. We examine how systems are set up to draw human cognitive labor into their functioning without acknowledging that labor.

6.2 Cost Saving: Value Extraction through Cognitive Labor

6.2.1 Self-Service Technologies: Cognitive Labor of Customers

A large number of technologies in a wide variety of industries belong to what comes under the rubric of self-service. Everyday life is increasingly populated by machines embellished with the label “automated”: automated teller machines (ATMs) in banks, Automated Voice Response in insurance, retail, and customer support, automated check-in at airports, clinics, and hospitals, and automated checkout in supermarkets and fast food outlets. These applications and their devices and software even have their own industry magazine, The Self-Service World (www.kioskmarketplace.com). What entitles these machines to the label is their alleged capability to perform without, or in lieu of, a human being: a teller, a customer service representative, a cashier. The reality is more complicated. A close look reveals that a good part of the labor is, in fact, done by another group of human beings—the consumer, the end user, an intermediary such as a family member, or, in some cases, a new type of casual laborer. This labor costs capital little or nothing. In many cases, it replaces people who once had conventional jobs (with benefits and security), such as customer service agents at airports.

Here we examine ATMs and automated checkout systems.

6.2.1.1 Automated Teller Machines

Banking provides a fairly straightforward case of the reallocation of tasks from paid workers to customers. ATMs, enabled by a vast infrastructure of secure networks, crediting systems, and transaction protocols, eliminate the human bank teller, saving banks money and providing a measure of convenience to customers. Online banking expands this logic by leveraging the connectivity of the internet to enable transactions without the need for physical presence. A more recent development seems to push the logic further through the introduction of the “ATM of the future,” which processes more complex transactions than simple cash withdrawals or deposits. These new machines allow bank customers to perform functions such as cashing a check or interacting with a “virtual teller,” who is a real person in a call center (NPR 2014)1.

An industry analyst explains: “There is a very large need for banks to continue to control their costs, and of course, branches, branch personnel, etc., are one of the banks’ largest costs” (ibid.). The scene of a bank equipped with self-service kiosks is not dissimilar to the notion of the “lights-out factory”—emptied of workers, this factory would need no lighting. The bank of the future does need lighting, however, because the customers are there working. Large-screen computers (rather than manufacturing robots) occupy the space (figure 6.1).

10767_006_fig_001.jpg

Figure 6.1 The bank of the future (Bank Innovation 2012).

A proponent of the technology describes the “customer interaction model”: “Tellers can walk over to the kiosk and show [customers] how to use it. By enabling and empowering the customer, we’re enriching the interaction with the teller and the branch” (Bank Innovation 2012).2 The rhetoric of “customer empowerment” obfuscates the fact that the customer has to do work that used to be done by bank employees, and that the “enrichment” goes to the bank in cost savings. So inured are we to the long lines pursuant to today’s cost-cutting management practices that the bank’s rhetoric unashamedly portrays the situation as a choice between bad and worse: “With self-service kiosks placed in teller lines, customers can interact with tellers for necessary functions, and perform self-service tasks while they wait. No one likes waiting in line” (ibid.). The coercive character of the arrangement is presented as an inevitable quandary, or even a favor being done on behalf of the customer.

6.2.1.2 Interactive Voice Response

Convenience, however, is not even a consideration in cases such as IVR. Every one of us has had the experience of interacting with voice systems that are frustrating because of their limited repertoire of responses or annoying because of the robotic delivery of individually recorded words strung together. Long menus with numerous choices, extraneous information (hours of operation, extension numbers), options that continue at each point in several languages, and incomprehensible voice prompts are some of the common features of these systems that impose cognitive burden on the caller who must navigate the system with patience and persistence. Despite the inconvenience, “Large and small businesses have adopted IVR technology because it saves money that would otherwise be spent on living, breathing (expensive) employees” (Roos 2015)3.

While noticeable progress has been made in the area of speech comprehension in the last few years, with developers boasting “retail-like simplicity” of their systems (www.voxeo.com), studies show that frustration with these systems is still prevalent, running as high as 80 percent in the popular perception (Interactions 2012)4. People sometimes find workarounds to reduce the frustration (e.g., www.pleasepress1.com, gethuman.com), but these initiatives do not eliminate the cognitive burden on end users, who must deal with such systems on an ongoing basis.

6.2.1.3 Automated Checkouts

A relatively recent addition to the world of automation, self-checkout kiosks have been on the rise in the last few years. In 2012 alone, 27,000 of these systems were installed in the United States. That IBM is involved in their manufacture says something about their growing significance.5 Kiosks are currently adopted by a wide variety of service industries, from fast food to postal service to pharmacies. Market projections predict a $7.5 billion industry for automated pharmacies by 2018 (Huffman 2014).

The key concern of critics is focused on loss of jobs to these technologies, with the implicit assumption that machines are replacing humans. While job loss is real, what is missing from these analyses is that machines are not fully replacing humans, but rather reconfiguring the labor into the heteromated labor of end users who do the cognitive work. Since heteromated labor occurs on the margins of machines and organizations, often it is not even recognized as labor. Observant commentators note that many service occupations of the past no longer exist—“people who pump your gasoline, people who punch the button or your floor in an elevator … clerk[s] selling candy and knick knacks at the hotel snack bar” (Huffman 2014). The commentators also notice that, “Today, consumers perform all those tasks for themselves and a lot more” (ibid.). As a matter of fact, consumers perform the tasks for corporations, not for themselves. The tasks are forms of heteromated labor costing individuals time and sometimes money (see Ritzer and Jurgenson 2010).

These examples illustrate the growing trend toward what is oxymoronically called “self-service” across various sectors and industries. These arrangements offer “service”—a word that means one person doing something for another—that is no service at all. People increasingly find themselves struggling with kiosks in stressful situations at airports, banks, hotels, clinics, and, very soon, in pharmacies. Business discourse does not recognize these actions as labor because corporations stand to benefit from hiding this relation. Their own explanations to journalists, however, often recognize the labor, e.g., in the comment about “expensive” employees, as well as the mention of Facebook’s “unpaid workforce” (chapter 5).

6.2.2 Punish or Pardon: Cognitive Labor of End Users

We turn now to a different kind of cognitive labor—that performed by gamers who play the video game League of Legends. The coercive nature of the systems we have discussed so far in this chapter is absent—gamers willingly contribute labor to enhance their play experience.

Imagine you are a video game company and you receive tens of thousands of aggrieved emails every day from your player base of 67 million players worldwide. In 2011, this was the situation for Santa Monica–based Riot Games, one of the most successful video game companies in the world. In a perfect storm of technical affordances and demography, a large group of undisciplined players was spoiling the game for other players, and those other players were writing to Riot Games, the creator of League of Legends. The game is played in small teams of computer-matched players who generally do not know one another. They play short matches of about 30–60 minutes, and are unlikely to see one another again. Most “LoL” players are young men and boys, the demographic most likely to act outside conventional social norms, or even to be unaware of conventional social norms. LoL’s mix of social and ludic factors created what Riot Games calls “toxic behavior,” including crude or hostile language, deliberately sabotaging matches, and displaying a bad attitude in an environment that is supposed to be fun. Players complained about toxic behavior, declaring that it was ruining their game (Kou and Nardi 2013).

Because multiplayer games demand the presence of other players as functioning parts of the system (Ekbia and Nardi 2012), if players leave the game, there is no game. People are readily induced to participate online, but, once there, they are not always docile and compliant. Game companies must find ways to govern the unruly, or their systems will fail. Riot Games thus instituted a form of heteromated labor in a system called “The Tribunal.” This system allowed players to report disruptive players after a match through an online form. If a player was reported frequently, and by multiple players, the automated system prepared a case that considered the number and nature of the complaints. Players could log into The Tribunal and judge cases by examining direct evidence, such as chatlogs. Judges chose “Punish” or “Pardon” based on their assessment of the case. If the majority voted to Punish, the Tribunal suspended the player’s account for varying periods depending on the pattern of infractions, possibly ending in a ban from the game in the case of persistently toxic behavior (ibid.).

The Tribunal apparently mitigated at least some of the behavioral problems (Senior 2011), and was declared a success. It is not in use at the time of this writing because after three years, Riot felt it had established positive community norms and improved player behavior significantly.6 A designer we interviewed said, “One of the core philosophies of The Tribunal is to engage and collaborate with the community to try to solve player behavior together” (Kou and Nardi 2014). Drawing free labor from players reduced the costs of player governance and served to bind players even more closely to the game through the increased investment of their active participation. While the cognitive labor required of the Tribunal was willingly given, its economic value was hidden by high-touch words and phrases such as “collaborate,” “community,” and “solve problems together.”

Such collaboration is arguably a positive development. We note, however, that in cases where gaming companies disagree with the player community, they have not hesitated to take legal action against player interests and/or to use their control of the software itself to enforce their choices, despite players’ contributions of heteromated labor (see Nardi 2010, Postigo 2010, Kow and Nardi 2012, and chapter 7 in this book). Riot Games motivated players to do the work of establishing community norms and putting undisciplined players on notice, while reaping economic benefits, and yet true shared governance has not been tested. Judging by other cases, such governance is far from a sure thing.

6.2.3 Amazon Mechanical Turk: Cognitive Labor of Casual Employees

Not all heteromated cognitive labor is performed by consumers or players motivated by a spirit of participation. A different kind of motivation drives people to perform cognitive tasks in online labor markets such as AMT, TextBroker, and CrowdFlower. “Turkers” on AMT, for instance, enter into a contractual relationship with “requesters” to perform “Human Intelligence Tasks (HITs)” of varying difficulty. The requesters, who are often businesses but also academic researchers and other professionals, post tasks such as choosing the best among several photographs of a storefront, writing product descriptions, or responding to a survey. Turkers are paid by requesters according to the complexity of the task, as well as their own history, experience, and status. Payments vary accordingly, averaging between $2 and $3 per hour for basic tasks to $6 to $12 per hour for more complex tasks. Perhaps more important, hourly rates depend on how fast Turkers work and the acuity with which they select tasks. An elite group has become adept at not only working at speed, but at knowing how to choose tasks that can be completed quickly and that are put out by requesters with the best returns. A majority of this group lives in the United States, debunking claims about the “Third-World” weight of online labor (Silberman 2015).

Statistical information on the overall contribution of this type of labor to the economy is unavailable, but its value for requesters is hardly disputable, whether we consider businesses, researchers, or others. Studies have shown the overall quality of the work produced by Turkers for academic research is high, despite concerns over self-selection bias and other methodological issues (Sprouse 2011). (This point is disputed; see also Paolacci and Chandler 2014.) The bigger beneficiaries of the online labor market, however, are businesses, who manage microworkers at very low cost and without the contractual obligations of formal employment, including health insurance, retirement benefits, workers’ compensation, sick leave, and vacation pay. In fact, the impersonal and algorithmic character of AMT and other mediators in the online labor market has arguably turned these laborers into “second-class citizens” (Silberman 2015).

6.3 The Invisible Value of Human Cognitive Capacities

While League of Legends players are incited to participate in heteromated labor to make their play space more appealing, many Turkers (and other online laborers) are driven by economic incentives. Some are meeting basic economic needs, while others seek small bits of money for discretionary purchases (Jiang, Wagner, and Nardi 2015). Silberman is very clear that some Turkers perform microwork “not by choice but because they are unable to secure other employment” (Silberman 2015, p. 14; see also Martin et al. 2014; Jiang, Wagner, and Nardi 2015). That these workers must work for multiple requesters distorts the true nature of the relationship as employment. The mediated character of the relationship enacted through the application programming interface (API) reinforces the distortion, allowing requesters “to manage human workers through software, as if workers were themselves software rather than people” (Silberman 2015, p. 20). All of this leads to the notion that what is really happening is computation, not labor—a notion that finds an unfortunate resonance in the new area of research called “human computation.” The person and the person’s labor disappear; only the output—the computation—is present, revealing once again the marginal character of persons performing heteromated labor. The machine, as the defining object, stands tall, and the human “steps to the side,” as Marx said.

In between gamers and Turkers, we have the large group of consumers who increasingly find themselves caught in the grip of heteromated systems, without much recourse to alternatives. Pundits argue that consumers favor self-checkout systems because of speed, convenience, ease of use, and privacy. Surveys by business outlets claim that 85 percent of American and Canadian consumers are more likely to do business with a store that offers self-service7. A 2011 study of self-service in the hospitality industry reports that “consumers are not willing to use self-service technology in both lodging and food service environments, but 55 percent of those surveyed would be more likely to visit a hotel that offers the options to check-in/out” (Hospitality Industry 2011). To the extent that these reports are accurate, the fact of the matter is that consumers are intricately nudged toward certain types of preferences, or as one business analyst put it, “Self-service actually reinvents the customer experience” (Hospitality Industry 2009). The survey, for instance, shows that consumer preferences are largely driven by shorter waiting lines, faster service, accuracy, and privacy. While more than 70 percent of people surveyed mentioned these elements as motivators, only 40 percent favored “no interaction with clerk” in hotels and restaurants as reasons for considering self-service technology. In answering these questions, consumers may be thinking about the “long lines that no one likes” when choosing automated options, rather than formulating the question as one of a choice between a capable human vs. a machine. Business literature, however, tends to twist this, portraying choices as “interest in the technology and the belief in its potential for improving service” (ibid., p. 4).8 One only need read TripAdvisor and Yelp reviews to see how much reviewers appreciate, and remark on, good customer service, informing other potential customers with this critical information for their choices.

6.4 A Paradox of Cognitive Capacity

One of the promises of technology is that it will free us for challenging, creative activity while the machines take over the menial labor. This vision was put forward in strong form in the era of augmentation, as we discussed in chapter 1. The examples in this chapter show that now we are veering off this path to a significant degree, and it is often the humans who end up performing the menial tasks. The increasing use of self-service devices and algorithmic management in systems such as Mechanical Turk marginalize humans, whose intelligence and creativity are sidelined. These systems ask mostly that we be patient and carry out the lockstep actions they present for us to perform. We conduct this labor because we must (to complete a task or for meager compensation), or because we agree to donate time in the absence of human mediators (e.g., LoL players). Sometimes we do so because it is more convenient when there is no human assistance.

What kind of a trade-off is this? Does our participation in such labor change who we are as social beings? Some people prefer to check their own groceries, to trust in an algorithmic system to discipline others, to order their food at a kiosk in a fast food restaurant. Humans can be unfriendly, unpredictable, unreliable—even toxic! The more alienated we become from one another (in part due to separation, precarity, futility, and monotony), the less socially attractive we become. Perhaps a self-reinforcing cycle is emerging, driving us toward increasing ease with machines, turning our behavior more machine-like all the while. Has anyone asked us—the average shopper, customer, user, player—whether or not we favor this change?

Notes