The world of contemporary machine learning (ML)—specifically in the domain of the multilayered “deep” neural networks, generative adversarial networks, differentiable programming, and related novelties in what is known as artificial intelligence (AI)—poses difficulties for those in the social sciences, like us, who wish to take its rich and varied phenomena as objects of study. We want, ideally, to be able to offer timely contributions to present-day, pressing debates regarding these technologies and their impacts; but at the same time, we would like to make claims that persist beyond the specific features of today’s (or yesterday’s) innovations. The rapid pace of technical and institutional change in ML today—in which researchers, practitioners, think tanks, and policymakers are breathlessly playing a game of catch-up with each other—only exacerbates this tension. While the topic of AI has attracted interest from social scientists and humanists in the past, the recent conjunction of ML hype, massive allocations of technological and financial resources, internal scientific controversies about the validity of connectionist approaches, and discourses about hopes and fears all mark the rise to prominence of twenty-first-century machine learning and deep learning (DL) as a paradigmatically novel sociotechnical phenomenon. In a nutshell, what we are witnessing is nothing less than an epistemic shock or what Pasquinelli (2015) has referred to as an epistemic “trauma.” For scholars of cultural life—such as sociologists, media scholars, and those affiliated with science and technology studies—this situation forces us to ask by what methods we can possibly stay up to date with these radical transformations‚ while also being able to provide commentary of some significance. How, especially, would it be possible to make sense of the present challenges posed by ML, but in a way that allows for a more complex (and indeed “deeper”) understanding currently unavailable to ML’s practitioners? In this introduction, we want to wager that it may be more productive to embrace these tensions than to attempt to fully resolve them. For instance, it is certainly possible to be technically precise while proposing perspectives quite distant from the computing sciences—the different chapters assembled here are a testimony to this—and it is certainly possible to engage with these technologies and their many subtleties while remaining focused (or, indeed, “trained”) on the more historical and cultural if not mythical aspects of their deployment. The list of dualities does not stop there, of course. ML and modern AI models are simultaneously agents for epistemology and, increasingly, ontology; that is to say, they are a way of knowing as well as of being in the world. They are part of a discourse as much as they are a mode of action, and they are a description of the world and its social composition as much as a prescription of what it ought to be. In turn, the study of machine learning must be aware of this epistemological/ontological tension and be willing to carefully navigate it.
It should perhaps not be surprising that this is not the first time that critical reflections on artificial intelligence emerging from the social sciences have had to fight for their legitimacy. In the mid-1980s, Bloomfield’s “The Culture of Artificial Intelligence” (1987)—a work today almost entirely forgotten—forcefully argued against the “exclusion of sociological questions from any serious examination of AI” and the “foreclosure of sociology to questions of social impact” (pp. 63–67). Around the same time, a better-remembered piece by Woolgar (1985) raised the question: “why not a sociology of machines?”—primarily to indicate that such an endeavor must go beyond simply examining the impacts of technology and attend to its genesis and social construction. What these kinds of positions had in common was a commitment to develop a more holistic approach, in which no aspect of these so-called intelligent technologies would be left out of consideration; so we see in Schwartz (1989) the idea that a proper sociology of AI could ask “under what conditions and in what settings is a model deemed adequate?,” and in Forsythe’s (1993) work the argument that “engineers’ assumptions have some unintended negative consequences for their practice, for the systems they build, and (potentially at least) the broader society” (p. 448). Fast forward some 30-plus years, and the need to make social-scientific discourse on what one might call “21st-century” AI both socially pertinent and accurate has returned with a vengeance. If we consider the sociotechnical genesis of these techniques as “upstream” and their eventual social impact as “downstream,” then we can see critics like Powles and Nissenbaum (2018), who write of the “seductive diversion of ‘solving’ bias in artificial intelligence,” as warning against an overemphasis on upstream engineering dilemmas without considering how “scientific fairness” comes to be deployed in practice; and we can see Roberge, Senneville, and Morin (2020) discussion of regulatory bodies such as Quebec’s Observatory on the Social Impact of AI (OBVIA) as warning of a corresponding overemphasis on “downstream” social impact, which does not see that said social impact is explicitly entangled with the development of the commercial AI research power center known as the Montréal hub.
As a corrective, we want to propose the need for what could be called—with a wink and a nod to deep learning methodology—an end-to-end sociology of contemporary ML/AI, which understands this explicit entanglement of “upstream” and “downstream” and instead trains itself on the entire sociotechnical and political process of modern machine learning from genesis to impact and back again. In this, we find ourselves in line with scholars like Sloane and Moss (2019) who have recently argued, for an audience of AI practitioners, that it is necessary to overcome “AI’s social science deficit” by “leveraging qualitative ways of knowing the sociotechnical world.” Such a stance justifies the value of historical, theoretical, and political research at both an epistemological level of how AI/ML comes to produce and justify knowledge, and at an ontological level of understanding the essence of these technologies and how we can come to coexist with them in everyday practice. But to do so requires an epistemic step that ML practitioners have not fully accepted themselves, namely, to insist on a definition of ML/AI as a “co-production requiring the interaction of social and technical processes” (Holton & Boyd, 2019, p. 2). Radford and Joseph (2020), for their part, have proposed a comparable framework that they call “theory in, theory out,” in which “social theory helps us solve problems arising at every step in the machine learning for social data pipeline” (p. 2; emphasis added). These perspectives represent threads that weave in and out of the chapters in this book as they address machine learning and artificial intelligence from differing historical, theoretical, and political perspectives from their epistemic genesis to sociotechnical implementations to social impact. These chapters can be seen to represent a different attempt to bring these proposals into reality with empirically motivated thinking and research.
To engage with machine learning requires, to some extent, understanding better what these techniques and technologies are about in the first place for its practitioners. What are the baseline assumptions and technical-historical roots of ML? What ways of knowing do these assumptions promote? While it is not uncommon to read that ML represents a “black boxed” technology by both insiders and outsiders, it is nonetheless important to stress how counterproductive such a claim can be, in part because of its bland ubiquity. Yes, ML can be difficult to grasp due to its apparent (if not always actual) complexity of large numbers of model parameters, the rapid pace of its development in computer science, and the array of sub-techniques it encompasses (whether they be the genres of learning, such as supervised, unsupervised, self-supervised, or the specific algorithmic models such as decision trees, support vector machines, or neural networks). As of late, different scholars have tried to warn that the “widespread notion of algorithms as black boxes may prevent research more than encouraging it” (Bucher, 2016, p. 84; see also Burrell, 2016; Geiger, 2017; Sudmann, 2018). Hence, the contrary dictum—“do not fear the black box” (Bucher, 2016, p. 85)—encourages us to deconstruct ML’s fundamental claims about itself, while simultaneously paying special attention to its internal logics and characteristics and, to some degree, aligning social scientists with AI researchers who are also genuinely curious about the apparent successes and potentially serious limitations of today’s ML models (even if their tactics are limited to the quantitative). While the difficulty of knowing what’s going on inside a neural network should not be seen as a conspiracy, it is the case that certain ideological underpinnings can be exposed by determining what aspects of the “black box” are in fact known and unknown to practitioners.
One fundamental characteristic of contemporary machine learning, which one can best observe in the “connectionist machine” (Cardon, Cointet, & Mazières, 2018) of deep learning, is precisely this pragmatic and model-centric culture. It is with deep learning that we can most easily recognize as social scientists that we have moved from an analytical world of the algorithm to the world of the model, a relatively inert, sequential, and/or recurrent structure of matrices and vectors (which nevertheless is, of course, trained in a processual manner). For DL practitioners, the only truly important “algorithm” dates from the mid-nineteenth century: namely, Cauchy’s (1847) method of gradient descent. Much of the rest of deep learning’s logic often seems more art than science: a grab-bag of techniques that researchers must confront and overcome with practice and for which there can be no formal guidance. These are the notable “Tricks of the Trade” (Orr & Müller, 1998) that the previous marginalized wave of neural network research came to circulate among themselves; today they refer, for example, to the “hyperparameters” that exist outside both the model and the algorithm and yet crucially determine its success (in often unpredictable ways). This relates to a second fundamental characteristic: the flexibility and dynamic, cybernetic quality of contemporary machine learning. Training a model on millions of training examples is a genetic process, during which the model develops over time. But it is not just the model that develops, but the social world of which the model is but a part; every deep learning researcher is, more so than in other sciences, attuned to each other and each other’s models, because an innovation in one field (such as machine translation) might be profitably transduced to new domains (such as computer vision).
As we can see, it is not just the training processes of contemporary machine learning that randomly explores to find a good local minimum (e.g., using backpropagation and stochastic gradient descent): the entire sociotechnical and cultural endeavor of ML mirrors that mechanism. “Machine learning is not a one-shot process of building a dataset and running a learner,” Domingos notes, “but rather an iterative process of running the learner, analyzing the result, modifying the data and/or the learner and repeating” (Domingos, 2012, as cited by Mackenzie, 2015). That the same can be said of both the field’s model architectures and the field in general reflects the self-referentiality that is a third fundamental characteristic of contemporary machine learning, in which machine learning practitioners, implicitly or explicitly, see their own behavior in terms of the epistemology of their techniques. This inward quality was also found among the researchers of an earlier generation of AI, who saw the height of intelligence as the chess-playing manipulator of symbolic mathematical equations (Cohen-Cole, 2005); today we should be unsurprised that a reinforcement-learning agent with superhuman skill at various Atari video games (Mnih et al., 2013) was considered by some practitioners as a harbinger of machine superintelligence. This represents the logic of a closed community in which the only known social theory is game theory (Castelle, 2020).
Machines using supervised learning to recognize images, speech, and text are not only connectionist, but “inductive machines” by nature (Cardon et al., 2018). ML (and especially DL) methodologies hold firm in this grounded approach where reality emerges from data and knowledge emerges from observation, and the assumptions are often (if not always) straightforward: i.e., that there must be self-evident, objective ties between what is “out there” and what is to be modeled and monitored. These inductivist views, in other words, offer a kind of realism and pragmatism that is only reinforced by the migration of architectures for image recognition—such as the famous ImageNet-based models, which try to identify 1000 different types of objects in bitmap photos (Krizhevsky, Sutskever, & Hinton 2012)—to the more agentive world of real-time surveillance systems (Stark, 2019) or autonomous vehicles (Stilgoe, 2018). These embodied, real-world systems retain the ideology of simpler models, where to “recognize” is to decipher differences in pixels, to “see” is to detect edges, textures, and shapes and to ultimately pair an object with a preexisting label: this is a leopard, this is a container ship, and so on. Instead, these core principles of image recognition have remained unchallenged—namely that the task at hand is one of projecting the realm of the visual onto a flat taxonomy of concepts. And this is where signs of vulnerability inevitably appear: isn’t it all too easy to be adequate in this domain? Crawford and Paglen (2019) have notably raised this issue of the fundamental ambiguity of the visual world by noting that “the automated interpretation of images is an inherently social and political project, rather than a purely technical one.”
Such an argument nicely sums up what we meant earlier for the necessity of the social sciences to engage with machine learning on its own epistemological grounds. The idea is not to deny the possibility of reflexivity within ML cultures, but to instead relentlessly question the robustness of said reflexivity, especially outside of narrow technical contexts. The debate is thus on, and at present finds itself to be an interesting echo of the argument that the rise of big data should be associated with an “end of theory” (Anderson, 2008). Then, the term “theory” referred to traditional statistical models and scientific hypotheses, which would be hypothetically rendered irrelevant in the face of massive data sets and millions of fine-grained correlations (boyd & Crawford, 2012). But instead of big data’s crisis of empiricism, in the case of machine learning we have—as we have suggested above—a crisis of epistemology and ontology, as ML models become more present and take on ever more agency in our everyday lives. At present, machine learning culture is held together by what Elish and boyd (2018) call “epistemological duct tape,” and the different chapters in this book are, in part, a testimony to this marked instability.
How to Categorize Meanings
It has become increasingly difficult to ignore the level of hype associated with ML and AI in the past decade, whether it be claims about how the latest developments represent a “tsunami” (Manning, 2015), a “revolution” (Sejnowski, 2018), or—to be more critical—something of a myth (Natale & Ballatore, 2020; Roberge et al., 2020) or a magical tale (Elish & boyd, 2018). This is what we intend to capture in saying that ML has developed a cultural life of its own. The question, of course, is to understand how this is possible; and on closer inspection, it seems apparent that what has allowed ML to become such a meaningful endeavor is its claim to meaning itself. Once one looks, one begins to see it everywhere: from Mark Zuckerberg noting that “most of [Facebook’s] AI research is focused on understanding the meaning of what people share” (Zuckerberg, 2015; emphasis added) to Yoshua Bengio for whom the conversation is “about computers gradually making sense of the world around us by observation” (Bengio, 2016). Similar quotes can be found regarding specific tasks like object recognition, in which the goal is “to translate the meaning of an image” (LeCun, Bengio, & Hinton, 2015) and/or to develop a “fuller understanding of the 3D as well as the semantic visual world” (Li quoted in Knight, 2016).
This latent desire to “solve” the question of meaning within the formerly deeply symbol-centric world of artificial intelligence here manifests itself as claims of an unfolding conquest, but not everyone is convinced; Mitchell (2018), for example, shows how contemporary AI time and again crashes into the “barrier of meaning.” Mitchell argues that this is because AI’s associationist training methodologies (a) do not have “commonsense knowledge” of the world and how other actors in the world behave, and (b) are unable to generalize to develop more abstract concepts and to “flexibly adapt … concepts to new situations.” We would argue that a better distinction might be between decontextualized meaning, i.e., the sense-relations that seem to be carried by signs independent of context, and pragmatic reference, which is largely dependent on context (Wertsch, 1983). It is with the former that machine learning excels—for example in the “sorting things out” of classification models (Bowker & Star, 1999), and in the sense-relations seemingly captured by word embeddings (Mikolov, Sutskever, Chen, Corrado, & Dean, 2013)—but with the latter, models can only struggle to accommodate pragmatic reference by decontextualizing as much input as possible (one will notice that so-called natural language processing has far more to do with decontextualized sentences of written text than with real-world utterances between two or more humans). ML practitioners, in general, tend to have a limited sense of what “context” is, in contrast to the term’s use by anthropologists to indicate how the sociocultural situations in which communicative utterances occur affect and transform their meaning. For ML, this insatiable effort to calculate meaning by relentlessly making so-called context out of co-text (Lyons, 1995, p. 271), however, tends to opens the door to existing processes of commensuration (Espeland & Stevens, 1998), and does not tend to any increased reflexivity on behalf of its researchers and developers about the nature of communication, meaning, and even learning. Social scientists and philosophers—especially those concerned with hermeneutics, as we will describe below—will recognize the epistemological and ontological issues in the predominance of such a myopic worldview.
What is left after these processes of decontextualization and entextualization (Bauman & Briggs, 1990) are the materials for the numerous classification tasks at which modern machine learning excels. In the social-scientific literature it is Mackenzie (2017) who has discussed these models at the greatest length; for him, ML is a “diagramming machine” spanning processes of “vectorization, optimization, probabilization, pattern recognition, regularization, and propagation” (p. 18); and by “diagram” he indicates, via Peirce (1931), a semiotic form that produces meaning iconically and indexically, or through some kind of similarity and physical contiguity; this, again, differs from the sign-systems dominant in the history of computing, namely those that are largely symbolic. So, for example, Mackenzie can see deep learning’s fundamental practice of vectorization—which projects all data (whether input, intermediate data, or output) into some high-dimensional vector space—as a historical development of the process of power/knowledge begun with the grids and tables described by Foucault (1970). While this classical episteme was associated primarily with unidimensional and symbolic practices of ordering, ranking, sorting, and joining, such as those of the relational database (Castelle, 2013), the vectorized world is one in which the similarity of data points is literally a geometrical transformation (e.g., the “cosine distance”) or sequence of such transformations.
While one might associate the thousand categories of ImageNet-based object recognition models with the regime of hierarchical order, the operationalization of deep learning’s object recognition—and its predecessor, pattern recognition—ignores any taxonomy of its object categories (i.e., it ignores the “Net” in the original “ImageNet” database). Instead, ML/DL’s conquest of iconicity—its ability to calculate the likeness between a picture of a tiger and an arbitrary value or category denoted as “tiger”—is performed through a layered, directional (and thus indexical) flow of linear and nonlinear transformations. In a well-trained model, this sequence of transformations produces the appropriate category as its output without reference to any “common sense” or semantic knowledge base. But by producing categorical outputs, ML/DL necessary morphs into something prescriptive. To return to Bowker and Star (1999), “each standard and each category valorizes some point of view and silences another” (p. 5). And because the data used today in ML, and especially DL, is all too human—from musical taste to surveillance camera footage, from commuting routes to interpersonal conversations—the significance of this becomes enormous.
In his contribution to this volume, Aaron Mendon-Plasek offers an historical account of how precisely machine learning came to categorize meanings. As today’s practitioners tend to understand it, machine learning was invented in the late 1950s with Samuel’s (1959) checkers-playing program and then goes mysteriously silent for much of the 1960s and 1970s during a period of dominance by “good old fashioned” symbolic AI (in part incurred by Minsky and Papert’s attack on neural networks), only for inductive methods to emerge again in the 1980s. Mendon-Plasek detonates this standard just-so narrative by showing how the field of pattern recognition in fact emerged in 1955 (with work on character recognition by Oliver Selfridge and Gerald Paul Dinneen), remaining relevant throughout the 1960s and 1970s, and from the outset had a modus operandi identical to that of today’s machine learning: that of “mechanizing contextually contingent significance.” His central argument is that with this framing, the stakes became nothing less than the elaboration of a different episteme, in the Foucauldian sense. This epistemic worldview valorized the percentage of correct classifications as the measure of meaning, resulting in a very efficient if mundane “legitimation through performance” (Lash, 2007, p. 67), to which we will come back later in this introduction. However, the notion of a “legitimate” methodology might be something of an oxymoron, as the credit and confidence that produce legitimacy must be mobilized by broader social and cultural forces.
To develop an historical and (socio)-theoretical account of ML is also of interest to Tyler Reigeluth and Michael Castelle in their contribution. The question they raise is of the highest stakes: if machine learning is a kind of “learning,” then how should we think of the system of “education” it implicitly proposes? This question forces one to think about the condition of possibilities for, and potential distinction between, human and mechanical/computational/technological ways of acquiring and nurturing knowledge. The authors revisit the work of psychologists Vygotsky, Luria, and Leont’ev, originating in the 1920s and 1930s in the Soviet Union, and their emphasis on the social and cultural dimensions of pedagogy. To learn implies a genetic process, i.e., an engagement in a developmental and transformational activity; but it also implies a dialectical process occurring between individuals and society: the self, others, and groups of others, i.e., teachers, communities of peers, etc. This is made possible through what Vygotsky calls “mediation”—which, usefully for the comparison between human learning and machine learning, can take place as either linguistic/semiotic communications or in the form of technical interventions—which in turn is how learners make sense of culture and the production of meaning. Reigeluth and Castelle go on to (re)frame the issue by arguing, with Vygotsky, that “a concept’s meaning actually develops through learning as a social relation … [t]he meaning of a signifier is not presupposed nor is it intrinsically attached to a word. Rather, it is the result of a dialectical process through which meaning develops socially.” Here we see a stark distance between what counts as learning in Soviet psychology and what counts as learning for proponents and users of machine learning; for the former, learning is always fundamentally social, and for the latter, there is rarely even a concept of a “teacher” (even the “supervision” of supervised learning is merely an inert list of labels). That is to say, learning—as the acquisition and constant transformation of knowledge—goes beyond the finite capabilities of individuals, and it is through sociocultural mediation that it becomes possible and meaningful; and so there must be something fundamentally misguided with most forms of machine learning. This insight mandates the opening of a dialogue between ML, the social sciences, phenomenology, and hermeneutics, rather than foreclosing it.
To be fair, it is not the case that discussions of interpretation, explanation, and understanding are left out of current discussions around ML. They are, in fact, quite prevalent, in the form of the burgeoning literature on “interpretable AI,” “explainable AI” (or “xAI”), and “human-understandable AI” (Biran & Cotton, 2017; Gilpin et al., 2018), categories that overlap in various ways but all of which signal a sudden discovery of hermeneutics among computer scientists—without, of course, discovering the word “hermeneutics” itself. These concepts of interpretation, explanation, and understanding indeed have a very long history in philosophical and biblical exegesis, and their respective definitions were at the core of the Methodenstreit in the late eighteenth and early nineteenth century that came to define what we now know as the social sciences. But as of yet, there are few in computer science who have dared to make this connection with centuries of existing thought (building up to the philosophical hermeneutics of the late twentieth century), although some interventions have been made on behalf of social and cognitive psychology (Miller, 2019). Instead, the growing autoreferential repurposing and rebranding of these terms into historically detached subfields of, e.g., “xAI” do less to mitigate the black-box qualities and inductive ambiguities of machine learning and instead add yet another layer to the problem by providing approximate, “local surrogate,” or linear models instead of addressing the intrinsically interactive, or dialogical, nature of interpretation and understanding (Gadamer, 1977; Mittelstadt, Russell, & Wachter, 2019). Instead, “explainable AI” is largely (if often unconsciously) a positivist project designed to, on the one hand, encourage acceptance of increasingly agentive machine learning models and, on the other, to convince computer scientists that interpretation is an agreed-upon concept which practitioners can “wield … in a quasi-mathematical way” (Lipton, 2016). Other commentators argue that explainable AI represents a catch-22 in that if it were possible to explain a decision, we would not need to be using ML in the first place (Robbins, 2019). Is it possible to continue along this path in the absence of reflexivity? Can interpretability escape questions about self-understanding? Can interrogations about how go without interrogations about why? These are certainly crucial matters related to meaning-making and situatedness, values and change, and therefore refer to more than simply a question of method (Gadamer, 1975).
Fenwick McKelvey’s contribution to this volume focuses on the missteps and missed opportunities that have punctuated the relation between machine learning and the social sciences or, in the case of his study, the specific relationship of predictive computational analysis to the rise of the “New Political Science” during the Cold War era. What he offers is a “genealogy of artificial intelligence as a political epistemology” whose goal is to explain “how we came to believe that humans—especially their political behavior—could be modelled by computers in the first place.” Certain conditions were necessary to achieve just that, including an emphasis on how rather than why, a straightforward view of social determinism, and the associated belief that social categories were more important than individual agency or even specific geographical location. Through an integration of the political-scientific ideology known as behavioralism with a nascent mathematical modeling, the Simulmatics Corporation, for instance, was referred to as the “A-bomb of social-science” for its experimental attempts to model the US electorate in terms of “issue clusters” based on surveys and demographic information. These were informed by the assumption of what was called a human “subjective consistency” that would permit not only observing but estimating, and not only simulating and modelling but forecasting and predicting how people would react to different political propositions. Politics was thus to become the object of a kind of cybernetics. One could say that the New Political Science of that time developed as “an engine, not a camera” (MacKenzie, 2006), in which “the opinion poll is an instrument of political action” (Bourdieu, 1979) with substantial implications that, more than ever, we are witnessing today. Specifically, these developments also paved the way for a computational brand of social science to be more and more involved in decision making and, with different degrees of legitimacy, in political steering—e.g., the now-infamous Cambridge Analytica as the predictive core of a propaganda machine. In short, McKelvey describes the origins of a “thin citizenship”—a situation in which “data functions as a proxy for the voter”—which has become utterly dominant today.
ML’s Quest for Agency
From what we have seen so far it is clear that machine learning both represents and intervenes. Yet, it remains to be seen exactly why these two dimensions of meaning making and action are so fundamentally inseparable. ML’s “algorithmic modeling” (Breiman, 2001) differs somewhat from traditional statistical modeling, in that the goal of the former is primarily attaining high “prediction” accuracy on a held-out dataset and not necessarily a parsimonious parameterized model as in the latter; i.e., sometimes a neural network with large numbers of uninterpretable parameter weights will do. As such, machine learning culture is more directly involved with the possibility of taking action. (For example, in the example of email spam classification, it is not really enough to assess an email as being spam or to know why an email has been assessed as spam, but it is very useful to actively label it as such and automatically move it to the spam folder.) It is likely the case that this agentive use of ML is in part responsible for the increased attention given to machine learning by social scientists in recent years after decades of quiet existence within computer science. While traditional statistical models often remain inside the ivory tower and only induce action through the work of strategic policymakers, machine learning models are readymade as (semi-) autonomous; the act of classification, whose accuracy is optimized during training, can become an act of decision-making during deployment. It is one and the same operation. And just as the model itself is internally a process of small optimizations, so is the operationalization of the problem it is trying to solve. The inherent pragmatism of ML compels practitioners to tweak their models (and their surrounding sociotechnical environment) to find, as practical guidebooks recommend, “the level of detection that is useful to you” (Dunning & Friedman quoted in Amoore, 2019, p. 7). The resultant models are thus both flexible and capable of operationalization; their proposed solutions take the form of actions thrown into a greater course-of-life or world action in an attempt to alter that course and produce a desired outcome.
On the level of signification, data mining techniques attribute meaning to every variable in relation to a purpose; on the level of performativity, the move to increasingly integrated digital infrastructures means that every classificatory decision can be pushed back into the world instantly, showing a specific ad, hiding a specific post, refusing a loan to a specific applicant, setting the price of a product to a specific level, and so forth. No data point remains innocent. (pp. 110–111)
Another, related way to look at ML’s complicated and very much entrenched agency is with its constant attempt at anticipating the near future. ML models exist because they are “predictive” engines, which does not mean that they will precisely see the future but that prediction is what they “want” (Mackenzie, 2015). Reigeluth (2018) points to how this is an essentially cybernetic quality; “action,” he notes, “is always ahead of itself, it is already prediction in action, which makes it possible to say that prediction is internal to action” (p. 57; emphasis added). This captures the sense of anticipation that is central to both the training process and the execution of ML models. Mackenzie (2015) offers a compact interpretation based on the current modus operandi of real-world digital platforms: ML is set in motion by a “desire to predict desire” (p. 431). This desire is captured by various metrics that either measure performance of a model directly (e.g., the root-mean-squared error metric) or by indirect captivation metrics that measure the amount of user time spent on the platform (Seaver, 2019). For a company like Netflix, success is measured by its ability to choose for its own customers and vice versa, i.e., “the production of sophisticated recommendations produces greater customer satisfaction which produces more customer data which in turn produces more sophisticated recommendations, and so on” (Hallinan & Striphas, 2016, p. 122). Anticipation, expectation, the manufacturing of choice, and informed trading on outcomes all become part of a spiral in which the future’s indeterminacy is generated, managed, and potentially conquered. Throughout this process, prediction is inevitably not only just a form of prescription and normalization but of popular legitimacy; consider the $1 million Netflix Prize, which drew attention to big data–based recommendation systems and to Netflix’s expertise in said techniques. The more ML prediction is applied, the more it becomes accepted; the more it spreads, the more it gets entrenched in the fabric of everyday life. Following scholars like Benbouzid and Cardon (2018), it would then be possible to assert that prediction has now become a key form of intervention into society in such a fashion that it becomes within ML’s agency to be able to fundamentally change that broader reality.
In her contribution to this volume, Ceyda Yolgörmez proposes to take the sociology of AI seriously, both by incorporating STS scholars such as Woolgar and the classical social theorist G. H. Mead alongside Alan Turing, and by moving beyond existing technological-deterministic approaches. Her goal is to reassess the condition of possibility that would make it feasible to think about “AI as an integral part of a social interaction.” One such condition is dynamism; something that, as we describe above, is now a fundamental internal aspect of predictive models but also something that is external in those models’ capacity to operate in the real world. For Yolgörmez, dynamism is thus synonymous with openness and indeterminacy, which in turn is what makes possible the dialogue between humans and machines. It is as if both human and machine were similarly imperfect in their inability to fully control the space of potential and actual interactions between them. The chapter’s main point thus relates to the creation of a “distributed understanding of agency” inspired by Mead, one that would nonetheless allow for a non-negligible dose of autonomy and reflexivity for the actors of such a “new” sociability. For example, Mead (1934) explains the interacting individual as being composed of a me which is “the organized set of attitudes of others which one himself assumes,” and an I in which “we surprise ourselves by our own action.” Such a distinction between cognition and action points to a way out of the “barriers of meaning” mentioned in the previous section, a distinction which is currently realized to some extent in the technical innovation of generative adversarial networks (GANs) at the core of the “creative AI” movement; such architectures are composed of two neural networks, a discriminator that learns directly from examples and a generator that attempts to fool the discriminator (Goodfellow et al., 2014). Machine intelligence, in such a case, is capable not only of deviation, spurious correlation, and error, but all these at once, considered as a novel, unique act. As it turns out, if a program in the sociology of AI is to emerge, it would need a theory of novelty to occupy the central stage so that the encounter between humans and machines would become a significant experience seen through a meaningful lens.
Werner Binder’s contribution to this volume takes the form of a historical narrative of how games have been instrumental in the development of ML not only as an abstraction, but in the form of specific culturally rich and centuries-old games such as chess and Go. The apparently insurmountable complexity of the latter makes it the perfect challenge for machine learning; one might assume that whoever conquered it would conquer the world. This is indeed what happened in the mid-2010s, when DeepMind’s AlphaGo model attempted and ultimately prevailed at beating human professional masters. Match after match, event after event, the rising success of the machine amounted to a “cognitive breach” in that the reports of its technical breakthroughs were in and of itself a demonstration of a deeper, more symbolic debate regarding the nature of intelligence and creativity. As “deep play” or “social drama,” à la the cultural anthropologists Clifford Geertz and Victor Turner respectively, the stakes could not have been higher for the new wave of “deep learning.” AlphaGo embodied what it means for an automated model to combine meaning making and decision making almost seamlessly, and in doing so become an agent both epistemologically and ontologically. The strength of the argument is that it is able to detour around the philosophical debates about machine consciousness that have surrounded artificial intelligence for decades. Here, as Binder puts it, the “attribution of agency and intelligence on the basis of an entity’s performance has [the] advantage [that] it remains agnostic regarding the question of what intelligence really is.” So what kind of a genius player is AlphaGo? What are its “divine” moves and how were they described by the Go community in chat rooms and blogs? Many enthusiastic comments mention the creative and even aggressive style of the machine, with arguments in favor of an “intentional stance” showing “genuine strategic insight” that in turn “defies conventional wisdom,” etc. AlphaGo and subsequent iterations learned from self-play; they valued guessing and experience over mathematical prowess, and this is how it became not only successful but respected. The game of Go, all along, has been a reality test, one that DeepMind’s brand of machine learning has passed if judged by its “symbolic inclusion” into the broader social imaginary. In the end, that is what was at stake for ML at this crucial historical moment, i.e., the capacity to mingle within the lifeworld and cultural context—in a way, to make sense of its own situatedness.
Context Matters
Earlier in this introduction, we argued that machine learning is never truly devoid of meanings. To interpret and perform, to categorize and implement, is how ML technology finds a place in this world, its situations, and its contexts. As Seaver (2015) notes, “the nice thing about context is that everyone has it.” More interesting, however, would be to follow his lead further and consider that the “controversy lies in determining what context is.” This is what is finally at stake for ML, namely, the (in)capability to make sense of the fact that context always matters. The question is thus about reflexivity, and the problem is thus about problematization. In a series of recent high-profile interactions and debates between AI researchers Yoshua Bengio and Gary Marcus, the latter criticized ML for its inherent difficulty in engaging with the causal aspects of reality. Marcus’ complaint that deep learning has no way of handling compositional thought processes and no way to incorporate and depend on background knowledge is effectively a cognitivist’s way of saying that these models do not handle context. Tacit understanding, collective representation, and symbolic interaction remain of the utmost importance in social reality despite being ignored for most of the history of ML development. So what is it then that the social sciences can do? What position could they occupy that would not necessarily come to help or save ML, but that would reaffirm the social sciences’ own contextual status, i.e., the field’s always complicated self-positioning in terms of scientificity and reflexivity? This is also a problem about problematization, one that a social theory of machine learning would have to grasp epistemologically as well as ontologically. A possibility here—albeit by no means the only one—could be to be more sensitive to hermeneutical tenets, such as the importance of experience, the active role of interpretation, and the historicity of knowledge. If digital hermeneutics has seen a resurgence of interest of late (e.g., Capurro, 2010; Hongladarom, 2020), it is in part because machines increasingly come to perform acts of interpretation by transforming and transducing signs into other signs, which may then be taken up for interpretations by others. One can then try to apply simple, yet powerful principles such as, for example, the concept of the hermeneutic circle between the whole and the parts; as Gadamer (1975/2004) famously puts it, “The anticipation of meaning in which the whole is envisaged becomes actual understanding when the parts that are determined by the whole themselves also determine this whole.” But this is not the type of interpretation that machine learning is attempting at present. Facial or emotion recognition, to give two examples, fundamentally lack an understanding of why faces are historically and axiologically so important in social interaction; what is more, they lack the capacity to reflect on the discriminatory traditions that come with “calculating” facial features as well as awareness of the fact that they reproduce such biases and prejudices (Buolamwini & Gebru, 2018; Introna, 2005; Stark, 2019). Putting the machine learning encoding of “bias” and “prejudice” in perspective is to problematize them; it is to question their broader meaning as a way to gain some distance and, ultimately, some reflexivity.
Issues dealing with “prejudice” are sure to be central in any discussion about context. A fairly common mistake made by those who have studied the polemics between Gadamer’s hermeneutics and Habermas’ critical theory during the 1960s and 1970s, for instance, is to think that Gadamer was in favor of “prejudice”—what he calls the Vorstruktur des Verstehens, the prestructure of understanding—and the authority of tradition, while Habermas was diametrically opposed (see Apel, 1971; Habermas, 1967/1988). In this debate, however, Gadamer never stated that prejudices should be “accepted,” but rather that they should be recognized for their entrenchment within a historical condition. And it is these historically entrenched biases that even the social sciences cannot completely free themselves from; Habermas agreed with this, but also insisted that it is possible to see that these historical conditions are rooted in power and coercion. Numerous such convergences exist between hermeneutics and critical theory, the most important of which is that a comprehension of history introduces some sort of critical distance and, at the same time, critique can be supported by a reinterpretation of cultural history (see Bubner, 1975; Hekman, 1986; Roberge, 2011). This, in turn, sheds new light on ML’s deficient handling of bias and prejudice. The problem in the current state of affairs is that racist and sexist preconceptions, for example, are treated as if they were a glitch to be fixed technically, one which is before and thus separate from the computational core of ML. In contrast, a critical hermeneutics-based approach would focus on the entire social construction of ML as an end-to-end problem, addressing not just how bias and prejudice are molded within ML models but how they are molded in those who seek ML as a solution to social problems. Self-interpretation and critical understanding go hand in hand here, which is also to say that a coherent and reflexive account of ML in general, and ML’s relation to bias and prejudice in particular, is by its very essence a political one. As we have said, ML is a vastly interested form of knowledge. Its situatedness is bound with a unique historical moment of political economy, and this book is a testimony to how it could be possible to make sense of this idea. The purpose is not necessarily to align with one specific thesis—for instance, the rise of surveillance capitalism à la Zuboff (2019)—as much as to ask tough questions regarding the kinds of value extraction that are now developing, the dynamics of ML’s rampant development in contemporary life, and the legitimacy that is given to its deployment in a tripartite and very much intertwined cultural, economic, and political sense.
Théo Lepage-Richer’s contribution to this volume explores the limits inherent to ML’s knowledge from a historical as well as epistemological perspective. He locates key moments in which the idea of neural networks gained momentum, and examines how researchers came to conceive of the analogy between the brain and the computer as a way to attempt to overcome their respective limitations. A first such moment is the development of the McCulloch-Pitts model in the mid-1940s, in which its two authors posited that a new experimental science could quantitatively determine the operation and thus self-regulation of a mind, be it embodied in the human brain or formed of mechanical wiring. Epistemologically, the model instituted a “shared conceptualization of the unknown as something that can be contained and operationalized by networks of interconnected units.” For Lepage-Richer, the model’s logic was then picked up in a second, yet closely-related moment corresponding to the emergence of cybernetics in the late 1940s and 1950s. In the work of cyberneticist Norbert Wiener, for instance, a very similar endeavor against uncertainty led him to conflate knowledge and order; when he talks about “the great torrent of disorganization,” for instance, he displays a sort of philosophy of nature. Yet more striking is the fact that cybernetics is historically inseparable from the context of the Cold War. Control and communication, for example, were reformulated “in light of new needs and imperatives [where] the laboratory and the battlefield quickly emerged as interchangeable settings in terms of how knowledge was redefined as an adversarial endeavor.” From there, the genealogy toward a third, more contemporary moment is quite ambiguous; while today’s AI continues to “work” on adversaries, such adversaries have been, as mentioned above, incorporated into the very architectures of neural nets. Targeted perturbations are willingly introduced to expose and then overcome blind spots, as in the case of cybersecurity or the “adversarial examples” that disrupt contemporary neural networks (Goodfellow, Shlens, & Szegedy, 2015). “Failures are [now] framed,” the chapter notes, “as constitutive of neural networks by providing the opportunity to improve future iterations of these systems.” And this is where the entire epistemology of connectionist machine learning becomes so problematic, namely, the notion that nothing can truly lie beyond neural networks, not even social and political issues; ML’s own failures in terms of biases are effectively recycled and washed away. Instead of being collectively addressed, their consequences are becoming not only understood but accepted as simple engineering problems.
Orit Halpern’s contribution to this volume also looks at how the management of uncertainty is the new “core business” of ML. She is especially interested in understanding how today’s massive infrastructural endeavors present themselves as a sort of doubling-down, in which an increased penetration of computing allows for an expansion of both science and capital. The Event Horizon Telescope array is her initial case in point; in 2019, its tremendous sensory capabilities allowed it to capture the first image of a black hole using the Earth itself as a sensory recording medium. How is this instrument of extraterritorial political economy able to change what it means to govern life on Earth? For Halpern, the answer to this important question lies in a very practical setting, namely, the Atacama Desert in Chile, which is the location of both the “sublime” infrastructure of radio telescopes and one of the largest lithium mines in existence. The metal is highly strategic as it is one of the lightest and of increasingly high demand in a world of battery-powered devices and vehicles. In Chile, these lithium mines have been privatized along with their water sources, and it is now up to corporate actors to privately handle the environmental footprint of this exploitation. “This new infrastructure of corporate actors,” Halpern notes, “[merges] high tech with salt and water in order to support our fantasies of eternal growth.” The rise in extraction comes with a rise in an interest in optimizing; applied mathematics and machine learning are deployed in reinforcement. Endless data organized in endless loops, in other words, are there to make the best out of a finitude of resources. Yet, this is not the sum of the Atacama Desert story. During the Pinochet dictatorship, it was used as a site for the summary execution of dissidents. Human memories—those of mothers searching for their children’s bodies—thus mesh with the more abstract form of spatial and practical exploration and exploitation of the land. For Halpern, these convergences and their reformulation of time and space create the conditions under which a truly reflexive critique of both modern imperialism and artificial intelligence could (re-)emerge and forge a new meaning for what counts as our human horizon.
In the final contribution to this volume, Luke Stark, Daniel Greene, and Anna Lauren Hoffman propose to contextualize today’s primary discourses about the ethics and governance of ML and AI systems. A critical perspective would have to see these discourses as belonging to a broader frame of culture, economy, and politics, especially as these, by their very definition, imply legitimacy and power. A first and rather common proposed solution is to develop more tools and technical fixes, as discussed above; but “fairness,” the control of bias, and the like often leave aside important dimensions of the problems they address, so that, for the authors and others in the STS and critical legal studies literature, they turn out to be “entirely insufficient to address the full spectrum of sociotechnical problems created by AI systems.” Another fairly common type of governance posits itself through axiological principles, as is the case, for instance, in the Vatican’s recent “Call for AI Ethics.” The problem then is that in declaration after declaration, the same abstract principles get promoted alongside what is also often a self-promotional attempt by political actors to justify themselves. States and their regulatory bodies face similar issues when it comes to regulating and offering standards. They too are reluctant to propose drastic changes to a field they only partly understand and which they more often than not see as a site of economic growth and/or global competition, including in military terms, and also fail to “address the wide range of AI-equipped analytic technologies designed to surveil elements of human bodies and behavior.” Approaches based on human rights tend to have stricter boundaries, but they nonetheless have difficulties grappling with the structural logics that would need to be addressed such as the incompatibility between human rights covenants and corporate systems manufacturers; approaches based on cybersecurity entail similar conflicts. For the authors, what all of these approaches to AI/ML governance have in common is to “seek to restrict debates around the societal impact of AI to a coterie of technical experts whose positions are posited, chiefly by themselves, as technocratic and thus apolitical,” and this means it is necessary to develop alternatives, emphasizing communities of practice, which might be inspired by recent labor action by tech workers or by the call for mass mobilization in the abolitionist policies of the Movement for Black Lives. What role can social justice take in ML/AI cultures? How can affected communities come to have a voice in matters of governance? These questions are difficult but essential in the present context.
To understand the cultural life of machine learning is certainly an open endeavor. What the eight chapters assembled here have in common is to problematize the internal logic of these techniques and technologies, but also to offer unexpected possibilities to think anew our relationship with them. For instance, the chapters all point toward the question of who, in the end, is the receiver and thus the destination of the varied outputs of deployed machine learning systems. Keeping in line with hermeneutics, it is not unrealistic to say that questions about meanings and significance are questions about appropriation and self-understanding. Individuals can actively make sense of these technologies as they are not only the users and consumers of their various enhanced platform settings—“sharing” their ML-massaged data—but are also their real-time interpreters, social actors, and citizens. And this is where issues at an individual level connect with issues of the collective and where questions related to processes of appropriation connect with questions about processes of reflexivity. To be reflexive in the brave new world of ML is to constantly interrogate its purposes, probe its actions, and reassess its broader consequences with the hope that the knowledge being produced might somehow alter its course. Reflexivity is agentic in that regard; and it offers a different kind of “quest for agency” than that of the machine learning models discussed above, one that would have a deeper sense of the politics at play. To (re)politicize the context, to say it matters, is thus to recognize that collectivities are being shaped by ML deployment as much as these collectivities have the capability to shape ML, most notably in the direction of social justice. This, in turn, implies the necessity for increased dialogue and public debate. It also implies that social science scholars currently vested in the exploration of ML will need to constantly reassess their own (social) role. On the one hand, doing so may very much be about finding better ways to deeply engage with these technologies, i.e., to be hands-on and perform closer and more interested examinations of its contingencies. On the other hand, the critical study of ML/AI deployment might have to find its own stance with the appropriate distance to shed its own specific light on the phenomenon. Are calls for both proximity and distance then antithetical? Does this tension force a double bind that would be problematic to the future of the critical study of machine learning and artificial intelligence? Earlier in this introduction we proposed that it might be better to embrace such tension‚ and the following chapters reflect their authors’ willingness to be daring in this regard.