Greg Lusk
People increasing rely on social technologies to make all sorts of social decisions. Examples from daily life abound: smartphone apps estimate the density of cars and their speed in order to detect traffic and suggest time-saving routes; websites compare home prices and neighborhood tax information to estimate values for would-be buyers; and fitness trackers count our steps, swimming strokes, and hill climbs against our personal health targets, automatically posting successes to social media. The algorithms, and the quantitative data they rely on, give users knowledge that previously required consulting the traffic desk of the local radio station, real estate agents, and personal trainers. Armed with this knowledge, users can make decisions “for themselves,” which is often equivalent to trusting such technologies to decide for them.
Of course, the desire for easily understood information that could enhance decision-making is not reserved for individuals driving to the airport or buying houses. In his influential history of quantification, Trust in Numbers (1995), Theodore Porter locates the prestige of quantification in its ability to support social technologies that tame subjectivity and enhance managerial power. As he sees it, the widespread emphasis on numbers in public spaces is not likely to be merely a case of “physics envy”: businesspeople, administrators, and government officials use social technologies like cost-benefit analysis to assert control over the decision-making process by reducing reliance on experts. Quantification helps enable such technologies, in turn enabling the managerial and administrative class to make decisions for themselves, displacing once-necessary expertise. Viewed in this way, quantification, and the processes that utilize it, plays a central role in shaping social decision-making by providing information that foregoes subtlety and depth to achieve usefulness.
From Porter’s work, it is easy to see why the Original Critique of numbers in decision contexts—in short (and as outlined in the introduction to this volume) that the seductiveness of numbers obscures nonquantifiable aspects of phenomena—was so powerful. Not only did certain methods of quantification render invisible that which could not be expressed by numbers, but those at the helm of this process were often those with substantial power. As a consequence, other methods of quantification were sometimes quelled, forgotten, or ostracized, along with the less powerful who might have benefited from these different numerical approaches. Given this, it seems unsurprising that there is now a growing populace skeptical of experts and their currency of quantification in the United States and United Kingdom. Some within this skeptical movement feel left out of the equation. This skepticism makes clear what has always been true, but was not always emphasized in the Original Critique: Who or what is quantified is always a matter of decision. Quantification itself is neither virtuous nor vicious; the character of numbers depends on how they are wielded. Good numbers can be useful and bad ones can easily mislead or obscure.
How do we make and use numbers in a responsible way? This chapter begins to address that deceptively simple question. I will show that there is a central insight shared between those who articulated the Original Critique (using Porter as an exemplar) and certain views of measurement in philosophy of science. The shared insight is that quantification is itself perspectival. Numbers—despite claims to objectivity—always carry with them a certain orientation toward that which they represent. This orientation connects the ethical impacts of quantification to the epistemic perspective that such numbers promote. In such a case, then, one sign of a virtuous method of quantification would be alignment between quantification’s representational capacities and laudable social ends. Achieving this alignment is harder than it may seem, as I show through a coupled ethical-epistemic analysis of extreme weather attribution. Extreme weather attribution is a developing methodology in climate science that proponents argue is a promising social technology that can be used to spur adaption and climate justice. This analysis shows how and why certain numbers may or may not align with their intended purposes, but also how one might begin to assess the virtues and vices of quantification that bridge science and decision-making.
Porter (1995) advances two theses in an attempt to explain the prestige and power of quantification in the modern world. The first thesis claims that the arrow of explanation runs counter to what might be intuitively supposed: the appeal of quantification to businesspeople, administrators, and government officials explains aspects of numerical reasoning in the natural and social sciences, rather than the other way around. The second thesis claims that quantification smoothes over the need to perform deep and nuanced analysis in a way that allows relatively unskilled persons to make decisions that functionally replace expert judgment. Following Ed Levy (2001), I will call the first the reverse transfer thesis and the second the judgment replacement thesis.
For Porter, quantification often serves as a social technology that drives a practical imperative he calls “the accounting ideal” (1995, 50), which is crucial for the management of people and nature. Porter explores this ideal by examining historical episodes, most notably through a comparative examination of the adoption of cost-benefit and risk analysis by French state engineers and the US Army Corps. However, one simple example, which Porter (47–48) borrows from William Cronon (1991), demonstrates how the accounting ideal furthers judgment replacement: prior to 1850, wheat in the Midwest United States was shipped via river in bushel-sized sacks that differed in weight due to the density of how they were packed. Each of these sacks was examined individually by a miller or wholesaler in order to establish its value. There was at this time hardly any common notion of the “price of a bushel of wheat,” or if there was, it was known only to those who routinely established its value. But by 1860, the Chicago Board of Trade had defined the bushel in terms of a standard weight and divided wheat quality into four separate grades. Once quantified, wheat could be bought and sold on the Chicago Exchange by those with comparatively little knowledge about wheat production or quality. Porter sums up the episode nicely: “The knowledge needed to trade wheat had been separated from the wheat and the chaff. It now consisted of price data and production data, which were to be found in printed documents produced minute by minute” (1995, 48). No longer was the judgment of a miller needed to price wheat; the locus of wheat-valuing expertise had shifted to traders and investors.
Quantification replaces judgment, on Porter’s account, by establishing a set of rules that enable “mechanical objectivity.” Such rules displace the subjective judgment and beliefs of individuals from the process of social management, replacing them with recipes for quantifying objects or processes of interest. When a community is small and its members trust each other, quantification is largely unnecessary. What quantification provides, via mechanical objectivity, is the capacity for management at a distance: numbers become tools of communication whose objectivity establishes their authority and usefulness. Numbers produced according to rules allow those without expertise to engage in practices that previously required expert judgment, as the wheat example above shows. Porter notes that when quantification successfully proceeds in this way, it “nearly always comes at some cost to subtlety and depth” (1995, 86).
Thinking about quantification as a means of refining the social division of labor through the displacement of judgment, as Porter does, is not traditionally the approach taken in philosophy of science. The default approach there, if there is one, is to view numbers, when produced by trustworthy measurements, experiments, and computer simulations, as representing phenomena in a way that, at least under certain conditions, allows them to be shared as empirical evidence among researchers.1 Thus philosophers have been much more interested in characterizing the general inferential patterns—the judgment that is often replaced on Porter’s account—that result in successful mathematical representation of physical or social phenomena. The power of quantification, for many philosophers, is located within the deep and nuanced reasoning that produced particular numbers that successfully reflect phenomena out in the world. As Ed Levy (2001) points out, quantification is often the result of rigorous expert judgment, not a lack of depth and subtlety.
However, these two views of quantification are more complementary than critics like Levy would make them seem. After all, there is a great deal upon which many philosophers of science, and Original Critique historians like Porter, agree. Take, for example, the newly revived philosophy of measurement, whose goal is to provide an epistemology for the practice of representing phenomena by numbers (an area long neglected by philosophers of science). One of the lessons emerging from this literature is that measurement “provides a representation of the measured item, but also represents it as thus or so” (van Fraassen 2009, 180). For example, when measuring the temperature of a cup of tea, the measurement represents the tea as it “looks” from within a particular measurement setup in a particular measurement environment. What vantage point to look from—at least before the kind of standardizing norms that result in mechanical objectivity are established—are largely left to the judgment of scientists. Measurement, and in fact numerical representation writ large, is thus a selective form of representation: how the represented object is quantified will be a result of the way which it is “viewed.”
Where the Original Critique of quantification reflected in Porter’s work and the representational approach typical of philosophy of science come together is precisely on this point regarding the inevitability of perspective: numerical reasoning always has a specific vantage point. Of course, the vantage point emphasized in these literatures is different. Proponents of the Original Critique emphasize the vantage point of social actors who use quantification to shift the locus of judgment, while philosophers of science have emphasized how numbers representationally carry with them a particular vantage point based on the decisions of their creators. Regardless, both admit that different vantage points may lead to different judgments. And while proponents of the Original Critique and philosophers of science acknowledge the important role of mechanical objectivity, mere adherence to rules of quantification does not remove the influence of perspective. The rules that underwrite mechanical objectivity encode the choices that give numbers their perspectival character. In many ways, these complementary perspectives are two sides of the same coin.
When answering normative questions about when it is permissible to allow numbers to guide social decisions—at least those numbers that purportedly represent physical phenomena—a dual analysis therefore seems to be in order. One needs to examine both the goals that such decisions aim to achieve and whether the kinds of quantifications involved are fit to achieve those goals. Thus, we can say that there needs to be alignment between two sets of vantage points—those that guide number creation, and those that guide number use. Setting aside for a moment the questions regarding the desirability of goals, this kind of analysis serves as a minimal criterion that can help establish whether certain kinds of numbers are adequate for the purposes for which they are often used.2
In the sections that follow, I engage in a coupled ethical-epistemic analysis of extreme weather event attribution by way of demonstrating one approach we might take to normative questions about number use. Such analyses are a relatively new way of linking scientific research with social outcomes to help determine how research should be used (see Tuana 2013 and Tuana et al. 2012 in the context of climate science, and Katikireddi and Valles 2015 for a biomedical application). Coupled ethical-epistemic analyses aim to connect the methodological and epistemological aspects of scientific research with the ethical consequences that research has when it is used socially, particularly in decision-making. Such analyses acknowledge that science—despite its aim of objectivity—often involves the use of contextual and social values within its practice, especially when science aims for public relevance. Put in the vocabulary of this section, the choice of quantitative perspective is often a value-laden one that has consequences for the way research should be used when addressing social problems. Coupled ethical-epistemic analyses are useful because of their dual character. I will deploy this kind of analysis here to demonstrate both that the epistemic choices made in quantification often have ethical consequences, and that our ethical or social choices should inform our means of quantification.
There is little doubt among climate scientists that the observed trend toward a higher global mean temperature is largely the result of human actions (Cook et al. 2013; Oreskes 2004). Anthropogenic forcings, dominated by the release of greenhouse gases as well as land-use changes, are responsible for altering the Earth’s radiation budget by trapping a larger amount of thermal radiation in the atmosphere. This trapping of thermal radiation—that is, heat—is commonly known as the greenhouse effect. Less heat leaves the planet’s atmosphere, resulting in higher global mean temperatures. As the latest Intergovernmental Panel on Climate Change (IPCC) report indicates, there is an unequivocal human influence on the climate system that has likely already committed the planet to at least a 1.5°C rise in mean temperature (Stocker et al. 2013).
While scientific investigations of climate change often take place on the global scale, scientists are equally interested in the repercussions of an altered atmosphere on regional and local scales. One area of significant concern is extreme weather, such as heat waves, cold snaps, droughts, deluges, and hurricanes. These can cause large-scale destruction of human infrastructure, resulting in costly physical damage and the breakdown of food systems and supply infrastructure. Extreme weather event attribution attempts to do for these events what scientists have already done for the rising trend in global mean temperature: show that they are influenced by human actions.
Such attribution involves assessing how the properties of extreme events have changed given anthropogenic influences. Often the change that scientists are interested in is the frequency of occurrence of a particular type of event in a particular place. The type of event is defined by threshold exceedance in a specific region of the globe, for example, the record high July mean temperature in western Europe. Scientists may use the old record high temperature as a threshold to assess how the probability of that record being broken has changed due to anthropogenic forcings. This form of event attribution, known as probabilistic event attribution, involves establishing the frequency of occurrence of a particular type of event in the “natural world,” where anthropogenic forcings are absent, and comparing it to the frequency of extreme event occurrence in the “actual world,” where anthropogenic forcings are active. The difference in the frequency of occurrence between the two worlds can therefore be blamed on the presence of anthropogenic forcings. This “blame” is the attribution: one attributes the observed increase in probability of occurrence to anthropogenic factors. This is a risk-based approach, in that it says how much more likely a type of event is given anthropogenic climate change (i.e., how much more “risk” of occurrence there is).
There are at least two methods of computing the frequency of event occurrence in the natural and actual worlds. The first might be referred to as the “classical statistical” approach (Hulme 2014) and the second the “model-based” approach. In the statistical approach, time series are used to detect outliers or changes in trends. For example, Luterbacher et al. (2004) reconstruct European monthly and seasonal temperatures from a host of sources, including proxy data. The reconstruction is used to demonstrate that summers like that of 2003 are outliers, and 2003 was likely the warmest European summer since 1500. Scientists often use these reconstructions to calculate the return period of such events. Luterbacher et al. calculate the return period for a 2°C summer anomaly at millions of years for early twentieth-century conditions and less than 100 years for recent summers (2004, 1502). The implication is that the change in return time over the twentieth century—that heat waves of this kind are returning significantly more frequently—should be credited to anthropogenic global warming.
Another means of computing these frequencies involves the use of computer models to calculate the changed odds of an event. This approach involves running one set of computer simulations of the climate system (or a subregion of that system), which explicitly represents anthropogenic forcings, and another set with the same structure except with the anthropogenic forcings removed. Scientists use the products of these two sets of simulations to estimate the probability of a particular kind of event in the current climate, and in a nonanthropogenically altered climate. They then compare these two probabilities and compute what is called the “fraction of attributable risk,” or FAR. This quantity tracks how much of the current risk of event occurrence is due to anthropogenic forcings; for example, if FAR = 0.5, then half of the current risk of the event occurring is due to anthropogenic factors or, put another way, the probability that an event of a defined type will occur has doubled.
It is important to note that probabilistic attribution, regardless of the approach, does not attribute an actual weather event to anthropogenic factors. In fact, it is admitted that any particular extreme could have happened completely naturally. Probabilistic event attribution attributes only the increased probability of occurrence to anthropogenic factors. Thus, one cannot “blame” any particular occurrence of an individual extreme event on anthropogenic forcings; one can attribute only an increased chance of a particular kind of event.
It is also important to note that probabilistic event attribution is not forward-looking, and says nothing about the future. Probabilistic event attribution merely compares how the current risk of an extreme event differs from the risk of an event in the natural scenario (or, in the case of the classical statistical method, some prior point in time when anthropogenic forcings were negligible). It says nothing about how the future risk of extreme events might differ from today’s risk. After all, one cannot attribute something that has not yet occurred. Nonetheless, scientists want to use the practice to garner trust among the public and decision makers by replacing the sometimes contentious expert judgments of climate scientists.
The case of extreme weather attribution is interesting because it has been deliberately positioned as a social technology that will support social decision-making. As a National Academy of Sciences report indicates, “the primary motivation for event attribution goes beyond science”; it is to provide valuable information to emergency managers, regional planners, and policymakers who need to grapple with a changing climate (National Academies of Sciences, Engineering, and Medicine 2016, ix).
Most of this positioning has been done by a small group of probabilistic event attribution proponents affiliated with the University of Oxford and the United Kingdom’s Met Office, most notably Myles Allen (Oxford), Friederike Otto (Oxford), and Peter Stott (Met Office). They seemingly subscribe to Porter’s reverse transfer thesis: they have argued that extreme weather attribution is valuable because it would allow for enhanced management of the impact of global warming by those outside of climate science. As Myles Allen has stated, “Because money is on the table, it’s suddenly going to be in everybody’s interest to be a victim of climate change. . . . We need urgently to develop the science base to be able to distinguish genuine impacts of climate change from unfortunate consequences of bad weather” (quoted in Gillis 2011). In other words, because decision makers are going to face a whole host of questions about how to distribute resources in a changing climate—including questions about compensation—we should adopt scientific strategies of quantification so that those decisions can be made efficiently and fairly. This message of social usefulness dominates discussions about extreme event attribution; scientific benefits are scarcely mentioned.
Allen and Stott recognize that in order for event attribution results to replace specialized judgments for the purpose of social management, the results need to be seen as objective. Allen and Stott argue that probabilistic event attribution could constitute a “relatively objective” approach to extreme weather so long as an “industry standard” methodology—like the one they propose—is adopted (Allen et al. 2007). In a strategy straight out of Porter’s playbook, they argue that adopting such a standard would eliminate bias and minimize the need for expert judgment.
Allen and his colleagues want to avoid inaction due to dueling experts: climate change is a contentious issue, and at least on the finer details, the experts sometimes disagree. Such disagreement is often seen by the public as a sign of more general uncertainty, which leads to social inaction. By adopting an industry standard—or as Porter puts it, instituting some form of mechanical objectivity—Allen and colleagues seek to replace expert judgment by algorithmic calculation. Unsurprisingly, the judgment they want to retain is their own, as encoded in the very form of probabilistic event attribution they developed. In essence, probabilistic event attribution packs the judgment of a subset of scientists into a numerical figure to enable social decision-making.
Scientists, particularly those working in the spirit of Allen and Stott, have touted numerous areas of social need that would benefit from extreme weather attribution. The following sections examine the fitness of probabilistic event attribution for the social ends which motivate its development. Specifically, I examine if extreme event attribution can meet the following goals: (1) provide information about the future risks of extreme events to emergency managers, regional planners, and policy makers; (2) help society build adaptive capacity and ensure proposed adaptations target the right kind of events; and (3) attribute particular events so that victims of anthropogenic climate change can be recognized and distinguished from victims of naturally extreme weather—cases of bad luck—and perhaps also be compensated.
The need to ensure alignment between the means of number production and the proposed or actual use of such numbers is evident when considering goal 1 above, the use of probabilistic event attribution to inform decision makers about the future risks of extreme weather. There seems to be a conceptual confusion within the broad scientific community and science journalists regarding the character of event attribution studies that could mislead potential consumers of the information. For example, the National Academies of Sciences, Engineering, and Medicine (2016) make a claim that is often repeated in popular conversations about event attribution: a “defensible analysis of the attribution of extreme weather events . . . can provide valuable information about the future risks of such events to emergency managers, regional planners, and policy makers” (ix; emphasis added). Similarly, Stott et al. (2013) suggest event attribution could help guide resource allocation, preventing investment in protections against weather events that will decrease in severity in the future.
However common, statements like these are misleading since, as mentioned earlier, weather attribution studies do not say anything about the risk of future events, and are hence distinct from predictions or forecasts. Weather attribution only provides information about how present-day risk differs from risk in a natural environment (or past environments) (see Lusk 2017 for a more detailed argument on this point). Unless assumptions are made that the climate is static (which is known to be false), weather attribution does not have direct policy relevance where estimates of future risk are required. The perspective of probabilistic event attribution does not align with the goal of informing decision makers about future risks. Thus, we begin to see the value of a dual kind of analysis for adjudicating the value of methods of quantification for furthering our social goals.
Assessing the alignment between the methodology of event attribution and goals 2 and 3 requires deeper analysis. For goal 2 to be viable given the backward-looking character of extreme event attribution, the methodology would have to provide a fairly accurate view of how the present-day risk of certain kinds of extreme events has changed. This ties the social usefulness of event attribution to its reliability: unreliable attributions could lead to poor ethical decisions and maladaptive actions (Christidis et al. 2013). Similarly, if we want to separate victims of anthropogenically attributable extreme weather from those who suffered from naturally extreme weather for the purposes of recognition or compensation, as in goal 3, then probabilistic event attribution should be accurate and capable of attributing the breadth of event types that victims are likely to suffer from. Yet while many scientists (e.g., Thompson and Otto 2015) tout the reliability of event attribution, philosophical and scientific work rightfully questions whether confidence in event attribution results is justified.
While a full analysis of the accuracy of event attribution is beyond the scope of this chapter, examining two issues that arise when computer simulation models are used in attribution illustrates how problems of accuracy might result in a misalignment between attribution and its social goals. The first issue arises from the use of observational data to validate the reliability of the models used in attribution, and the second arises when scientists attempt to account for various kinds of uncertainties that emerge from the use of models. I examine each of these issues in turn.
First, scientists test the skill of their models against observational data for situations related to the ones of interest. Thus, they might deem a model adequate for attributing an extreme event if, in the region of investigation, the model reproduces the observed statistics for the relevant quantity (i.e., the distribution of temperature, precipitation, etc.). The results of a model are checked against the historical record: model adequacy is established when the distribution produced from an ensemble of many model-runs of a historical period matches the observed distribution for that period for the quantity of interest.
Tests of adequacy like this face two challenges. First, they rely on the reproduction of fairly robust statistical quantities that may not be sensitive to the weather extremes that scientists seek to attribute.3 When scientists check their models, they want to see them reproduce the observed distribution across the entire range, including the tails where the extremes lie. However, scientists acknowledge that observations of extremes are too sparse to draw definitive conclusions regarding model adequacy for the extreme part of the distribution. Thus, they rely on other properties of the distribution to test their model. Stone and Allen (2005) write, “the only option is to look at more robust measures of the physical quantity, like its mean and variance, and assume that verification of our understanding of the physics and forcings relevant to these measures applies to its extremes as well” (313). The problem is that goodness of fit between model output and the nonextreme observations does not necessarily indicate fitness of the model for attributing extremes—particularly near record-breaking and never-before-seen events—that tend to lie in the tails of distributions. A model may perfectly capture the extant observations, yet its capacity to predict extreme events could still be questionable. There is thus reason to doubt that such models capture extreme events with the desired reliability.
The second challenge is demonstrating that models succeed in meeting validation criteria for the right reasons, that is, that they capture the mechanisms responsible for the event type in question. Capturing these mechanisms correctly underwrites the justification that the probabilities gathered in the natural scenario are realistic and is therefore crucial to demonstrating that estimates of FAR are reliable. Getting the right answer for the right reasons requires properly representing the relations between the mechanisms in the structure of the model. Scientists are aware that their models contain what is known as structural model error (SME) due to the nonrepresentation of relevant features, or inaccuracies in the represented relations between features, and this knowledge of SME results in what are called “structural model uncertainties.” What exactly encompasses structural model uncertainty is still up for debate, but it is construed by some as uncertainty about what would constitute a perfect model and by others as uncertainty about what would constitute an adequate model structure for a particular purpose (see Parker 2010).
Event attribution studies typically employ only one model structure, and thus the potential impact of different model structures is unaccounted for in the (first-order) uncertainty associated with the results. It is possible that two models, which both adequately reproduce observed events, will differ in their predictions of events under the natural scenario. This possibility raises doubt regarding the accuracy of model predictions in the natural scenario, and is a source of what is called “second-order uncertainty” (uncertainty about uncertainty). Since these second-order uncertainties are potentially large, they undermine our confidence that we have an adequate model structure in this case. Thus, these uncertainties can be pernicious, and may undermine claims to the reliability of models used in attribution.
Just how pernicious they are is a matter of significant philosophical and mathematical debate. Recent results suggest that any SME might have a significant effect on attribution results. Frigg et al. (2014) show that some nonlinear dynamical models with SME are instable: small differences in model structure can lead to very different trajectories for the system. This is called the “hawkmoth effect,” in homage to the well-known butterfly effect.4 The notion of butterfly effect identifies a sensitivity to initial conditions: the same system begun from two close but nonidentical initial conditions may have trajectories that diverge significantly. The hawkmoth effect is similar but structural in nature: two very similar but different model structures, started from the same initial conditions, can result in very different trajectories. Hence, closeness in model structure does not imply accurate or useful results. The hawkmoth effect could arise in nonlinear dynamical climate models, and hence the hawkmoth effect jeopardizes the accuracy of probabilistic predictions of climate states. The kind of computer simulation models often used in event attribution are of the sort that might be susceptible to the hawkmoth effect, and thus if Frigg and his colleagues are right, such models would fail to be relevant for decision-making.
The actual impact of the hawkmoth effect is not above contention, however. Philosophers Winsberg and Goodwin (2016) place the burden of proof back on Frigg and his colleagues, challenging them to demonstrate that their conclusion has any impact on the use of nonlinear dynamical models as actually employed in areas like climate science. They claim that the impact of the hawkmoth effect is determined by the presence of SME and instabilities in the model, but also a significant number of other factors: the time scale of the desired prediction, the means by which the results are obtained, and the question under study. This topic deserves more philosophical and scientific attention, but clearly the result of the debate could have significant impact on the way extreme event attribution is understood.
Examining the accuracy of one kind of event attribution, we can see that there are good reasons to think that at least this kind of event attribution is not yet reliable in the way that goals 2 and 3 require. Given the arguments above, event attribution could lead to maladaptive decisions, and therefore we have reason to think that its ability to support goals 2 and 3 are limited at best. Furthermore, it is widely admitted that even the limited reliability that probabilistic event attribution might enjoy applies to a small number of event types—predominantly extreme heat and cold—in limited geographic regions (National Academies of Sciences, Engineering, and Medicine 2016). Whether or not the capacities of extreme weather attribution will ever align with the social motivations for its use is an open question, but at the moment we have reason to believe that such alignment has yet to be achieved.
So far, we have considered how the epistemic aspects of the dominant forms of event attribution align with the social goals that certain proponents of event attribution have advanced. But we can also ask whether the values that scientists have employed in developing probabilistic event attribution align with or further the social ends to which event attribution may be put.5
As Lloyd and Oreskes (2018) have pointed out with regard to extreme events, the preference to avoid errors of a certain type have led to contentious debates about extreme attribution methodology that have a significant bearing on how event attribution studies can be used socially. Demonstrating how the chance of event occurrence has changed due to anthropogenic forcings is not the only way to attribute extreme events to climate change.
The so-called storyline method, developed more recently by Trenberth et al. (2015), attempts to clarify how particular events would have been different in a nonanthropogenically forced environment. Rather than look at how often a type of extreme event occurs, the storyline approach assumes the occurrence of a particular event and then attempts to assess how the atmospheric conditions that gave rise to that event would have been different in a “natural” climate. These assessments rely heavily on theoretical knowledge, particularly thermodynamical relations like the Clausius–Clapeyron relation, which specifies how much more moisture the atmosphere can hold per degree of warming. This approach therefore examines questions regarding why a particular event behaved as it did or what climate factors played a role in making the event so extreme—questions that are both conditioned on the occurrence of the event.
For example, Trenberth et al. (2015) analyzed a five-day extreme rainfall that occurred in Colorado in September 2013, which resulted in significant flooding in the area. Trenberth and colleagues reasoned that the event’s development was significantly impacted by unusually high sea-surface temperatures thought to be the consequence of climate change. These temperatures were likely responsible, at least in part, for where and how the storm formed and for the significant amount of moisture it was able to carry and then release onto Colorado.
Proponents of risk-based probabilistic event attribution have vehemently objected to the storyline approach. Such proponents claim the storyline approach—given that it tends to examine thermodynamic features of storms at the expense of dynamical ones—would result in a failure of scientists’ mission to serve society because almost every extreme event could be connected to climate change. As Allen (2011) notes, “blaming all weather events on climate change is not helpful either” (931). If so, this practice would jeopardize many of the social aims touted by proponents of the risk-based approach. Misattributing extreme events might also damage the credibility of scientists.
What Lloyd and Oreskes (2018) demonstrate is that much of the disagreement between advocates of the storyline approach and advocates of the risk-based approach rests on nonepistemic values that come to influence attribution science. For example, the two camps seem to disagree on what kind of information the public needs to be given; those advocating for a risk-based approach claim that “attribution analysis aiming to answer questions of most relevance to stakeholders should . . . follow a holistic [FAR] approach in order to prioritize an understanding of the changes of overall risk rather than the contribution of different factors” (Eden et al. 2016, 9; see also Otto et al. 2016). This reasoning—that the storyline approach fails to deliver the requisite information—leads to the view that permitting it would be a failure of scientific duty. But of course, the storyline advocates disagree.
Social values are similarly at the root of risk-based attribution proponents’ claim that the storyline approach is not scientifically fit for social decision-making. Scientists operating under the risk-based paradigm value rigor and ensuring the number of false positives is small, even if this entails the risk of understating effects (type II error), in order to maintain scientific credibility. Scientists operating under the storyline approach have a different attitude to error: they are more comfortable with potentially overstating the effects of human actions (i.e., type I error). They value more highly tools that are likely to detect anthropogenic influences, even if it means false positives. The point to note is that the two methods of quantifying extreme weather are based on two different types of value commitments. In order to decide which should guide policy, one needs to know what goals a society is committed to.
If scientists’ values may be carried with the quantified results they produce, there is a real danger that they may have an influence that goes unnoticed when used for social decision-making. As Porter points out, in many contexts numbers have an allure: once something is quantified, it seems natural to engage in judgment replacement. The ease with which numbers can replace judgment makes engaging in a coupled epistemic-social analysis all the more important.
Take, for example, goal 3: using risk-based extreme event attribution to identify the victims of climate change for ethical purposes like recognition and compensation (Thompson and Otto 2015). Risk-based methods, if they are accurate, quantify the change in frequency of a particular kind of weather event. In order to be compensated for damage incurred as a result of an extreme event, it needs to be demonstrated that the likelihood of the event had changed to some sufficient degree over time.
It would be easy then to put the burden on individuals wanting relief to demonstrate that the kind of event from which they suffered was anthropogenic in origin; that is, to require those harmed by extreme events to produce the right kind of numbers showing that the likelihood of the damaging event had increased. But of course, there are other ways to be victims of extreme weather for climate-change related reasons. The fossil fuels that power climate change, and the economies they run, have far-reaching consequences for land use, infrastructure, and culture that can significantly alter one’s level of vulnerability. The same anthropogenic actions that cause climate change can lead to vulnerabilities that result in harm due to extreme weather events, regardless of whether the odds of those events have shifted. As Hulme et al. (2011) note, climate action is needed where vulnerabilities are high, not where storms are most attributable to climate change. Proposals that rely on risk-based analyses are likely to overlook victims of climate change who seemingly deserve recognition or compensation in the wake of extreme events but cannot produce the right kind of numbers (for more, see Lusk 2017). We need to articulate what kind of climate action should be achieved before we can say whether or not event attribution is sufficient to separate victims from nonvictims.
I have argued here that processes of quantification bestow on their results a certain perspective, despite attempts to achieve mechanical objectivity. This perspective is an important element that must be addressed when assessing the normative implications of using numbers in social policy decisions. As a minimal normative criterion of adequacy, I suggested that there must be an alignment between the perspective embedded in the process of quantification and the social goals that such quantification is intended to support. To demonstrate how one might go about assessing this alignment, I performed a coupled ethical-epistemic analysis of extreme weather attribution, showing that many of the ends touted by proponents of risk-based assessment seemed difficult to achieve given the current state of the science. The advantage of this kind of assessment is that it can make manifest which goals could be supported by a particular method of quantification, as well as what a method of quantification needs to look like in order to support particular social goals.
My employment of this kind of analysis is advanced only as a first step to answering normative questions about what kinds of numbers should be appealed to in social decision-making. One major drawback of the way I have deployed the ethical-epistemic analysis here is that certain dynamics of decision-making involving quantification are ignored. I have assumed that goals are static and that the development of methods of quantification does not impact those goals. Put another way, I have assumed that our goals do not change just because there is a convenient number available. The Original Critique suggests otherwise, namely that there is an appeal to numbers that does influence agendas. The Original Critique is certainly right on this point. But seeing the goals as relatively static does have at least one advantage, in that we realize we need not be seduced by numbers. To adopt certain numbers is to adopt a vantage point, and we should not do that uncritically when attempting to advance, or better specify, our social goals.
1. Philosophical literature on measurement (see Tal 2013) and scientific inference from data (e.g., Bogen and Woodward 1988) embody this view.
2. Here I’m taking inspiration from the “adequacy for purpose” account of scientific modeling (see Parker 2009) which, in short, claims that the standard for scientific model evaluation should be adequacy for a specified use, rather than a true image of the world.
3. It is worth noting that the difficulty here in demonstrating model adequacy is unique to this practice and stems directly from its focus on extreme events. Many other applications of climate modeling do not encounter this problem because they deal with more robust quantities, like yearly global mean temperature.
4. Erica Thompson, a colleague of Roman Frigg and his coauthor Leonard Smith, originally coined the term “hawkmoth effect.”
5. Whether social values (e.g., value of human life, religious values, personal preferences) should or inevitably do influence science when producing quantitative results that are socially relevant, are topics of lively debate in philosophy of science (see Douglas 2009), especially when it comes to climate science (see Betz 2013; John 2015; Steel 2016; Winsberg 2012). The emerging consensus seems to be that social values are unavoidable in this kind of work, though there are ways to handle social values such that they play a legitimate role in scientific reasoning.