“Agreement” in the IPCC Confidence measure

Studies in History and Philosophy of Modern Physics ∎ (∎∎∎∎) ∎∎∎–∎∎∎

Contents lists available at ScienceDirect

Studies in History and Philosophy of Modern Physics journal homepage: www.elsevier.com/locate/shpsb

“Agreement” in the IPCC Confidence measure William Rehg, Kent Staley n Department of Philosophy, Saint Louis University, 3800 Lindell Blvd., St. Louis, MO 63108, USA

art ic l e i nf o

a b s t r a c t

Article history: Received 23 June 2016 Received in revised form 24 September 2016 Accepted 23 October 2016

The Intergovernmental Panel on Climate Change (IPCC) has, in its most recent Assessment Report (AR5), articulated guidelines for evaluating and communicating uncertainty that include a qualitative scale of confidence. We examine one factor included in that scale: the “degree of agreement.” Some discussions of the degree of agreement in AR5 suggest that the IPCC is employing a consensus-oriented social epistemology. We consider the application of the degree of agreement factor in practice in AR5. Our findings, though based on a limited examination, suggest that agreement attributions do not so much track the overall consensus among investigators as the degree to which relevant research findings substantively converge in offering support for IPCC claims. We articulate a principle guiding confidence attributions in AR5 that centers not on consensus but on the notion of support. In concluding, we tentatively suggest a pluralist approach to the notion of support. & 2016 Elsevier Ltd. All rights reserved.

Keywords: Uncertainty Confidence Agreement Consensus Climate science Support IPCC

When citing this paper, please use the full journal title Studies in History and Philosophy of Modern Physics

1. Introduction In its most recent Fifth Assessment Report (AR5), the Intergovernmental Panel on Climate Change (IPCC) takes pains to clarify its methods of estimating the uncertainty of its various claims about climate change and its impacts on ecosystems and society. The IPCC distinguishes two methods for estimating uncertainty: a quantitative likelihood scale, and a qualitative confidence scale, which authors were to use when they could not provide a quantitative estimate. In fact, the confidence scale is all encompassing, inasmuch as likelihood estimates presuppose at least a high level of confidence, unless stipulated otherwise.1 In assigning a level of confidence IPCC authors consider both the evidence (specifically, its amount, type, quality, and consistency) and the “degree of agreement.” High or very high confidence generally requires a high level of agreement and “robust” evidence. To date, philosophical treatments of climate science have focused heavily on the evaluation and evidential status of models n

Corresponding author. E-mail addresses: [email protected] (W. Rehg), [email protected] (K. Staley). 1 AR5, Working Group II Report, Summary for Policymakers (SPM), p. 6, Background Box SPM 3 (IPCC 2014a; hereafter AR5-II, 6). Note that all three working groups use the same uncertainty measures.

that simulate climate systems.2 But climate models make up only one source of evidence for IPCC reports, which also rely on other sorts of evidence, in particular observational evidence.3 In any case, the IPCC qualitative confidence scale has not received much philosophical attention.4 Though the evidence dimension of confidence raises interesting questions in its own right,5 in this essay we focus in particular on the agreement dimension of confidence. The IPCC seems to attribute an epistemic role to agreement, which in turn has implications for the meaning of confidence. What makes agreement especially interesting is its association with a consensus-based social epistemology of science. Though IPCC 2 E.g., Parker (2010); Lloyd (2010); Winsberg (2010, chap. 6); for a critical overview of different approaches, see Katzav (2014). 3 To be sure, observational evidence also depends on computer modeling in various ways; indeed, physicists have long employed simulations as part of their efforts to “observe” high-energy particles (Galison, 1997, chap. 8; Morrison, 2015; regarding climate science, see Edwards 2010 and Parker 2009), so the distinction between these sorts of evidence may be more a matter of degree than kind. 4 For a detailed analysis of the uncertainties in climate modeling that make a qualitative approach necessary, see Stainforth, Allen, Tredger, & Smith (2007); Katzav (2014) treats the IPCC notion of confidence briefly, again in relation to climate simulations. 5 In particular, there are questions regarding the IPCC understanding of robust evidence, but that is matter for another study. Note that the IPCC notion of robust evidence appears to differ from the use of that term in the philosophy of science literature (e.g. Wimsatt, 1981).

http://dx.doi.org/10.1016/j.shpsb.2016.10.008 1355-2198/& 2016 Elsevier Ltd. All rights reserved.

Please cite this article as: Rehg, W., & Staley, K. “Agreement” in the IPCC Confidence measure. Studies in History and Philosophy of Modern Physics (2016), http://dx.doi.org/10.1016/j.shpsb.2016.10.008i

2

W. Rehg, K. Staley / Studies in History and Philosophy of Modern Physics ∎ (∎∎∎∎) ∎∎∎–∎∎∎

statements about agreement are far from precise, and not wholly uniform, multiple statements suggest an epistemology of science in which the consensus of investigators plays a central role. We will call this the “consensus version” of the agreement measure: “agreement” refers to consensus in the relevant community of investigators on the IPCC finding at issue. The IPCC's apparent allusions to a consensus measure provide occasion for making philosophical analysis relevant for practicing scientists, or more accurately, for scientists on expert committees, of which the IPCC is the most ambitious to date. More precisely, we should like to know if the IPCC use of agreement as a partial basis for confidence can be supported by a plausible social epistemology. To answer that question, we must first understand exactly how the IPCC links its agreement attributions with its confidence estimates. As it turns out on closer inspection, however, the consensus version of agreement in AR5 squares neither with the entirety of IPCC explanations of agreement, nor with the actual practice of attributing degrees of agreement in the chapters that assess relevant literature. We find that the actual practice diverges from the consensus version in interesting ways, which do not match what we should expect to find in a consensus epistemology. Agreement attributions do not so much track the overall consensus among investigators as the degree to which relevant research findings substantively converge in offering support for IPCC claims. We start by by examining the explanations of agreement put forth by the statements on uncertainty in each working group report.6 We also take into account the 2010 IPCC “Guidance Note” (GN) on the treatment of uncertainty and an associated article by the authors of the GN (Section 2). After clarifying the structure of the “traceable accounts” with which AR5 authors support their confidence judgments (Section 3), we examine in detail the actual practice of agreement attributions in two examples, the first regarding past warming trends, the second concerned with phenology: though one might defend a consensus interpretation of the first case, the second case is clearly at odds with the consensus version of agreement (Sections 4, 5). We then approach the matter from the philosophical side, asking whether a consensus-based epistemology of science could capture the sense of IPCC agreement attributions. We find that neither Peirce's veritistic model of consensus, nor a more recent justification-based model, fits IPCC practice as represented by the traceable accounts we have considered (Section 6). Rather, agreement in actual IPCC practice has more to do with convergence of research than opinion. We conclude by formulating a general confidence principle that captures IPCC practice. That principle, we suggest, provides a starting point for posing further questions regarding the epistemic force of IPCC confidence attributions (Section 7).

2. IPCC statements about agreement In response to criticisms of the Fourth Assessment Report, AR5 authors took pains to employ a uniform method of treating uncertainty across the three working groups.7 Thus each of the three Working Group Reports in AR5 affirms the same confidence chart, whose outcomes depend on both evidence and agreement. And each working group, in its section on the treatment of uncertainty, at some point explicitly links agreement with expert consensus (although other understandings of agreement appear alongside these links). 6 IPCC assessments are composed of three main reports, each written by a different “working group” (IPCC, 2013, 2014a, 2014b, resp.; hereafter cited as AR5-I, AR5-II, and AR5-III): Working Group I (WGI) treats the physical science basis of climate change, WGII, its environmental and social impacts, and WGIII, mitigation. 7 For the criticisms, see the InterAcademy Council report (2010).

In the section of its report devoted to the treatment of uncertainty, Working Group I, without explicitly invoking the term ‘agreement,’ states that its aim is to provide “a consistent, calibrated set of words through which to communicate the uncertainty, confidence, and degree of consensus prevailing in the scientific literature” (AR5-I, 142). As part of its articulation of the rationale for its contribution to the Assessment Report, Working Group I states that the “IPCC process is aimed at assessing the literature as it stands and attempts to reflect the level of reasonable scientific consensus as well as disagreement” (AR5-I, 123, emphasis added). Both of these statements are compatible with multiple understandings of the term ‘agreement’ as employed in IPCC statements about the treatment of uncertainty, including a consensus version of agreement. The first quotation, however, referring to consensus “in the scientific literature,” suggests an alternative view of agreement in terms of concurrence amongst findings published in the relevant scientific literature. As we document below, this concurrence version of agreement is suggested also in an important Guidance Note regarding the treatment of uncertainties aimed at AR5 lead authors (Mastrandrea et al., 2010, Annex A, p. 1). The Working Group II Report explicitly contrasts agreement with evidential consistency: “Beyond consistency of evidence, the degree of agreement indicates the consensus within the scientific community on a topic and the degree to which established, competing, or speculative scientific explanations exist” (AR5-II, 177). This statement uses language similar to that in another paper written by authors of the AR5 Guidance Note (Mastrandrea et al., 2011). We will consider this construction more closely below. Working Group III again draws upon the Guidance Note (GN), but appears to regard it as employing a consensus version of agreement: “The GN recommends reporting the degree of certainty and/or uncertainty of a given topic as a measure of consensus or agreement across the scientific community. Confidence expresses the extent to which the IPCC authors do in fact support a key finding” (AR5-III, 157). The mixture of consensus and concurrence understandings that we find in the Working Group Reports reflect the heterogeneity in the Guidance Notes prepared for IPCC authors in 2005 and revised in 2010. The earlier document explains that the confidence measure “considers both the amount of evidence available in support of findings and the degree of consensus among experts on its interpretation” (IPCC, 2005, 3, emphasis added). The main body of the 2010 Guidance Note uses the term ‘agreement’ without explanation. However, the authors present in Annex A to that document a comparison between the approaches to uncertainty of AR4 and AR5. The AR5 Guidance Note authors interpret the AR4 notion of agreement as “the level of concurrence in the literature on a particular finding” (Mastrandrea et al., 2010, Annex A, 1, emphasis added), and treat this as the operative understanding of agreement for AR5. In different places, then, AR5 invokes both the notion of “concurrence in the literature on a particular finding” and “consensus among experts.”8 Finally, consider the treatment of agreement in a 2011 paper by the authors of the 2010 Guidance Note. This may be the most developed statement by IPCC authors regarding agreement, though it retains the mixture of distinct understandings found in our previous examples: The degree of agreement is a measure of the consensus across the scientific community on a given topic and not just across an author team. It indicates, for example, the degree to which a 8 Consider also this interpretation from AR5-I, in a discussion of the evaluation of climate models: “The degree of agreement measures whether different studies come to the same conclusions or not” (AR5-I, 822).

Please cite this article as: Rehg, W., & Staley, K. “Agreement” in the IPCC Confidence measure. Studies in History and Philosophy of Modern Physics (2016), http://dx.doi.org/10.1016/j.shpsb.2016.10.008i

W. Rehg, K. Staley / Studies in History and Philosophy of Modern Physics ∎ (∎∎∎∎) ∎∎∎–∎∎∎

finding follows from established, competing, or speculative scientific explanations. Agreement is not equivalent to consistency. Whether or not consistent evidence corresponds to a high degree of agreement is determined by other aspects of evidence such as its amount and quality; evidence can be consistent yet low in quality (Mastrandrea et al., 2011, 678). This passage begins with the idea that there is agreement regarding a finding to the extent that it is a matter of consensus in the scientific community. The next statement, however, treats agreement as indicating the “degree to which a finding follows from established, competing, or speculative scientific explanations.” This could be understood as providing a more precise notion of concurrence, understood as research results converging in providing support for an IPCC finding. Presumably, the idea here is that the support for a finding diminishes as the cited research goes from established, to competing, and finally to speculative.9 In sum, IPCC authors articulate the meaning of agreement in different ways, which are not apparently equivalent. Two interpretations stand out in our survey of statements: agreement as consensus in the scientific community and agreement as concurrence of published results. The direct juxtaposition of these two interpretations in the quotation above suggests that the authors either have not attended to the difference between them, or do not consider the difference important for attributing confidence in IPCC findings. But consensus and concurrence are not philosophically equivalent. Whereas the idea of consensus links confidence with a particular social epistemology, concurrence points to an epistemology of cogent substantive support for IPCC conclusions. Before we analyze these deeper philosophical implications, we must document the actual practice of agreement attributions in AR5. After clarifying the structure of traceable accounts in AR5 in the next section, we consider in some detail two examples of such accounts and consider what understandings of agreement are compatible with these examples. We will argue that both examples fit the concurrence interpretation but only one fits the consensus version.

3. IPCC argument-making: the structure of traceable accounts To understand the epistemic role of agreement in AR5, we must first be clear about the claims to which authors attach their confidence estimates and the basis for those estimates. The 2010 Guidance Notes on the Consistent Treatment of Uncertainty charge IPCC authors with providing “traceable accounts” in support of their confidence attributions (Mastrandrea et al., 2010, 2). We must therefore begin by laying out the structure of such accounts, which in effect function as arguments for IPCC conclusions. In fact, one can find two lines of argument in the IPCC. The first line starts with the Summary for Policymakers (SPM), which summarizes the central conclusions that are most relevant for political decision-making. These conclusions are arranged hierarchically, with broad statements opening each section, followed by more specific claims and sub-claims, and ultimately ending with references to the specific report chapters that set forth and evaluate the relevant research findings in the literature. Although formats differ somewhat across the different Working Group 9 The final sentence in the quoted passage seems to suggest that a finding with consistent evidence can become one with high agreement when that evidence is also of high quality and large in quantity. Given that the latter are requirements for “robust evidence” in the evidence dimension of the confidence assessment, it becomes unclear how the agreement dimension is supposed to differ from the evidence dimension of assessment.

3

Reports, the WGI Summary for Policymakers (hereafter SPM-1) illustrates the basic idea. SPM-1 is divided into a number of sections (A. Introduction; B. Observed Changes in the Climate System; C. Drivers of Climate Change; etc.), some of which are divided into subsections. Each section begins with a brief explanation of the scientific background, followed by a boxed summary statement of a general conclusion for the area of research. If the conclusion covers several areas of research, then subsections follow. For example, the section B conclusion covers observations of changes across the entire climate system (atmosphere, ocean, cryosphere, and so on). Because this conclusion is complex, subsections follow for each component. Each of these begins with its own boxed conclusion, which is followed by a series of more detailed bullet statements. Consider, for example, section B.1 in SPM-1, regarding atmospheric effects of climate change. That section starts with the following conclusion (SPM-1, 5): Each of the last three decades has been successively warmer at the Earth's surface than any preceding decade since 1850 (see Figure SPM.1). In the Northern Hemisphere, 1983–2012 was likely the warmest 30-year period of the last 1400 years (medium confidence) (2.4, 5.3).10 This general conclusion references only two chapter sections, 2.4 and 5.3. But the boxed conclusion is followed by seven bullet statements, two figures, and a table, all of which refer to WGI chapter sections, where one can find “the basis for substantive paragraphs” in the SPM (ibid., 4). One could trace out these references to determine what research supports each of the claims with its specific uncertainty estimate. However, for the more detailed traceable account one should follow the second line of argument, which starts in the Technical Summary (TS). The authors explain that the TS “serves as a starting point for those readers who seek the full information on more specific topics covered by this assessment” (AR5-I, 35). These statements suggest that we move from SPM statements to corresponding TS claims, which should then guide us to more detailed bases of support. Consider, for example, the second half of the B.1 claim above: “In the Northern Hemisphere, 1983 – 2012 was likely the warmest 30-year period of the last 1400 years (medium confidence) (2.4, 5.3).” This claim comes both with a confidence estimate and two references, though none of the B.1 bullet statements further explicate the claim. If we look to the TS, we find the following corresponding two-part statement about Northern Hemisphere warming, which we henceforth label NH: For average annual N[orthern] H[emisphere] temperatures, the period 1983–2012 was very likely the warmest 30-year period of the last 800 years (high confidence) and likely the warmest 30-year period of the last 1400 years (medium confidence) (AR5-I, TS.2.2.1, 37). NH expands on the B.1 claim in the SPM, distinguishing two claims with different levels of confidence. In addition, unlike B.1, this TS-statement is followed by an argument sketch of sorts: “This [NH] is supported by comparison of instrumental temperatures with multiple reconstructions from a variety of proxy data and statistical methods, and is consistent with AR4.” The passage goes on to discuss the Medieval Climate Anomaly (MCA), a period that skeptics have sometimes cited in criticisms of IPCC conclusions. The authors acknowledge that some regions were as warm in the MCA as today, but then note that “these regional warm periods 10 Emphasis in quotations from IPCC reports is in original unless otherwise specified.

Please cite this article as: Rehg, W., & Staley, K. “Agreement” in the IPCC Confidence measure. Studies in History and Philosophy of Modern Physics (2016), http://dx.doi.org/10.1016/j.shpsb.2016.10.008i

4

W. Rehg, K. Staley / Studies in History and Philosophy of Modern Physics ∎ (∎∎∎∎) ∎∎∎–∎∎∎

Fig. 1. A map of the traceable account for claim NH. Solid arrows point toward the sources of cited references. The dotted line connects two parts of AR5-I that make the same claim without explicitly citing one another.

were not as synchronous across regions as the warming since the mid-20th century.” The paragraph ends by citing two sections from the chapter on paleoclimate: 5.3.5 and 5.5.1. However, one must keep in the mind that the full argument also depends on evidence gathered instrumentally since the mid-nineteenth century. The evidence relative to that period is summarized in the paragraphs preceding the one quoted above. The two cited sections deal with “Temperature Variations During the Last 2000 Years” (5.3.5) and “Northern Hemisphere Mid to High Latitudes” (5.5.1.1; note that 5.5.1.2 deals with the southern hemisphere) (AR5-I, 409-15; 417-19). These sections provide detailed support for the NH-claim. The sections reference research in two ways: intralinear references to individual studies, and graphs and tables representing reconstructions from different research units (e.g., Fig. 5.7, Table 5.4; see AR5-I, 409, 410). We might map out this traceable account for NH as shown in Figure 1 below. In the figure, the use of a dotted line from SPM to TS indicates that B.1 makes no explicit link to the TS version of NH, though the same claim appears in both places. The solid arrows indicate that both SPM and TS make explicit references to AR5-I chapters that in turn cite findings in the literature in support of NH, though TS provides the more complete basis for NH. As the diagram indicates, the IPCC argument for NH bottoms out in specific studies, which are documented in Table 5.A.6 (AR5-I, 464). In such studies we find the evidence for the two more specific warming claims that compose NH (i.e., the 800-year claim and the 1400-year claim, or NH-800 and NH-1400).

4. The confidence estimates for Northern Hemisphere warming Let us now examine the more detailed basis for the confidence measures invoked in NH, namely “medium confidence” and “high confidence.” What interests us is the operative understanding of the agreement measure that informs these particular confidence estimates. In Section 5.3.5, we find the following explanation for these estimates (ellipses indicate omitted references): Based on multiple lines of evidence (using different statistical methods or different compilations of proxy records…), published reconstructions and their uncertainty estimates indicate, with high confidence, that the mean NH temperature of the last 30 or 50 years very likely exceeded any previous 30- or 50-year mean during the past 800 years (Table 5.4). The timing of warm and cold periods is mostly consistent across reconstructions (in

some cases this is because they use similar proxy compilations) but the magnitude of the changes is clearly sensitive to the statistical method and to the target domain (land or land and sea; the full hemisphere or only the extra-tropics…). Even accounting for these uncertainties, almost all reconstructions agree that each 30-year (50-year) period from 1200 to 1899 was very likely colder in the NH than the 1983–2012 (1963– 2012) instrumental temperature. NH reconstructions covering part or all of the first millennium suggest that some earlier 50year periods might have been as warm as the 1963–2012 mean instrumental temperature, but the higher temperature of the last 30 years appear [sic] to be at least likely the warmest 30year period in all reconstructions (Table 5.4). However, the confidence in this finding is lower prior to 1200, because the evidence is less reliable and there are fewer independent lines of evidence. There are fewer proxy records, thus yielding less independence among the reconstructions while making them more susceptible to errors in individual proxy records. The published uncertainty ranges do not include all sources of error …, and some proxy records and uncertainty estimates do not fully represent variables on time scales as short as the 30 years considered in Table 5.4. Considering these caveats, there is medium confidence that the last 30 years were likely the warmest 30-year period of the last 1400 years (AR5-I, 409–411). We quote this explanation at length because it illustrates a number of features. First, the authors employ both the quantitative and qualitative scales, estimating the confidence level for the likelihood of NH. Second, in the confidence estimate, the authors make both dimensions explicit. In the evidence dimension, they explicitly refer to the types of evidence (proxy and instrumental records, different statistical methods), the quantity (the relative paucity of proxy records prior to 1200), consistency (in timing of warm and cold periods), and independence (of reconstructions). In the agreement dimension, they cite a high level of agreement: “almost all reconstructions agree” on the probability estimate (very likely) for the second of the two claims in NH. In saying that reconstructions agree, the authors cite a near unanimous consensus on published conclusions. Third, they explain the two confidence estimates they attach to the two parts of NH—that is, they explain why their confidence drops prior to 1200. On this last point, it bears noting that although all but one of 16 reconstructions agree that recent Northern Hemisphere temperature has been very likely warmer than anything after 1000 (see Table 5.4, AR5-I, 410), the authors do not assign high confidence to that claim because of deficits in the evidence dimension before 1200. Thus the high likelihood estimate only holds for the last 800 years. Fourth, it is important to notice that in this example the primary target of the agreement assessment appears to be the published research results, particularly the reconstructions. This point is significant for interpreting the nature and epistemic role of the agreement measure in the IPCC reports. Although the research teams that produced the reconstructions are presumably in consensus regarding NH, it is not clear that the agreement attribution at issue here is primarily a claim about consensus in the sense of collective scientific judgment in some discipline or area of research, regarding a relevant IPCC claim. In fact, in their treatment of the credibility of models (WGI, chap. 9.8), IPCC authors interpret the agreement measure as follows: “The degree of agreement measures whether different studies come to the same conclusions or not” (AR5-I, 822). “Agreement” appears to refer more to the agreement of published outcomes rather than consensus among investigators. One might argue, however, that agreement of published outcomes and consensus among investigators are effectively equivalent: in publishing a result, researchers commit themselves to its

Please cite this article as: Rehg, W., & Staley, K. “Agreement” in the IPCC Confidence measure. Studies in History and Philosophy of Modern Physics (2016), http://dx.doi.org/10.1016/j.shpsb.2016.10.008i

W. Rehg, K. Staley / Studies in History and Philosophy of Modern Physics ∎ (∎∎∎∎) ∎∎∎–∎∎∎

public acceptability. Because there is a high level of agreement11 in published conclusions for the 800-year NH-claim, one might hold that this agreement of conclusions reveals a consensus in the community. To be sure, there is a question here regarding the relevant community. More on that in section 6.2. Though we do not claim to have scrutinized IPCC confidence attributions exhaustively—each report probably contains over a thousand such attributions—an informal sampling of AR5 gives the impression that something like an “agreement-in-conclusions” interpretation fits many of the IPCC claims. For example, in WGI this interpretation holds for sections that tend to report likelihood estimates without making degree of agreement explicit. Unless noted otherwise, such estimates have at least a high level of confidence, which suggests strong agreement across the literature. In many cases, one can identify such agreement (or its lack) by visually examining the close (or poor) alignment of published trends, reproduced in IPCC figures; in other cases, the IPCC makes a weaker claim that is compatible with the specific numerical results of published studies.12

5. Are scientists in a position to agree with IPCC conclusions? The phenology claim The research studies the IPCC cited to support the NH-claims do not all extend across the same time frame as that covered by the two claims. Four of the studies, for example, only reach back to 1600. This observation raises a more general worry about the consensus interpretation of agreement: in some cases the research teams might not be in a position to record their agreement with the IPCC claim at issue, for the simple reason that their research only tests a more limited claim. If we use published results as an indication of consensus, then we may not infer any consensus beyond the extent of the published conclusions. On that view, the four shorter studies are at most in partial agreement with the 800year NH-claim. While they do not present conflicting results (disagreement), their results also do not fully agree with the IPCC conclusion. Although the authors of those reconstructions are not in a position to make the 800-year NH-claim, IPCC authors seem to regard them as providing further support for that claim, when taken together with the other studies. This last observation is worth underlining. IPCC conclusions, particularly as presented in the SPMs, are often broad in scope, cutting across multiple lines of research. The above case illustrates this very point: NH is only part of the conclusion expressed in SPM-1, B.1, which in turn is only part of the overarching claim that opens section B. Indeed, even at the level of chapter sections, where IPCC authors advance more specific conclusions, the IPCC conclusion can lie at a higher level than the claims made by authors of particular articles cited in support of the IPCC claim. In this section we examine a chapter section in WGII where this 11 Though not perfect agreement: four of the reconstructions reported in Table 5.4 extend back only to the year 1600, and thus necessarily reach less ambitious conclusions than the other studies cited in support of NH. 12 See, for example, Figure 2.6, which supports the claim that a recent decline in stratospheric ozone is “certain” (AR5-I, 172, 171, resp.). By contrast, IPCC authors place low confidence in claims about trends in stratospheric water; this is evident from discrepancies displayed in Fig. 2.5 (ibid., 171). Regarding confidence in sealevel rise, authors cite “improved physical understanding” and “inclusion of icesheet dynamical changes,” but also the “improved agreement of process-based models with observations” (AR5-I, 1139, our emphasis). In their treatment of sealevel rise during the Last Interglacial Period, the authors report “robust evidence and high agreement” on the claim that the sea level was higher than today; more specifically, they cite two high-quality studies, each using different methods, which reached “consistent” results. This gives the authors very high confidence in a maximum rise of 45 m compared to today's sea level. The two studies agree on this particular conclusion, in the sense that their numerical results both fall above the IPCC's 5-meter estimate (ibid., 1146).

5

heterogeneity is especially noticeable. In its treatment of phenology (timing of seasonal species activities), the IPCC makes the claim “that hundreds of (but not all) species of plants and animals have changed functioning to some degree over the last decades to centuries on all continents (high confidence due to robust evidence but only medium agreement across all species)” (AR5-II, 989). This is a sweeping, global claim—call it P. But some of the studies cited in support of P are limited in extent, and thus not in a position to make the claim themselves. The IPCC authors support their confidence claim by first citing three studies. Menzel et al. (2006) is a metastudy of 542 plant and 19 animal species in 21 European countries (from 1971–2000). Thus their study does not give them the basis for making a claim about all continents. Cook et al. (2012) examines two phenological databases for plant species in the Northern Hemisphere: their study extends neither to all continents, nor to animal species. Peñuelas et al. (2013) appears to come closest of the three to the IPCC claim, as it summarizes a massive literature on phenological and other species (plant and animal) responses to warming. The IPCC account then moves on to a host of studies that confirm the results reached in the three cited studies; we note that some of these further studies are limited to single species. Together they provide what the IPCC calls “robust evidence” from a range of perspectives, using different methodologies, that support high confidence in the phenological claim P. In fact, the IPCC authors’ use of the agreement measure is puzzling in this case. “Agreement across species” is medium because not all species exhibit clear phenological responses to climate. But that sort of lack of agreement does not mean that authors themselves disagree over phenology. Indeed, the claim P itself is formulated to take non-responsive species into account. Strictly speaking, no study disagrees with P. Unlike the NH example, the phenology example does not allow us to infer from the published results that the community of investigators is in consensus on P, for the simple reason that many of the studies reach less ambitious conclusions than P. To be sure, we might infer from this observation that community consensus explicitly on P, as judged from published results, is only medium. But there is no indication that the IPCC authors intend that reading. We might also dismiss this example as exceptional, a case in which the chapter authors failed to follow the operative consensus understanding of agreement, which has nothing to do with “agreement across species.” But the more important feature of the phenology example lies in the way IPCC authors marshal a range of studies, differing in scope, to support a confidence estimate for P. The authors of the Guidance Note clearly intended such a procedure to be included in the toolbox of IPCC authors evaluating the uncertainty of IPCC claims. They describe how an author team “may consider constructing a key finding that extends beyond the spatial or temporal scale of the available evidence,” noting how doing so will “affect the degree of certainty” of the finding thus constructed and emphasizing the importance of communicating the assumptions made in such a construction (Mastrandrea et al., 2011). Accordingly, this kind of traceable account of the building of a broader scope claim from more limited scope studies is hardly unique in AR5.13 The two examples we have just presented show a discrepancy, greater in the second case than the first, between the consensus version of agreement and the actual practice of attributing agreement. Although these two cases are very limited, they are unlikely to be idiosyncratic (see notes 12 and 13 for further 13 The nature of many IPCC claims regarding global phenomena is such that support comes from a range of studies of diverse scope. For example, the IPCC cites a wide range of mainly regional studies to support its claim of medium confidence in the absolute value of glacial mass loss at both the regional and global levels (AR5-I, 340-41).

Please cite this article as: Rehg, W., & Staley, K. “Agreement” in the IPCC Confidence measure. Studies in History and Philosophy of Modern Physics (2016), http://dx.doi.org/10.1016/j.shpsb.2016.10.008i

6

W. Rehg, K. Staley / Studies in History and Philosophy of Modern Physics ∎ (∎∎∎∎) ∎∎∎–∎∎∎

examples). A thorough investigation of IPCC practice would be a monumental undertaking. Such an effort would involve two timeconsuming steps: first, to search the IPCC working group reports for instances of the word “agreement,” then for each instance, to examine the literature that is cited in support of the agreement attribution. In examining the literature, one would have to investigate the scope and content of the published claims, as we have done for the two cases here presented. In the next section we approach the question from the philosophical side. More specifically, we clarify some epistemic requirements imposed by consensus epistemologies. If the IPCC authors intend their agreement attributions to measure consensus as an epistemic consideration, then those attributions should attend to the epistemic conditions that make consensus relevant as a partial ground for confidence.

6. Consensus as an epistemic measure: philosophical approaches In policy-making contexts, to attribute high confidence to a claim suggests we are ready to accept the claim as a reliable basis for decision-making.14 Although AR5 (in contrast with AR4) eschews any quantitative interpretation of its confidence scale, attributions of confidence in a policymaking context imply some connection between confidence in a claim and whether we may reasonably accept the claim as a basis of action. Thus, in making the level of agreement a co-determinant of confidence, IPCC authors apparently understand expert agreement as an epistemic measure. If we understand consensus as a stand-in for collective belief (or acceptance15), and if we assume that knowledge involves justified true belief, then we can distinguish two ways of conceiving consensus as an epistemic measure. Veritistic conceptions link consensus with truth, whereas justificatory conceptions link it with justification.16 Neither, we argue, adequately captures IPCC practice. (6.1) Peirce provides the obvious starting point.17 His consensus theory of truth provides the locus classicus on the relation between inquiry, consensus, and scientific knowledge. But Peirce's model has clear shortcomings for understanding IPCC agreement attributions. First, Peirce was primarily concerned with how we ought to understand the concept of truth. He invoked an ideal of final consensus in order to provide a pragmatic definition of truth, not as a usable qualitative measure of confidence. Second, even if we do invert the logic of the truth-consensus relationship, the veritistic consensus that matters for Peirce is the consensus that would obtain, were the science community to pursue its inquiry on the matter as far as it could fruitfully go (Misak, 1991). Thus, if we are to take consensus on h as a basis for inferring the qualitatively probable truth of h, then we must first have grounds for regarding the inquiry on h as final—we must be warranted in thinking that inquiry on the matter has gone as far as it fruitfully 14 It should be noted that lower-confidence claims may also be relevant when assessing overall risk. Here we only intend to note the practical significance of attributing high confidence to a finding. 15 In working out a social epistemology of science, the idea of belief as a mental state has a number of disadvantages, beginning with the fact that scientific practice does not require scientists to believe their assumptions and results, but rather to accept them for practical purposes of further research and collaboration; see van Fraassen (1980, chap. 2). 16 To be sure, the consensus ideal has its critics (e.g., Solomon, 2001). We cannot address those criticisms here, except to point out that affirming that ideal does not mean one dismisses the epistemic value of ongoing dissent (cf. Beatty and Moore, 2010). 17 Wright (1992) offers a more recent example of a veritistic conception of consensus; at one point, Habermas also fell into this line of thought, but he has since renounced his earlier view (compare Habermas, 1986, 2003).

could go. Do IPCC conclusions enjoy such a warrant? Hardly. The understanding of global and regional climate systems remains relatively young. Though climate scientists regard some claims as “near certain” (e.g., that anthropogenic forcing are driving recent climate change), research is ongoing on a number of fronts, and crucial questions remain unresolved.18 Indeed, AR5 authors are quite explicit about areas of perduring uncertainty regarding the physical basis of climate change (AR5-I, 114-15). (6.2) A number of philosophers have attempted to conceive epistemic consensus in relation to the quality of scientific inquiry and discourse as a process of public justification. In these models, a consensus on h warrants our defeasibly taking h as true insofar as the consensus has resulted from a sufficiently rigorous process of inquiry and discourse. Although some analysts articulate the conditions of rigorous discourse in ideal terms, others provide conditions that are closer to the ground, so to speak.19 Whether idealized or not, these accounts tend to specify substantive and procedural conditions on epistemically reliable consensus. Substantive standards include shared logical and discipline-specific standards of good evidential arguments, whereas procedural standards specify the social-institutional conditions that ensure inclusion of viewpoints, equal voice, lack of coercion, and the like. Boaz Miller's (2013) analysis of epistemic consensus can serve as a succinct example of these recent models. His model exhibits the basic kinds of features found in well-known approaches (Longino, Habermas), but with the advantage of explicitly focusing on non-ideal consensus. For Miller, the central question is this: when may we legitimately regard scientific consensus as a mark of knowledge? In reply, he proposes a set of conditions that allow one to distinguish genuinely “knowledge-based” scientific consensus from deceptive forms of consensus.20 In general, a consensus on h expresses knowledge insofar as (a) there is wide—but not necessarily universal—acceptance of h in the relevant science community,21 (b) h is partially or approximately true, and (c) inquiry into h satisfies three conditions: “social calibration”— scientists share basic substantive assumptions, formalisms, and evidential standards (exemplars); apparent consilience of different sources of evidence exists; and the parties to the consensus exhibit social diversity. Notice that the last set of conditions (c) in effect functions as a public justification condition on scientific knowledge. Having identified a consensus, his model has us scrutinize precisely that condition. Some kind of consensus exists insofar as (a) is met: members of a discipline or research community generally accept a claim h. What we want to know is whether or not that consensus gives us grounds for taking h as true—that is, for thinking that the truth condition (b) on knowledge is met. The third element (c) allows us to resolve that doubt by setting forth conditions on the substance and social organization of inquiry that make a faux or deceptive consensus unlikely—a consensus that lacks real content, is merely fortuitous, false, or is based on non-cognitive reasons (e.g., political pressure). The social calibration condition 18 The attribution of certainty to the claim of declining ozone might be an exception; see AR5-I, 171–72. 19 For an idealized model of justificatory consensus, see Habermas (2003); Rehg (2009, chaps. 4–6) elaborates its implication for scientific inquiry. Though Longino (1990, 78, 197–200) compares her model of social objectivity to Habermas's idealizations, at least some of her standards (shared venues of discussion, shared standards) appear to be concretely realizable. Other standards, such as tempered equality of intellectual authority, are rightly understood as idealizations. 20 We pass over some details here; Miller frames this question as an inference to best explanation: does knowledge explain the consensus better than alternative explanations? If so, then the consensus is probably knowledge-based. 21 In line with work by Brad Wray and Margaret Gilbert, Miller does not require belief in h but rather acceptance (willingness to let h stand); see Wray (2001); Gilbert (1994).

Please cite this article as: Rehg, W., & Staley, K. “Agreement” in the IPCC Confidence measure. Studies in History and Philosophy of Modern Physics (2016), http://dx.doi.org/10.1016/j.shpsb.2016.10.008i

W. Rehg, K. Staley / Studies in History and Philosophy of Modern Physics ∎ (∎∎∎∎) ∎∎∎–∎∎∎

excludes a sham consensus that exists merely because key terms remain vague or loosely applied. Thus, this condition ensures that a consensus has real content. Apparent consilience of evidence and social diversity, Miller argues, together make the consensus robust, in Stegenga's sense that “there is plenty of evidence generated by multiple techniques that rely on different background assumptions” (Stegenga, 2009, 650; quoted in Miller, 2013, 1309). When these conditions are satisfied, the best explanation of the consensus is that it is genuinely knowledge-based. We do not need to delve into the details of Miller's account, except to notice three broad features. First, Miller understands an epistemically reliable consensus as one that meets conditions on adequate public justification. To determine whether a consensus rests on adequate justification, one must rule out phenomena that can undermine adequacy. Second, the kind of phenomena that make a consensus unreliable include social conditions, namely a low level of social diversity in the research community. Here Miller follows feminist critiques of male dominance in the sciences. A third feature makes Miller's model especially appropriate for application to non-idealized cases: he understands his conditions on adequate justification as “a matter of degree” (1294).22 This last feature fits with IPCC practice, which understands confidence as a scalar measure that comes in degrees (low, medium, high, very high). But do the other features of his model capture IPCC practice? If IPCC authors were to estimate confidence using Miller's conditions, they would have to make the following determinations:

 The degree to which a body of researchers accepts the relevant hypothesis h.

 The level of shared formalisms, ontological schemes, and model solutions or evidential standards,

 Degree of apparent consilience of different lines of research,  Level of social diversity among the relevant research community. To summarize these factors, we might formulate a confidence measure based on Miller's approach: h merits our confidence insofar as a socially diverse group of researchers, working within similar frameworks of fundamental assumptions, accepts h on the basis of different kinds of evidence that are apparently consilient. Do IPCC confidence attributions meet these conditions? Or do IPCC authors at least show concern for meeting these conditions when explaining their confidence attributions? If they do, then we might plausibly interpret their idea of agreement as at least roughly aligning with a philosophical conception of the consensus of investigators. The consilience condition proves to be the closest fit: IPCC authors often display an interest in the convergence of different kinds of evidence—indeed, the kind and independence of evidence are marks of “robust evidence,” according to the 2010 GN (Mastrandrea et al., 2010, 3): “Generally, evidence is most robust when there are multiple, consistent independent lines of highquality evidence.”23 The social calibration requirement presumably also holds, making allowance for the interdisciplinary nature of IPCC claims. The central IPCC claims are certainly not vague—the authors seek to find agreement with real content. What about social diversity? IPCC administrators are indeed concerned with diversity of membership within the IPCC, that is, representation from around the world. But they also show concern for the “geographic distribution of authors contributing to the climate change literature.” Though serious imbalances still exist, 22 Analogues to the first two features are found in idealized models like Habermas's; Longino's model of social objectivity also exhibits the third feature. 23 However, this mention of independence appears neither in the Technical Summary treatment of uncertainty nor in the main section on uncertainty in the Working Group I report (AR5-I, 36, 139–142).

7

the distribution of authors has expanded somewhat beyond Europe and North America, with an increasing number of non-English publications (AR5-II, 172; 173–74). However, it remains unclear how the IPCC authors understand the diversity at issue here. Much of their social analysis appears to be more concerned with the insufficiently global coverage of research (with research on southern hemisphere regions still sparse) than with social diversity among researchers. But social diversity differs from research coverage of all regions. The latter condition could be met if more North American and European scientists were to turn their attention on less-studied regions. But the social diversity condition is met only if the research community includes scientists who represent the full range of gender and ethnic backgrounds, empowered to challenge any monocultural bias that might slant an entire area of study. Moreover, the social diversity condition concerns not merely membership but aspects of the social process of science that affect the quality of its results. The IPCC has devoted considerable attention to the dialogical quality and social diversity of its own process over the years.24 But we do not detect in AR5 serious concern about the social structures that might affect the quality of climate change research in general.25 What about the first condition, the level of acceptance? One might be tempted to identify this condition with the IPCC agreement dimension. In that case, IPCC agreement attributions articulate the extent to which researchers agree in accepting the claim at issue. As mentioned in Section 4, that interpretation is not entirely incompatible with many IPCC attributions, such as their confidence in the NH-claims. At the least, it seems we can infer from the agreement in model results that (nearly all) the different research teams accept the 800-year warming claim, namely that in the Northern Hemisphere, “the period 1983–2012 was very likely the warmest 30-year period of the last 800 years.” However, two problems arise for the consensus interpretation of the NH-claim above and claims like it. First, as pointed out earlier, it is not clear that the IPCC is primarily concerned with the researchers’ judgments of acceptable claims, in contrast to the agreement of results themselves. Should IPCC authors want to know the level of community acceptance, then polling the scientists in the relevant community might be more efficient, and in any case would appear to be the most important kind of evidence relevant to attributing community consensus.26 But to our knowledge, AR5 does not report such a survey.27 Second, the consensus at issue in Miller's model is presumably a consensus among all members of the relevant science community. But that community is likely to extend beyond the members of research teams whose publications are cited in AR5. If IPCC authors are to infer a level of community acceptance of a claim from a set of publications whose conclusions agree, then IPCC authors must assume either that the teams involved in the cited research effectively exhaust the relevant community of scientists competent to judge the evidence for the claim, or that the acceptance of those teams in effect represents the best opinion of the relevant community. Both assumptions, however, are potentially controversial.28 One might object that the IPCC relies on background 24 See the procedures link on the IPCC website, www.ipcc.ch/organization/or ganization_procedures.shtml; for discussion of process, see Hulme et al. (2010). 25 At most, IPCC authors note how the paucity of research in some regions is negatively affected by certain institutional constraints, e.g., on funding (AR5-II, 172). 26 It is not clear a poll would be more efficient, since a poll assumes one knows which scientists comprise the relevant community for a particular claim, which in turn presupposes familiarity with the literature. 27 By contrast, in other forums of debate, the question of consensus in the climate community has been important; see Oreskes (2004); Doran and Zimmerman (2009). 28 See, for example, Pielke's (2005) criticism of Oreskes (2004).

Please cite this article as: Rehg, W., & Staley, K. “Agreement” in the IPCC Confidence measure. Studies in History and Philosophy of Modern Physics (2016), http://dx.doi.org/10.1016/j.shpsb.2016.10.008i

8

W. Rehg, K. Staley / Studies in History and Philosophy of Modern Physics ∎ (∎∎∎∎) ∎∎∎–∎∎∎

knowledge of the level of consensus in the relevant community warranted by the authors’ familiarity with both the research area and the extensive process of IPCC report reviews. Such an argument could appeal to AR5-I's discussion of the assessment procedure. In discussing the peer-reviewed literature, the authors point out: “Not all papers on a controversial point can be discussed individually in an assessment, but every effort has been made here to ensure that all views represented in the peer-reviewed literature are considered in the assessment process” (AR5-I, 123). This might support something like the two assumptions that we have just called into question in the previous paragraph. Granting the objector's point that IPCC authors know more than they can put explicitly into their traceable accounts, however, the question at hand concerns not what IPCC authors know that is relevant to their findings, but what reasons inform the assessment of the uncertainty of those findings. Those assessments are to be found in the traceable accounts. The AR5 Guidance Note articulates clearly this role for traceable accounts, noting that in choosing how to characterize the uncertainty of a finding “in all cases it is important to include a traceable account of relevant evidence and agreement” (Mastrandrea et al., 2010, 3). Even if the authors’ background knowledge includes details that cannot be included in the traceable account, then, we should be able to rely on those accounts to tell us what kind of information the authors regard as relevant to the evaluation of agreement. Even if one accepts the force of this objection in defense of the consensus version of agreement, that version cannot be reconciled with the phenology example previously discussed. For claims like that in the phenology case, which are based on diverse kinds of evidence and studies of differing scope, IPCC authors may not infer a level of community acceptance of IPCC claims from relevant published conclusions, simply because a significant number of those conclusions are not equivalent to the IPCC claim. Miller's conception of acceptance would at most capture only part of the IPCC practice, and then only on the debatable assumptions about the community of investigators stated above. Let us take stock before moving on. Philosophical models of justificatory consensus assume that (1) the consensus at issue arises as a collective acceptance by the research community of a particular hypothesis or claim, and (2) the epistemic quality of the consensus depends on the social organization and conduct of inquiry in the community.29 Many IPCC agreement attributions do not readily fit this model as applied to the entire community of scientists contributing research that is relevant to IPCC claims. Rather, a consensus epistemology better captures the IPCC itself as a community of investigators. As to (1), what matters for the IPCC is not so much the consensus of all researchers on a topic, but that of AR5 authors and reviewers. On the one hand, the researchers whose publications are cited in support of IPCC claims are often not in a position to join a consensus directly on the IPCC claim, simply because their individual contributions are too limited, or deal with only one part of the IPCC claim. On the other hand, as noted earlier, IPCC authors are charged with assessing, and not merely reporting on, the literature. They are thus expected to judge its quality—what conclusions does the best research point to? As to (2), the social process that matters for IPCC conclusions is that of the IPCC itself. Report drafts are subjected to the review and acceptance not only of scientists, but eventually of government representatives as well. We may thus regard IPCC reports as consensus statements that issue from a carefully organized social 29 We do not claim that Miller, Longino, and Habermas agree on details. In fact, it is not clear that consensus plays a big role in Longino's model, though her criteria for social objectivity presumably may be used to test the objectivity of scientific consensus. What matters is that each model regards the social organization of science as an important determinant of knowledge.

process designed to foster a collective judgment that is as reasonable and rigorous as feasible. The IPCC thus attempts to constitute itself as a socially diverse community of investigators that draws the relevant evidential arguments from the literature (in contrast to their own experiments) and judges their import for taking specific claims as publicly justified, and thus as warranting our confidence in their truth.30 In light of the foregoing, we maintain that IPCC practice as a whole may be more adequately characterized by the concurrence version of agreement suggested in the 2011 paper by the GN authors discussed in Section 2. That discussion included a treatment of agreement as an indication of “the degree to which a finding” is supported by results that are well-established in the literature (Mastrandrea et al., 2011, 678). In keeping with this interpretation, we suggest that agreement regarding an IPCC claim be evaluated by considering not the state of opinion of scientists themselves, but the extent to which studies relevant to that claim have produced evidence that, in amount and quality, lend strong support to the claim. In the next section, we take up this suggestion and attempt to articulate a principle we might take to underwrite confidence attributions in AR5.

7. Conclusion: a proposal and further questions In this essay we have focused on the agreement dimension of the IPCC's qualitative confidence estimate. Philosophical models of consensus, we argue, face problems in capturing the range of IPCC practices in attributing confidence to specific claims. To close our reflection, we propose an alternative conception of confidence that lies closer to actual IPCC practice and that can thus serve as a focal point for further analysis. Specifically, we suggest that the following formulation of confidence articulates the confidence attributions in AR5: (ConfIPCC) Insofar as investigators produce multiple, consistent independent lines of high-quality evidence (in the collective judgment of IPCC authors) that agree in providing support for h, we may legitimately have confidence in defeasibly taking h as true. Here h refers to the particular IPCC conclusion or claim. The formulation highlights the potentially indirect nature of agreement—evidential results agree in providing support for h, which need not mean that each cited researcher explicitly accepts h. We suggest that (ConfIPCC) can serve as a touchstone for further reflection on the epistemic force of IPCC confidence attributions. It gives us a kind of confidence principle that we believe captures IPCC practice—its practical epistemology, if you will—albeit without importing any particular philosophical rationale for that principle.31 In proposing (ConfIPCC), we do not commit ourselves to any particular epistemology of science. Rather, we claim that the IPCC is in effect employing such a principle when it makes estimates of degree of agreement. As the principle states, the relevant agreement is, in the first place, an agreement of conclusions in providing some measure of support for an IPCC claim h. To be sure, 30 On IPCC process, see www.ipcc.ch/organization/organization_procedures. shtml), for a case study, see Lahsen (1999). 31 Should ConfIPCC be formulated as a biconditional? Is the combination of high agreement and robust evidence necessary for confidence? Compared to IPCC practice, a biconditional appears overly demanding, given that some kinds of claim might warrant high confidence without the evidence satisfying all the conditions the IPCC associates with robust evidence. Moreover, IPCC attributions of confidence appear to be contextually sensitive, judging from the different ways in which such attributions are supported. But necessary-condition formulations tend to work against such sensitivity, inasmuch as they impose the same set of conditions on every context. A sufficient-condition formulation, by contrast, does not rule out attributions of confidence based on less demanding conditions.

Please cite this article as: Rehg, W., & Staley, K. “Agreement” in the IPCC Confidence measure. Studies in History and Philosophy of Modern Physics (2016), http://dx.doi.org/10.1016/j.shpsb.2016.10.008i

W. Rehg, K. Staley / Studies in History and Philosophy of Modern Physics ∎ (∎∎∎∎) ∎∎∎–∎∎∎

(ConfIPCC) does not exclude consensus among investigators, and to that extent does not exclude a community-consensus inference for some IPCC claims, on certain assumptions about the relevant community of investigators (see sec. 6.2). In any case, the key idea in (ConfIPCC) is not consensus but rather “support.” The immediate question that (ConfIPCC) raises, then, is this: what does it mean for different studies to “agree in providing support for” a claim? The two examples we presented point to a pluralist conception of support in IPCC traceable accounts. In some cases, a set of studies can support an IPCC claim in the sense that each of the studies reaches the same conclusion as the IPCC claim itself. Each study, as it were, directly confirms the IPCC claim. In other cases, studies of different scope might each address some part of the IPCC claim, reaching conclusions from which the IPCC claim is a generalization or extrapolation. In that case, each study functions like a partial experimental test that the IPCC hypothesis passes. A pluralist approach to the IPCC confidence measure acknowledges the complexity and diversity of scientific practices, and is not committed to a one-size-fits-all theory of climate science evidence. For example, the limited extant commentary on qualitative confidence in climate models regards the confirmation interpretation as over-reaching. Rather, model simulations at best describe what are “real possibilities” (Katzav, 2014, 235–237; cf. Betz, 2010). While an approach centered on possibilities might well provide a superior account of model simulations, it has two limitations as an overall theory of confidence. First, it applies primarily to climate simulations; thus its implications for other sorts of IPCC claims remain unclear. Second, claims about real possibilities will probably prove ineffective in contentious political contexts, when policymakers must decide in the face of uncertainty. Compare the IPCC confidence claims with legal verdicts, for example. In criminal law we do not convict someone on the basis of a “real possibility” of his or her wrongdoing, but on the basis of strong evidence that warrants high confidence in the attribution of guilt. Even the much weaker preponderance-of-evidence standard in civil law sets a higher bar than “real possibility.” In conclusion, both IPCC scientists and philosophers have work to do. On the one hand, IPCC authors should work to align their descriptions of agreement with the actual practice of attributing agreement. In AR5, the descriptions do not adequately articulate the operative notion of agreement, which is more aptly described by Mastrandrea et al. (2011) as a substantive convergence in the literature that lends support to IPCC findings. Philosophers of science have an object for epistemological analysis. (ConfIPCC) might appear intuitively reasonable, but to what extent and how different theories of evidence and evidential support underwrite this general principle calls for deeper analysis. A complete analysis must delve into the IPCC's understanding of traceable accounts and robust evidence, and clarify their relation to philosophical notions of consilience, robustness, confirmation, explanatory power, and related ideas. Presumably traceable accounts represent informal inductive arguments that move from the literature to general conclusions. Explaining the precise nature of such inductions is an important task for scientists and philosophers.

Acknowledgements The authors are grateful to Ben de Foy and two anonymous referees for helpful critical comments on earlier drafts of this paper.

9

References Beatty, J., & Moore, A. (2010). Should we aim for consensus? Episteme, 7(3), 198–214. Betz, Gregor (2010). What's the worst case? The methodology of possibilist prediction. Analyse und Kritik, 32(1), 87–106. Cook, B., Wolkovich, E., Davies, T. J., Ault, T., Betancourt, J., Allen, J., & Travers, K. (2012). Sensitivity of spring phenology to warming across temporal and spatial climate gradients in two independent databases. Ecosystems, 15(8), 1283–1294. Doran, Peter T., & Zimmerman, Maggie Kendall (2009). Examining the scientific consensus on climate change. EOS, 90(3), 22–23. Edwards, Paul N. (2010). A vast machine. Cambridge: MIT Press. Galison, Peter (1997). Image and logic. Chicago: University of Chicago Press. Gilbert, Margaret (1994). Remarks on collective belief In: F. F. Schmitt (Ed.), Socializing epistemology (pp. 235–256). Lanham, MD: Rowman Littlefield. Habermas, Jürgen (1986). Wahrheitstheorien. Vorstudien und Ergänzungen zur Theorie des kommunikativen Handelns ((2nd ed.). Frankfurt am Main: Suhrkamp. Habermas, Jürgen (2003). Truth and justification. Trans. B. Fultner. Cambridge: MIT Press. Hulme, M., Zorita, E., Stocker, Th. F., Price, J., & Christy, J. R. (2010). IPCC: Cherish it, tweak it or scrap it? Nature, 463(11), 730–732. InterAcademy Council 2010. Climate change assessments: Review of the processes and procedures of the IPCC. 〈http://reviewipcc.interacademycouncil.net/report.html〉 (accessed 4.10.15). IPCC 2005. Guidance notes for lead authors of the IPCC fourth assessment report on addressing uncertainties. July 2005. At 〈https://www.ipcc.ch/pdf/supporting-material/ uncertainty-guidance-note_ar4.pdf〉, (accessed 08.10.15). IPCC (2013). T. F. Stocker, D. Qin, G.-K. Plattner, M. Tignor, S. K. Allen, J. Boschung, A. Nauels, Y. Xia, V. Bex & P.M. Midgley (Eds.), Climate change 2013: The physical basis. Cambridge: Cambridge University Press. 〈https://www.ipcc.ch/report/ar5/wg1/〉. IPCC (2014a). C. B. Field, V. R. Barros, D. J. Dokken, K. J. Mach, M. D. Mastrandrea, T. E. Bilir, M. Chatterjee, K. L. Ebi, Y. O. Estrada, R. C. Genova, B. Girma, E. S. Kissel, A. N. Levy, S. MacCracken, P. R. Mastrandrea, & L. L. White. (Eds.), Climate change 2014a: Impacts, adaptation, and vulnerability. Part A: Global and sectoral aspects. Cambridge: Cambridge University Press. 〈https://www.ipcc.ch/report/ar5/wg2/〉. IPCC (2014b). O. Edenhofer, R. Pichs-Madruga, Y. Sokona, E. Farahani, S. Kadner, K. Seyboth, A. Adler, I. Baum, S. Brunner, P. Eickemeier, B. Kriemann, J. Savolainen, S. Schlömer, C. von Stechow, T. Zwickel & J. C. Minx (Eds.), Climate change 2014b: Mitigation of climate change. Cambridge: Cambridge University Press. 〈https://www. ipcc.ch/report/ar5/wg3/〉. Katzav, Joel (2014). The epistemology of climate models and some of its implications for climate science and the philosophy of science. Studies in the History and Philosophy of Modern Physics, 46, 228–238. Lahsen, Myanna (1999). The detection and attribution of conspiracies: The controversy over chapter 8 In: G. E. Marcus (Ed.), Paranoia within reason (pp. 111–136). Chicago: University of Chicago Press. Lloyd, Elizabeth A. (2010). Confirmation and robustness of climate models. Philosophy of Science, 77, 971–984. Longino, Helen E. (1990). Science as social knowledge. Princeton: Princeton University Press. Mastrandrea M.D., Field C.B., Stocker T.F., Edenhofer O., Ebi K.L., Frame D.J., … Zwiers, F.W. (2010). Guidance note for lead authors of the IPCC fifth assessment report on consistent treatment of uncertainties. Intergovernmental Panel on Climate Change (IPCC). Available at 〈http://www.ipcc.ch〉. Mastrandrea, M. D., Mach, K. J., Plattner, G.-K., Edenhofer, O., Stocker, T. F., Field, C. B., & Matschoss, K. L. (2011). The IPCC AR5 guidance note on consistent treatment of uncertainties: A common approach across the working groups. Climatic Science, 108, 675–691. Menzel A., Sparks T.H., Estrella N., Koch E., Aasa A., Ahas, R. ... Zust, K. (2006). European phenological response to climate change matches the warming pattern Global Change Biology, 12 (10), 1969-1976. Miller, Boaz (2013). When is consensus knowledge-based? Distinguishing shared knowledge from mere agreement. Synthese, 190, 1293–1316. Misak, Cheryl J. (1991). Truth and the end of inquiry. Oxford: Clarendon. Morrison, Margaret (2015). Reconstructing reality: Models, mathematics, and simulations. New York: Oxford University Press. Oreskes, Naomi (2004). The scientific consensus on climate change. Science, 306(5702), 1686. Parker, Wendy S. (2009). Does matter really matter? Computer simulations, experiments, and materiality. Synthese, 169, 483–496. Parker, Wendy S. (2010). Whose probabilities? Predicting climate change with ensembles of models. Philosophy of Science, 77, 985–997. Peñuelas J., Sardans J., Estiarte M., Ogaya R., Carnicer J., Coll, M. ... Garbulsky, A. (2013). Evidence of current impact of climate change on life: A walk from genes to the biosphere Global Change Biology 19 (8), 2303-2388. Pielke, Roger A., Jr. (2005). Consensus about climate change? Science, 308(5724), 952–954. Rehg, William (2009). Cogent science in context. Cambridge: MIT Press. Solomon, Miriam (2001). Social empiricism. Cambridge: MIT Press. Stainforth, D. A., Allen, M. R., Tredger, E. R., & Smith, L. A. (2007). Confidence, uncertainty, and decision-support relevance in climate predictions. Philosophical Transactions of the Royal Society A, 365, 2145–2161. Stegenga, Jacob (2009). Robustness, discordance, and relevance. Philosophy of Science, 76 (5), 650–661. Van Fraassen, Bas C. (1980). The scientific image. Oxford: Oxford University Press. Wimsatt, William (1981). Robustness, reliability, and overdetermination In: Marilynn B. Brewer, & Barry E. Collins (Eds.), Scientific inquiry and the social sciences (pp. 124– 163). San Francisco: Jossey-Bass. Winsberg, Eric B. (2010). Science in the age of computer simulation. Chicago: University of Chicago Press. Wray, Brad (2001). Collective belief and acceptance. Synthese, 129(3), 319–333. Wright, Crispin (1992). Truth and objectivity. Cambridge: Harvard University Press.

Please cite this article as: Rehg, W., & Staley, K. “Agreement” in the IPCC Confidence measure. Studies in History and Philosophy of Modern Physics (2016), http://dx.doi.org/10.1016/j.shpsb.2016.10.008i