National Academies Press: OpenBook
« Previous: 1 Introduction
Suggested Citation:"2 Major Overarching Comments." National Research Council. 2007. Review of the U.S. Climate Change Science Program's Synthesis and Assessment Product 5.2, "Best Practice Approaches for Characterizing, Communicating, and Incorporating Scientific Uncertainty in Climate Decision Making". Washington, DC: The National Academies Press. doi: 10.17226/11873.
×

2
Major Overarching Comments

Characterizing scientific uncertainty about climate change, effectively communicating that uncertainty to decision makers, and incorporating it in the decision making process are important tasks. The climate research community recognizes and understands the importance of characterizing uncertainty in research and assessment efforts, and decision-makers can benefit greatly from improved communication of uncertainty by the research and assessment communities. Thus the Climate Change Science Program’s (CCSP) Synthesis and Assessment Product (SAP) 5.2 will potentially be very beneficial to all stakeholders of climate change science. The committee commends CCSP and the National Oceanic and Atmospheric Administration (NOAA) for emphasizing the need to address this important topic.


This chapter provides an enumerated discussion of the major issues that, from the point of view of the review committee, the authors should strongly consider addressing in the revised version of SAP 5.2. In some cases, findings are simply noted without explicit recommendations. In other cases, the committee provides either a direct recommendation or alternatives for the authors to consider as they address the review findings. In subsequent chapters of this report, the committee provides further overarching thoughts on the draft document and then findings and recommendations specific to individual chapters of the draft. The major overarching comments follow.

  1. The draft provides a good treatment of cognitive challenges and expert elicitation issues. The committee finds that the draft SAP 5.2 provides a well-written and concise synthesis of some of the key issues regarding the characterization of scientific uncertainty vis-à-vis climate decision making. The draft provides a particularly good synthesis of the issues regarding cognitive challenges on an individual basis to characterizing uncertainty, the potential implications of those challenges for CCSP and other assessment efforts, and a method of characterizing uncertainty when conventional methods are not practical or adequate (expert elicitation).

  2. All of the audiences outlined in the prospectus are not addressed. The committee finds that the draft is written largely for an audience of those people involved in assessment efforts. Indeed, that is one key audiences for SAP 5.2. The intended audiences as outlined in the prospectus also include those people engaged in scientific research, the media, policymakers, and members of the public. The committee suggests that the authors might also acknowledge and discuss the unique needs of the private sector, which are quite different from those of researchers, academia, and the assessments

Suggested Citation:"2 Major Overarching Comments." National Research Council. 2007. Review of the U.S. Climate Change Science Program's Synthesis and Assessment Product 5.2, "Best Practice Approaches for Characterizing, Communicating, and Incorporating Scientific Uncertainty in Climate Decision Making". Washington, DC: The National Academies Press. doi: 10.17226/11873.
×

community. The private sector and policy and decision-makers in the public sector (e.g., congressional staff) fall into the category of users of assessments and need to understand the implications of uncertainty, in contrast to the research science community, who are generators of assessments and associated uncertainty information. The draft provides relatively little information for an audience of users, particularly information that could be used as guidelines for effective communication techniques.

  1. The range of best practices for characterizing uncertainty is not represented. The document fails to review the range of methods used to characterize uncertainty as called for in the study prospectus. Instead, it focuses almost exclusively on expert elicitation for use in a subjective Bayesian analysis. This focus neglects assessments, including other Synthesis and Assessment Products, associated with the observational record. There is a need to discuss more traditional frequentist methods, which remain dominant in scientific work, and objective Bayesian methods based on non-informative prior distributions. By focusing exclusively on the subjective Bayesian approach, the document also fails to elucidate ‘Best Practices’ for characterizing uncertainty as called for in the study prospectus. The committee understands this elucidation to involve a description of alternative approaches and a discussion of the strengths and weaknesses of each. The addition of a statistician to assist with the elucidation of traditional scientific methods would address a significant weakness in the report. This addition to the authorship team should be strongly considered, regardless of whether the current document (and its authorship team) is greatly expanded to address the additional audiences and issues described in the prospectus.

  2. The influence of social context and emotional factors is absent. Although some of the important cognitive factors in understanding and evaluating uncertainty are discussed in Chapter 3, this discussion is incomplete in two senses. First, the draft SAP 5.2 neglects the social context in which such understandings and evaluations are made; even within the narrow focus of discussing expert elicitation, responses will be influenced by how the questions are asked, the context of the interview, the expectations and knowledge of experts about what their peers are saying, and the cultural set of norms that attend the social groups (scientific institutions, universities and departments, etc.) to which respondents belong. Moreover, emotions have been shown to play scientifically measurable roles in estimations of uncertainty. Second, the discussion of cognitive biases, and the missing discussion of the social context and emotive factors in evaluating uncertainty are highly relevant to the communications and decision making chapters but are almost entirely absent in these too-brief chapters. In the communications chapter, the emphasis is on presentation of materials to an amorphous non-technical audience rather than on understanding the needs of multiple audiences within their social contexts, as specified in the prospectus. The decision making chapter neglects the large literatures on decision making in institutional settings where scientific uncertainty is only one of many factors influencing decisions.

  3. Introductory material is lacking. The draft would be improved if the brief paragraph at the top of page 1 was expanded into a formal introduction section that provides framing and context for the rest of the document. The authors could define who climate decision-makers are and discuss the importance of characterizing,

Suggested Citation:"2 Major Overarching Comments." National Research Council. 2007. Review of the U.S. Climate Change Science Program's Synthesis and Assessment Product 5.2, "Best Practice Approaches for Characterizing, Communicating, and Incorporating Scientific Uncertainty in Climate Decision Making". Washington, DC: The National Academies Press. doi: 10.17226/11873.
×

communicating, and incorporating scientific uncertainty for the decision making process. This could be done using an example of decision making under uncertainty, where the optimal decision depends on the characterization of uncertainty, and also an example of scientific inference, where uncertainty needs to be characterized to go beyond point estimation (i.e., to test an hypothesis or provide a confidence interval)." In its current form, the transition to technical material is far too abrupt. The introduction section could also outline the charge to the authors as they perceived it, and clearly define the goals and objectives of the document. As an alternative, this material could be included in a foreword. The foreword or introduction could also state explicitly what the document does not address vis-à-vis the broader contexts of scientific uncertainty and climate decision making.

  1. An executive summary is essential but has not been prepared. The committee finds that the lack of an executive summary hinders its accessibility to the audiences named in the prospectus. A concise and readable summary of the document, including key findings and recommendations, would enable all audiences -- producers of synthesis and assessment products, scientific researchers, decision-makers, media, and the general public -- to glean the main points and to locate further information that may be of interest to them. The summary should not be descriptive, but informative on the main points of the document.

Suggested Citation:"2 Major Overarching Comments." National Research Council. 2007. Review of the U.S. Climate Change Science Program's Synthesis and Assessment Product 5.2, "Best Practice Approaches for Characterizing, Communicating, and Incorporating Scientific Uncertainty in Climate Decision Making". Washington, DC: The National Academies Press. doi: 10.17226/11873.
×

This page intentionally left blank.

Suggested Citation:"2 Major Overarching Comments." National Research Council. 2007. Review of the U.S. Climate Change Science Program's Synthesis and Assessment Product 5.2, "Best Practice Approaches for Characterizing, Communicating, and Incorporating Scientific Uncertainty in Climate Decision Making". Washington, DC: The National Academies Press. doi: 10.17226/11873.
×
Page 7
Suggested Citation:"2 Major Overarching Comments." National Research Council. 2007. Review of the U.S. Climate Change Science Program's Synthesis and Assessment Product 5.2, "Best Practice Approaches for Characterizing, Communicating, and Incorporating Scientific Uncertainty in Climate Decision Making". Washington, DC: The National Academies Press. doi: 10.17226/11873.
×
Page 8
Suggested Citation:"2 Major Overarching Comments." National Research Council. 2007. Review of the U.S. Climate Change Science Program's Synthesis and Assessment Product 5.2, "Best Practice Approaches for Characterizing, Communicating, and Incorporating Scientific Uncertainty in Climate Decision Making". Washington, DC: The National Academies Press. doi: 10.17226/11873.
×
Page 9
Suggested Citation:"2 Major Overarching Comments." National Research Council. 2007. Review of the U.S. Climate Change Science Program's Synthesis and Assessment Product 5.2, "Best Practice Approaches for Characterizing, Communicating, and Incorporating Scientific Uncertainty in Climate Decision Making". Washington, DC: The National Academies Press. doi: 10.17226/11873.
×
Page 10
Next: 3 Additional Overarching Comments »
Review of the U.S. Climate Change Science Program's Synthesis and Assessment Product 5.2, "Best Practice Approaches for Characterizing, Communicating, and Incorporating Scientific Uncertainty in Climate Decision Making" Get This Book
×
Buy Paperback | $29.00 Buy Ebook | $23.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!