Evaluating Progress of the U.S. Climate Change Science Program: Methods and Preliminary Results (2007)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

1 Introduction T he U.S. government has sponsored a substantial coordinated re- search program on global climate and related environmental change for more than 15 years, initially under the U.S. Global Change Research Program (USGCRP) and currently under the Climate Change Sci- ence Program (CCSP). Research carried out under these programs has led to numerous scientific advances, but evidence of progress is largely anec- dotal. A formal evaluation of program performance would (1) demonstrate that investments in research and applications are generating returns, and (2) identify where adjustments should be made to improve results. The CCSP has been considering ways to evaluate progress since 2003. However, assessing progress has proved challenging for a program which comprises activities ranging from research, observations, and modeling on the atmospheric, ocean, and land systems; to human contributions and responses to climate change; to tools to support decision making (e.g., scenarios of possible impacts of climate change on North America). A National Research Council (NRC) report Thinking Strategically: The Ap- propriate Use of Metrics for the Climate Change Science Program (NRC, 2005) provided a framework that would enable CCSP managers to make strategic decisions about the entire program, and CCSP asked this commit- tee for guidance in building from it. The Committee on Strategic Advice on the Climate Change Science Program was established to carry out three tasks for the CCSP. The first task—to develop a process for evaluating progress and to make a prelimi- nary assessment of CCSP progress (Box 1.1)—is the subject of this report.  

 EVALUATING PROGRESS OF THE U.S. CCSP BOX 1.1  Committee Charge for Task 1 The committee will assist the CCSP in evaluating progress toward program goals. The CCSP Strategic Plan and the guidelines given in the 2005 NRC report Think- ing Strategically: The Appropriate Use of Metrics for the Climate Change Science Program will provide a starting point for this examination. This report will address two subtasks: 1a. Findings and recommendations on the process for evaluating progress toward the five goals in the CCSP strategic plan. The recommendations should be practical and consider the tradeoffs between strategic utility and program costs associated with implementing metrics. 1b. A preliminary assessment of progress made toward the program’s goals. The results will serve as an interim report for a more comprehensive analy- sis of the program’s progress to be completed in subsequent years. The committee began by examining evaluation approaches tried by CCSP managers as well as the comprehensive framework laid out in Think- ing Strategically (NRC, 2005). Based on this analysis, it developed an evalu- ation approach that would both identify strengths and weaknesses of the entire program and be practical to implement. The committee’s approach was used to carry out the preliminary evaluation of CCSP progress (task 1b). Input for the evaluation was gathered from CCSP reports, the scientific literature, briefings and responses to questionnaires from CCSP managers, and a community workshop. Although the assessment identified gaps and weaknesses in the program, no recommendations were made on ways to address them. Identifying future priorities for the CCSP is the subject of the committee’s second task. CLIMATE CHANGE SCIENCE PROGRAM The CCSP integrates the USGCRP and the Climate Change Research Initiative (CCRI). The USGCRP, the first federally coordinated program supporting climate change research, began as a presidential initiative in 1988 and received congressional support in 1990 under the Global Change Research Act. The act called for the development of a research program “to understand, assess, predict, and respond to human-induced and natu- ral processes of global change,” and it guided federally supported global   Public Law 101-606(11/16/90) 104 Stat. 3096-3104. 

INTRODUCTION  CCSP USGCRP + CCRI Overarching Overarching Overarching Overarching Overarching Goal 1 Goal 2 Goal 3 Goal 4 Goal 5 Near Term Priorities [CCRI] Research Elements [USGCRP] Principal Products Uncertainties Uncertainties Atmospheric composition 21 Synthesis & Assessment Aerosols Climate variability and change Products (associated with the Feedbacks (especially polar) Water cycle overarching goals) Carbon sources/sinks (North Land-use and land-cover change America) Carbon cycle 200+ deliverables (associated Ecosystems with the research elements) Modeling Human contributions and Data sets and physical Cause and impacts of climate responses quantities change: Policy aid Process studies Cross-Cutting Issues Models Observing Systems Assessments of uncertainty and Modeling predictability Observations and monitoring Synthesis and assessment Data management Risk assessment and Decision support resources management Communications Decision support International cooperation FIGURE 1.1  CCSP road map. figure 1-1 change research for the next decade. In 2001, President Bush launched the CCRI to investigate uncertainties and set new research priorities in climate change science. The CCRI also gave priority to research that could yield results within a few years, either by improving decision-making capabilities or by contributing to improved public understanding. The two programs were merged the following year, and a cabinet-level management structure was introduced to improve coordination between the CCSP, a parallel program to promote the development of new technologies for monitoring or eliminating greenhouse gas emissions (the Climate Change Technology Program), and the Office of the President. The CCSP is divided into three major components: overarching goals, research elements, and cross-cutting issues (Figure 1.1). The CCSP over- arching goals differ from previous USGCRP goals by placing a greater em- phasis on uncertainties and the use of research results in decision making. The five overarching CCSP goals follow:   See <http://www.climatescience.gov/>. 

10 EVALUATING PROGRESS OF THE U.S. CCSP 1. Improve knowledge of the Earth’s past and present climate and environment, including its natural variability, and improve understanding of the causes of observed variability and change. 2. Improve quantification of the forces bringing about changes in the Earth’s climate and related systems. 3. Reduce uncertainty in projections of how the Earth’s climate and related systems may change in the future. 4. Understand the sensitivity and adaptability of different natural and managed ecosystems and human systems to climate and related global changes. 5. Explore the uses and identify the limits of evolving knowledge to manage risks and opportunities related to climate variability and change (CCSP, 2003). Some of the research elements have also changed since the inception of the CCSP. In particular, paleoclimate research under the USGCRP was apparently folded into the CCSP climate variability and change research element, and a new research element on land use and land cover change was added. Finally, the CCSP has identified six issues (e.g., modeling, observa- tions) that cut across the research elements (Figure 1.1). The USGCRP supported long-term research results, but the CCSP also committed to create hundreds of products within two to four years (Table 1.1), beginning in July 2003. This ambitious plan is not likely to be met. For example, only two of the 21 planned synthesis and assessment products have been published and only three others are in the final draft stage (Ap- pendix A). Apparently, unforeseen delays were caused by implementation of the Federal Advisory Committee Act and new guidelines for peer review of federal government information. Thirteen agencies participate in the CCSP, which has an annual budget of about $1.7 billion (CCSP, 2006a). Coordination within the CCSP takes place at several levels (Figure 1.2). The program has a director (currently an acting director) appointed by the Department of Commerce. Strategic planning for the program as a whole is overseen by principals from each participating agency and liaisons from the Executive Office and related programs. Planning within the major components of the program is done through 11 interagency working groups, some of which have external sci- ence committees. A program office with less than 9 full-time equivalent staff members provides management and coordination support.   Presentation to the committee by James Mahoney, CCSP director, on April 27, 2006. See Public Law 106-554, (Information Quality Act) and Public Law 92-463 and amendments (Federal Advisory Committee Act).   See <http://www.climatescience.gov/about/staff/default.htm>. 

TABLE 1.1  Budgets, Products, and Advisory Structure for CCSP Research Elements Climate Land Use and Atmospheric Variability Carbon Water Land Cover Human Composition and Change Cycle Cycle Ecosystems Change Dimensions Fiscal Year 2006 Budget (million dollars)a Research $170.1 $286.9 $113.2 $137.9 $115.8 $32.5 $25–30b NASA satellites $62.9 $136.9 $89.4 $119.4 $49.0 $42.7 $0.0 Products < 2 years 0 3 3 5 1 13 3 2-4 years 11 37 20 19 8 12 12 ≥ 4 years 5 11 23 14 7 17 4 Coordination or Advisory Structure Interagency 9 agencies 6 agencies and 9 agencies and 9 agencies 9 agencies 8 agencies 8 agencies; also working group U.S. CLIVAR U.S. Carbon covers decision Project Office; Cycle Program support resources also covers Office modeling Science No Use NRC Yes, active Being No Yes, met in Use NRC committee committee reconstituted October 2005 committee (CRC) and May 2007 (CHDGC) NOTES: CHDGC = Committee on the Human Dimensions of Global Change; CLIVAR = Climate Variability and Predictability; CRC = Climate Research Committee; NASA = National Aeronautics and Space Administration aBudgets from CCSP (2006a). Research budget includes National Oceanic and Atmospheric Administration satellite costs, but not NASA satellite costs or CCRI activities. bThe budget for the CCSP human contributions and responses research element is $147.6 million, including $62.8 million for NASA programs on decision support and $57.2 million for National Institutes of Health programs on the health effects of ultraviolet radiation. Funding devoted to human dimensions research is probably $25 million to $30 million (see Appendix B). 11 

12 EVALUATING PROGRESS OF THE U.S. CCSP Climate Change Science Program Principals Research Element Cross-Cutting Issue IWGs IWGs Atmospheric Human contributions Climate variability composition / Decision support / Modeling Carbon cycle Communications Ecosystems International Land use and Observations and land cover Water cycle Monitoring FIGURE 1.2  Interagency coordination structure for the Climate Change Science Program. Interagency working groups (IWGs) shaded in gray have access to a science committee. Two of the IWGs (human contributions and responses/deci- sion support resources and climate variability and change/modeling) cover both a fig 1-2 research element and a cross-cutting issue. There is also an IWG for financial opera- tions. The CCSP Office supports coordination at all levels of the program. ORGANIZATION OF THE REPORT This report describes a method for evaluating progress of the CCSP and provides a preliminary assessment of progress over the last four years. The report is divided into two parts. Part I (Chapters 2 and 3) recommends an evaluation method and provides overarching conclusions from the prelimi- nary assessment of CCSP progress. The strengths and limitations of different evaluation approaches are analyzed in Chapter 2. The evaluation of CCSP progress was structured around a matrix developed by the committee (Ap- pendix C). Chapter 3 describes how the cells of the matrix were scored and presents conclusions on where the most and least progress has been made 

INTRODUCTION 13 in the program. Detailed supporting analysis for these conclusions appears in Part II (Chapters 4 and 5) for reference. Chapter 4 evaluates progress in the seven research elements (e.g., carbon cycle, ecosystems) and part of an overarching goal, and identifies challenges and opportunities to maintain- ing or speeding progress in the future. Progress in the cross-cutting issues (e.g., modeling, communications) is assessed in Chapter 5. Input for the evaluation was gathered from CCSP reports and presentations, the scientific literature, responses to a questionnaire on funding and programs under the human contributions and responses research element (summarized in Ap- pendix B), and a committee-organized workshop (see Appendix D).