1
Introduction

The number of global environmental change assessments undertaken as a result of international treaties and at the request of national, state, and local governments is steadily on the rise. The amount of resources and the number of scientists dedicated to such assessment activities are increasing as well. At the same time, a wealth of experience on how to conduct assessments has become available. Given the expected increasing need for assessment activities in the future, it is important to evaluate the approach and effectiveness of past assessments and learn from the available lessons.

Many assessments in the past few decades have focused on concerns about natural and human-induced climate change, with the most recent U.S. assessment activity being conducted by the U.S. Climate Change Science Program (CCSP). The CCSP is responsible for implementing the Global Change Research Act (GCRA) of 1990, which mandates periodic assessments of our understanding of global change and its impact on the nation. Before undertaking its next round of assessments, the CCSP wanted to draw from the collective experience of the past few decades of global change assessments. Therefore, CCSP asked the National Academies to look at lessons learned from past global change assessments and to provide the program with guidance on its approach to the next assessment activity. In response, an ad hoc committee was formed by the National Academies to conduct a comparative analysis of past global change assessments (Table 1.1) with goals similar to those of the CCSP. The purpose is to identify strengths and weaknesses in these processes and provide advice for future assessments conducted in the United States. (See Box 1.1 for the Full Statement of Task).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 13
Analysis of Global Change Assessments Lessons Learned 1 Introduction The number of global environmental change assessments undertaken as a result of international treaties and at the request of national, state, and local governments is steadily on the rise. The amount of resources and the number of scientists dedicated to such assessment activities are increasing as well. At the same time, a wealth of experience on how to conduct assessments has become available. Given the expected increasing need for assessment activities in the future, it is important to evaluate the approach and effectiveness of past assessments and learn from the available lessons. Many assessments in the past few decades have focused on concerns about natural and human-induced climate change, with the most recent U.S. assessment activity being conducted by the U.S. Climate Change Science Program (CCSP). The CCSP is responsible for implementing the Global Change Research Act (GCRA) of 1990, which mandates periodic assessments of our understanding of global change and its impact on the nation. Before undertaking its next round of assessments, the CCSP wanted to draw from the collective experience of the past few decades of global change assessments. Therefore, CCSP asked the National Academies to look at lessons learned from past global change assessments and to provide the program with guidance on its approach to the next assessment activity. In response, an ad hoc committee was formed by the National Academies to conduct a comparative analysis of past global change assessments (Table 1.1) with goals similar to those of the CCSP. The purpose is to identify strengths and weaknesses in these processes and provide advice for future assessments conducted in the United States. (See Box 1.1 for the Full Statement of Task).

OCR for page 13
Analysis of Global Change Assessments Lessons Learned TABLE 1.1 The Eight Examples of Assessment Processes Included in the Comparative Analysis Assessment Brief Description Stratospheric Ozone Assessments Prior to the 1987 Montreal Protocol, several national (including NRC) and international assessments analyzed ozone-depleting chemicals and the current and projected state of the stratospheric ozone layer (WMO 1982, 1986a). Following the treaty, a system of expert advisory panels was established to periodically assess the atmospheric science of the ozone layer (WMO 1990a, 1990b, 1992, 1995, 1999, 2003, 2007), the impacts of ozone loss (UNEP 1991a, 1994a, 1998a, 2002a), and the technology and economics of alternatives to ozone-depleting chemicals (UNEP 1991b, 1994b, 1998b, 2002b). Intergovernmental Panel on Climate Change (IPCC) IPCC analyzes scientific and socioeconomic information on climate change and its impacts, and assesses options for mitigation and adaptation. On request, it provides scientific, technological, and socioeconomic findings to the Conference of the Parties to the United Nations Framework Convention on Climate Change (IPCC 1990a,b,c, 1995a,b,c, 2001a,b,c). Global Biodiversity Assessment (GBA) GBA provides a synthesis and analysis of available science to support the work of the UN Convention on Biological Diversity (GBA 1995). National Assessment of Climate Change Impacts (NACCI) NACCI was undertaken in response to the Global Change Research Act (1990) to evaluate the impacts of climate change on the United States (NAST 2001). Arctic Climate Impact Assessment (ACIA) Primary objectives were to evaluate and synthesize knowledge and indicators of climate variability, climate change, and ultraviolet radiation in the region; to assess possible impacts of future changes in climate and radiation; and to provide reliable information to both governments and peoples of the region to support policy-making processes (ACIA 2004). Millennium Ecosystem Assessment (MA) MA was designed to answer questions fundamental to various UN conventions dealing with natural resource issues, in particular the consequences of diverse environmental changes on the functioning of ecosystems, including their continuing capacity to deliver services essential to human well-being (MA 2005a,b).

OCR for page 13
Analysis of Global Change Assessments Lessons Learned Assessment Brief Description German Enquete Kommission on “Preventive Measures to Protect the Earth’s Atmosphere” The Enquete Kommission brings scientists and policy makers together to assess, in this case, the importance and consequences of stratospheric ozone depletion and climate change for Germany among other dimensions of global environmental change (Enquete Kommission 1988, 1991). Synthesis and Assessment Products by the U.S. Climate Change Science Program (CCSP) The 21 current assessment products were designed to address the mandate of the Global Change Research Act, by considering science and policy issues spanning the range of topics addressed by the CCSP. The first product, on temperature trends in the lower atmosphere, was released in April 2006 (CCSP 2006). HISTORY OF CLIMATE CHANGE ASSESSMENTS AND POLICY IN THE UNITED STATES Early History In the United States, the first concerns about the ramifications of continued greenhouse gases can be traced to the late 1950s. In 1957, Roger Revelle, Scripps Institute of Oceanography, and Hans Suess, University of Chicago, suggested that the burning of fossil fuels would lead to significant increases in atmospheric carbon dioxide (CO2) concentrations and that humans were engaged in a “large-scale geophysical experiment” with long-term consequences (Revelle and Suess 1957). Charles David Keeling, Scripps Institute of Oceanography, began taking regular measurement of CO2 that same year at the Mauna Loa Observatory as part of the International Geo-physical Year in 1957-1958. Keeling’s early data showed a cyclical nature to annual Northern Hemisphere atmospheric composition that corresponds to the terrestrial “respiration” of the planet, as well as an overall upward trend superimposed on top of that cycle (Keeling et al. 1976, 1982). In 1965, a report of the President’s Science Advisory Committee (PSAC 1965) made clear that there was a sound basis for linking human activities to the increasing CO2 concentration and that this would lead to global warming (Table 1.2). The panel recommended augmented research efforts and attention to this environmental issue. By the early 1970s, a number of international groups, including several that advised the United Nations, had come to the same conclusion. As recommended, U.S. research efforts were intensified during the 1970s. At the same time, several additional activities concluded that humans were indeed in the process of altering the Earth’s

OCR for page 13
Analysis of Global Change Assessments Lessons Learned BOX 1.1 Statement of Task This committee was asked to identify lessons learned from past assessments to guide future global change assessment activities of the U.S. Climate Change Science Program (CCSP). The study had two steps. (1) The committee conducted a comparative analysis of past assessments that have stated objectives similar to those of the CCSP. Specifically, the committee examined the strengths and weaknesses of selected past assessments in the following areas: Establishing clear rationales and appropriate institutional structures; Designing and scheduling assessment activities; Involving the scientific community and other relevant experts in the preparation and review of assessment products; Engaging the potential users of assessment products; Accurately and effectively communicating scientific knowledge, uncertainty, and confidence limits; Guiding plans for future global change research activities, including observation, monitoring, and modeling of past and future changes; and Creating assessment products that are valued by their target audiences. (2) The committee identified approaches (in terms of geographic scale, scope, assessment entity, and timing) and products that are most effective for meeting the CCSP’s stated objectives for assessments. climate and made some climate projections into the twenty-first century that in many ways still hold true today (SMIC 1971, NRC 1977, 1979). As a consequence of the many reports on the topic, Congress established the National Climate Program in 1978, as part of the National Oceanic and Atmospheric Administration, and charged the program to conduct climate impact assessments and study policy options for reducing human-induced climate change. This program was reviewed in a report of the National Research Council (NRC) and judged to be producing significant scientific achievements but falling somewhat short in terms of policy options. The NRC report Toward an Understanding of Global Change: Initial Priorities for U.S. Contributions to the International Geosphere–Biosphere Program (NRC 1988) called for expansion of the program to include disciplines that could study the socioeconomic impacts of various policy options. The 1985 Villach report (WMO 1986b) by an international climate change conference renewed the interest of policy makers internationally

OCR for page 13
Analysis of Global Change Assessments Lessons Learned TABLE 1.2 Selected Activities during the 1970s and Early 1980s Highlighting the Concern that Humans Are Influencing Global Climate and Environment by Increasing CO2 Emissions Year Activities 1965 U.S. President’s Science Advisory Committee (PSAC) points to sound basis for link between human activities and increasing CO2 emissions (PSAC 1965). 1970 Study of Critical Environmental Problems (SCEP) highlights man’s impact on the global environment (SCEP 1970). 1971 Study of Man’s Impact on Climate (SMIC) points out the inadvertent climate modification due to human activity (SMIC 1971). 1977 National Resource Council (NRC) report Energy and Climate points to potential negative impact of heavy coal use on climate (NRC 1977). 1979 NRC report Carbon Dioxide and Climate: A Scientific Assessment concludes that climate sensitivity to a doubling of CO2 is 1.5 to 4.5°C (NRC 1979). 1979 American Association for the Advancement of Science workshop in Annapolis on climate impacts, sponsored by DOE, led to congressional hearings in 1981 with a call for the Administration to fund research on impacts. 1983 NRC report Carbon Dioxide and Climate: A Second Assessment confirms the first report’s finding regarding climate sensitivity and predicts the doubling to occur during the last quarter of the century. It calls for research into energy sources other than fossil fuel and suggests that if addressed now, climate change from greenhouse gases would be manageable (NRC 1983). 1985 During an international meteorologist conference jointly sponsored by the United Nations Environment Programme (UNEP), World Meteorological Organization (WMO), and International Council of Scientific Unions in Villach, Austria, a consensus is announced that, in the first half of the twenty-first century, a rise in global mean temperature could occur that is greater than any in man’s history (WMO 1986b). and nationally and was followed by several congressional hearings on this issue. The Reagan White House responded to these hearings by forming a White House Domestic Policy Council working group on climate change. Although the House of Representatives held hearings throughout the 1980s on the topic of rising CO2, the Senate did not turn to this issue until after the Villach report (WMO 1986b). After numerous hearings, Congress enacted the Global Climate Protection Act of 1987 (P.L. 100-204), which authorized the U.S. Environmental Protection Agency and the State Department to

OCR for page 13
Analysis of Global Change Assessments Lessons Learned develop climate change policy. However, both the Reagan and the G.H.W. Bush administrations limited the impact of the Global Climate Protection Act by retaining control over climate change policies in the executive office (GAO 1990). The efforts of various White House committees to coordinate climate science and provide the executive branch with policy options did not satisfy the desire of Congress to see progress on these issues. U.S. Global Change Research Program and Global Change Research Act Of 1990 In 1989, the U.S. Global Change Research Program (GCRP) was developed by the Committee on Earth Sciences, an interagency group under the Federal Coordinating Council for Science, Engineering, and Technology in the President’s Office of Science and Technology Policy. The GCRP was a Presidential Initiative indicating that it was a high-priority program with strong administrative backing. In 1990, Congress passed the Global Change Research Act (GCRA) (P.L. 101-606), which codified the GCRP (see Appendix B). According to this law, the GCRP is aimed “at understanding and responding to global change, including the cumulative effects of human activities and natural processes on the environment, to promote discussions toward international protocols in global change research, and for other purposes.” The GCRP was originally envisioned as a complete global change research program, covering research on natural climate change, human-induced climate change, impacts of climate and land-use change on the Earth system, and impacts of human activity on ecosystem health. Much of the research effort that was assembled into the GCRP originated with the National Aeronautics and Space Administration’s “Mission to Planet Earth” that formulated an interdisciplinary program to study the Earth as a total system in the early 1980s (CRS 1990). The GCRP’s priorities were established with input from the scientific community under the guidance of the Committee on Earth Science (see Figure 1.1) and published in “Our Changing Planet: A U.S. Strategy for Global Change Research” (GCRP 1989). Several criteria, although not applied systematically, were used to evaluate projects under each research element, including relevance and contribution to the overall goal of the program, scientific merit, ease or readiness of implementation, links to other agencies and international partners, cost, and agency approval. Initially, the early GCRP mission included the following: Documentation and analysis of Earth system changes, which include observation—using both ground- and space-based observation systems—and data management; and

OCR for page 13
Analysis of Global Change Assessments Lessons Learned FIGURE 1.1 The goal, objectives, and scientific framework for the U.S. Global Change Research Program (GCRP) with a ranking of science priorities. SOURCE: GCRP 1989. Process research to enhance understanding of the physical, geological, chemical, biological, and social processes that influence Earth system behavior; and integrated modeling and prediction of Earth system processes. In FY 1994, a new program element was added: assessment activities. This addition came in response to Congressional interest as specified in the GCRA and advanced many of the findings of the Congressional Office of Technology Assessment (OTA) report (OTA 1993) Preparing for an Uncer-

OCR for page 13
Analysis of Global Change Assessments Lessons Learned tain Climate that recommended more effort be put into assessment, analysis of climate impacts, contingency planning, and adaptation. Specifically, the GCRA calls for preparing and submitting to the President and Congress an assessment that: Integrates, evaluates, and interprets the findings of the GCRA; Analyzes the effects of global change on the natural environment, agriculture, energy production and use, land and water resources, transportation, human health and welfare, human social systems, and biological diversity; and Analyzes current trends in global change, both human-induced and natural, and projects major trends for the subsequent 25 to 100 years. Between 1994 and 1997, the GCRP conducted detailed annual reviews of both its core programs and areas in need of additional research, such as assessment and impacts. Each annual Our Changing Planet report highlighted the programmatic foci of the next year and the successes of the previous years. In 1997, the administration charged the GCRP to initiate the first comprehensive National Assessment. This effort, the National Assessment of Climate Change Impacts, was completed in the fall of 2000, after 20 regional workshops, 5 sectoral analyses, 2000 participants, and 3 layers of review (for additional details, see Chapter 4). Over the years, the GCRP benefited from ongoing external oversight provided by several boards, committees, and panels of the NRC. The NRC was responsible for evaluating the GCRP periodically for scientific merit and issued more than 30 reports that advised the GCRP on global change research (e.g., NRC 1988, 1989). The Genesis of the Climate Change Science Program On June 11, 2001, President G.W. Bush announced that his administration would “establish the U.S. Climate Change Research Initiative to study areas of uncertainty [about global climate change science] and identify priority areas where investments can make a difference.” The Secretary of Commerce, working with other agencies, was directed to “set priorities for additional investments in climate change research, review such investments, and to improve coordination amongst Federal agencies.” To respond to the President’s initiative and meet the requirements of the GCRA of 1990, the CCSP was initiated in 2002. Thirteen federal departments and agencies that fund or carry out global change research participate in the program and serve on the CCSP Interagency Committee, which is chaired by the CCSP Director, who is the Assistant Secretary for Oceans and Atmosphere. An Interagency Working Group appointed by the Interagency

OCR for page 13
Analysis of Global Change Assessments Lessons Learned Committee and consisting of specialists from the participating departments and agencies is responsible for implementing CCSP activities. A 10-year strategic plan to coordinate climate and global change research activities across federal agencies was developed (CCSP 2003) with input and review from an NRC committee (NRC 2003, 2004). The strategic plan identified five overarching goals (see Box 1.2), a wide range of research areas, and 21 synthesis and assessment activities to address these goals (Appendix C). The Climate Change Research Initiative (CCRI) represents a more focused program than the GCRP, dedicating its resources and attention to those elements of the GCRP that can best support improved public debate and decision making in the near term. In particular, a goal of the CCRI is to improve the integration of scientific knowledge, including measures of uncertainty, into effective decision-support systems. The CCRI intends to deliver products useful to policy makers in a short time frame (two to five years). To meet this goal, the CCRI aims to (1) reduce the most important uncertainties in climate science and advance climate modeling capabilities, BOX 1.2 Vision and Goals of the CCSP CCSP’s vision for the program: A nation and the global community empowered with the science-based knowledge to manage the risks and opportunities of change in the climate and related environmental systems. CCSP’s goals for the program: Improve knowledge of the Earth’s past and present climate and environment, including its natural variability, and improve understanding of the causes of observed variability and change. Improve quantification of the forces bringing about changes in the Earth’s climate and related systems. Reduce uncertainty in projections of how the Earth’s climate and related systems may change in the future. Understand the sensitivity and adaptability of different natural and managed ecosystems and human systems to climate and related global changes. Explore the uses and identify the limits of evolving knowledge to manage risks and opportunities related to climate variability and change. SOURCE: CCSP 2003.

OCR for page 13
Analysis of Global Change Assessments Lessons Learned (2) enhance observation and monitoring systems to support scientific and trend analyses, and (3) improve decision-support resources. DEFINITIONS OF KEY TERMS Definition of Assessment The committee has adopted the following definition of assessments, which is derived from its usage in the emerging literature (Parson 2003; Farrell et al. 2006): Assessments are collective, deliberative processes by which experts review, analyze, and synthesize scientific knowledge in response to users’ information needs relevant to key questions, uncertainties, or decisions. Embedded in this definition are three important and often underappreciated characteristics of assessments that need to be emphasized (Mitchell et al. 2006). First, assessments are processes that connect the domains of science and decision making (either public policy or decisions by some other actor) but differ significantly from both. For example, process assessments seek to build consensus around the latest scientific information, which is clearly distinct from the scientific process that solely reports observations and experimental results. Therefore, assessments cannot be evaluated according to the same criteria as either the process of democratic policy making or science (Clark and Majone 1985). Second, as mentioned in the definition, an assessment is a social, deliberative process and not merely an exercise of transcription or deduction. Rather, it involves synthesis directed toward a goal of supporting decisions, and its character is in many ways driven by the interaction of the state of scientific knowledge with the societal need for decision-relevant information. Third, while an assessment may generate a report, this is neither necessary nor the totality of the process. Some assessments do not yield reports, and for all assessments the process may be as effective as the report in affecting the decision-making process if the stakeholders and target audience are involved in the assessment process (Farrell et al. 2006). Defining Success: The Effectiveness of Assessments Assessments seek to inform decisions. It follows, therefore, that a measure of the effectiveness of an assessment would be its incorporation in the decision-making process. In some cases, it is possible to make a qualitative judgment about the impact an assessment has had on decision making; however, in practice, it is difficult to evaluate what other information has been considered by decision makers and how the value of the information

OCR for page 13
Analysis of Global Change Assessments Lessons Learned was weighted in the process (Social Learning Group 2001a). Economic analyses, along with numerous social and political factors, also influence decision makers, and it is virtually impossible to separate the impact of these factors from that of the assessment itself. Furthermore, a reliable evaluation of the ultimate impact of an assessment can only be done in retrospect, which requires a historical perspective not yet available for many of the assessments the committee analyzes in the following chapters (Social Learning Group 2001a). In view of the difficulty of evaluating an assessment’s effectiveness by its ultimate effect on decisions, assessment scholars have identified proximate (i.e., less dependent on context) characteristics that can be attributed to the assessment itself and that are plausibly linked to increased likelihood of ultimate effectiveness. A prominent set of such indicators has been developed by the Social Learning Group (2001a) and applied successfully in the Global Environmental Assessment project (Farrell et al. 2006; Mitchell et al. 2006). The literature identifies three essential properties of effective assessments: credibility, legitimacy, and salience (Ravetz 1971; Clark and Majone 1985; Social Learning Group 2001a; see also Box 1.3). Credibility concerns the technical quality of an assessment, as perceived by relevant scientific or other expert communities. Achieving credibility requires that the assessment avoids clear errors; involves respected experts with the right competencies; shows understanding of the relevant scientific BOX 1.3 Salience, Credibility, and Legitimacy The three essential properties of an effective assessment process are: Salience relates to the perceived relevance of information: Does the system provide information that decision makers think they need, in a form and at a time that they can use it? Credibility addresses the perceived technical quality of information. Does the system provide information that is perceived to be valid, accurate, or tested? Legitimacy concerns the perception that the system has the interests of the user in mind or, at a minimum, is not simply a vehicle for pushing the agenda and interests of other actors. Legitimacy relates to the perceived fairness of the process. SOURCE: Ravetz 1971; Clark and Majone 1985; Social Learning Group 2001a.

OCR for page 13
Analysis of Global Change Assessments Lessons Learned literature; makes use of accepted datasets, methods, tools, and models; and undergoes a scientific peer-review process. Legitimacy concerns the fairness and impartiality of an assessment, as perceived by all its users and stakeholders. It reflects their judgments that the assessment gave regard to their interests and concerns, at least was not simply a vehicle to advance the interests and concerns of some other actors. Concerns about legitimacy are most frequently expressed as objections to an assessment’s process. Achieving legitimacy depends on matters such as involving participants who represent a variety of key stakeholder groups, running a transparent process, providing avenues for input and consultation, and submitting the assessment to an open review process. Salience concerns an assessment’s ability to communicate with the users whose decisions it seeks to inform. Achieving salience depends on such matters as capturing users’ attention, addressing matters that are relevant to their concerns, communicating in terms they can understand, and presenting any recommendations in operational terms to the audience that can use them, at the time it needs them. Although these are rather academic concepts, they have attained widespread use both among scholars and in practical debates about assessment programs. The committee has found these criteria useful in analyzing assessment effectiveness. However, the committee recognized three points for which further clarification is required. First, these characteristics are ascribed by stakeholders and are not inherent characteristics of assessments. Therefore, they result from subjective judgments of the process, and the goal is to increase the number of stakeholders that attribute these characteristics to the assessment process. Certain stakeholder groups may diverge in their judgment of the process regardless of how well it was designed and implemented. Second, during the committee’s deliberation it became evident that there is the potential for confusion about the difference between credibility and legitimacy. This confusion is understandable because both credibility and legitimacy are fundamentally concerned with whether people judge that an assessment can be trusted. They must be distinguished, however, because they concern trust granted to an assessment by different audiences for different reasons. Credibility is ascribed by scientific experts if they regard an assessment as trustworthy, based on indicators similar to those they use to evaluate the trustworthiness of other scientific work (Social Learning Group 2001a). Stakeholders ascribe legitimacy if they regard an assessment as trustworthy, based on indicators of fairness, balance in representation, and transparency of the process similar to those they use to evaluate the trustworthiness of political, administrative, or legal processes. Drawing a clear distinction between credibility and legitimacy highlights the fact that

OCR for page 13
Analysis of Global Change Assessments Lessons Learned both types of trust may be required, but how each is earned might require different design choices and trade-offs between the two. Third, the term salience combines several characteristics that are all associated with effective communication with the intended audience (Ravetz 1971; Clark and Majone 1985; Social Learning Group 2001a). Most important, it includes the need for an assessment to be simultaneously relevant and widely recognized in order to capture users’ attention and, therefore, communicating in terms that they recognize as relevant to their concerns. Depending on the particular assessment and context, it may also include communicating through the right media, expressing results at a technical level, using terms and concepts that are matched to the audience, and taking regard of specific decision responsibilities and deadlines. Other Key Terms Stakeholders. The committee considers all “interested and affected parties” as stakeholders in the assessment process. This includes people whose material interests may be affected and also those who have an interest as citizens even if they do not stand to be materially affected. A distinction is made in this report between these and a specific stakeholder—the authorizing body of the assessment—that provides the assessment with its mandate and typically also with its funding. Most often, the authorizing body (i.e., those requesting an assessment) is part of a government or, in the case of the IPCC, of multiple governments. This distinction is made for the purpose of several discussions in this report, due to the fact that different processes may be required to structure the participation of the authorizing body and of all other stakeholders. Target audience. This refers to the potential users of assessments. Often, the primary target audience consists of decision makers in the federal government who are responsible for the decisions that the assessment is intended to inform. In addition, the target audience may also include state and municipal governments, private-sector users, the public, or intermediaries, who function as science translators to decision makers (e.g., congressional staff, business associations, environmental organizations). Framing. Framing refers to the process of defining the mandate of the assessment and the specific questions it is charged to address. In the framing process, the types of decision the assessment is intended to inform are identified together with the approach. STUDY APPROACH AND REPORT ROAD MAP The requested analysis of past global change assessments involved a three-pronged approach: building on existing scholarly work, drawing from

OCR for page 13
Analysis of Global Change Assessments Lessons Learned committee members’ collective experience as scholars and practitioners of assessments, and conducting some empirical analyses of selected case studies. The composition of the committee was such that it could draw from a broad range of experiences in various assessments. Besides representing many different scientific disciplines, committee members were selected to represent several major stakeholder groups: academia, decision makers, the private sector, and nongovernmental organizations. As instructed by the Statement of Task, the committee focused its analysis on assessments with goals similar to the CCSP’s. Despite the fact that this represents a relatively limited sample of assessments, the committee recognized the diversity of external and internal factors that can lead to success and the need to distinguish assessments based on their external conditions and goals when comparing and drawing conclusion regarding assessment design. Chapter 2 elaborates further on this distinction and provides a framework to illustrate the relevant factors in the assessments process, such as the context at the inception of the assessment, the conditions established at that time, and the design choices made within the conduct of an assessment. Chapter 3 reviews the scholarly literature on assessment processes to identify the key challenges in designing an effective assessment. It also draws some conclusions from the literature on what design choices increase the likelihood for an effective assessment. Based on the general framework provided in Chapter 2 and the key challenges identified in Chapter 3, the committee analyzes a selection of assessments in Chapter 4 and identifies the strengths and weaknesses in their approaches. Drawing from the literature review in Chapter 3, the analysis in Chapter 4, and Chapter 5 concludes with the committee’s overall findings and recommendations.