Strategy for Operational Climate Modeling and Data Distribution
The ability to dynamically simulate climate has existed for a little over 50 years (Phillips, 1956), even less if climate simulation is viewed as requiring coupled ocean-atmosphere models (Manabe and Bryan, 1969). Attempts to apply climate simulation to the problem of seasonal to interannual climate prediction, using dynamical ocean-atmosphere models with initial conditions based on observations, have been made for only the past quarter century (Cane et al., 1986). In contrast to the relative youth of climate prediction, the demand for weather forecast information in the United States officially dates to the 1870s when a national weather service was called for by Congress during the Grant Administration. Since then, the weather service mission has grown to include a multitude of products, including climate monitoring and outlooks, which are used daily by citizens, companies, and researchers. User sophistication has grown as well from a rudimentary expectation for advance warning of impending storms to the ability to ingest and interpret gigabytes of raw data from numerical models of the atmosphere and ocean (Chapter 12). Within the past two decades, the demand for future climate information has grown to include long-lead forecasts, seasonal outlooks, and climate change projections. These products are valuable to a wide range of sectors and regions.
Given the growing demand for climate prediction products, and the maturation of climate simulation and prediction to demonstrably useful levels of skill, a strategy for operational climate prediction is needed. Furthermore, given the sophistication of the user community and the rapidly growing number and complexity of potential climate prediction data products based on climate models with ever-increasing complexity and resolution, the strategy needs to take into account the distribution of data to the user community for application in a variety of socioeconomic sectors and for basic research.
CLIMATE MODEL DEVELOPMENT FOR OPERATIONAL PREDICTION
For the past several decades, increasingly complex climate models of increasing spatial resolution were developed as research tools to study scientific questions regarding the processes responsible for climate variability, change, and predictability (e.g., Delworth et al., 2006; Gent et al., 2011; Kiehl et al., 1998). Other than the development of seasonal prediction tools, the motivation for these advancements was primarily, until recently, to improve understanding of processes and reduce biases, not to address any particular societal need for climate predictions, although researchers realized that the results might have societal implications (NRC, 1979). More recently climate model development has been driven more by a desire to better understand the general impacts of anthropogenic climate change, and several recent reports (e.g., NRC, 2010b, Advancing the Science of Climate Change) and the U.S. Global Change Research Program 2012-2021 strategic plan (USGCRP, 2012) have noted that both scientific advancement and addressing specific societal needs should be viewed as drivers of climate model development.
The user community needs easily accessible and comprehensible climate information updated on a regular basis. One resource for users interested in decadal and longer time scales has evolved from a series of climate model comparison (or intercomparison) projects (MIPs), organized by the international research community primarily for the purpose of advising international assessments of climate change that are conducted periodically by the Intergovernmental Panel on Climate Change (IPCC, 2007a,b,c). These MIPs are described in more detail in Chapter 8. They encourage the participating model development groups to conduct a series of numerical climate change simulations that conform to a prescribed protocol, with standardized outputs placed in a distributed quasipublic archive. These simulations are increasingly used not only by IPCC and the research community but by a broad range of users as source material for assessments of climate variability and change and as inputs to other models specialized to particular applications.
A second resource for users is “operational” climate forecasts (see Box 9.1) for lead times of months to a few years. Several weather services around the world have developed climate models specifically to provide scheduled, real-time, forecast products. For example, the U.S. National Weather Service has developed the Climate Forecast System (Saha et al., 2006, 2010) to produce operational climate predictions with lead times of up to 9 months. The second generation of this system went into operation in March 2011. The European Centre for Medium-Range Weather Forecasts has developed a seasonal climate prediction system, soon to be in its fourth generation
BOX 9.1 OPERATIONAL PREDICTION
As in operational numerical weather prediction, several characteristics of operational climate prediction make it distinct from climate model research and development. First, the goals of operations are driven by a user community rather than scientific advancement. There is no value judgment implied by this, but the implication is that the needs of the user community have to be assessed regularly, operational products must respond to users’ needs, and there is an expectation for improvement over time in various aspects of users’ experience. Second, operations must conform to a specified schedule of generation and delivery of products. Users expect products to be available in time, on time, every time, which requires a mindset and a working protocol that is not necessarily appropriate in a research and development setting. Third, operational prediction requires dedicated resources and contingency (failsafe) planning. Model developers often work with resources that have been obtained through a competitive process, on an ad hoc basis, or through windfall opportunities, but those modes are far too undependable for operational requirements. Operations must have a platform for product generation that is fully functional when needed and a plan in place for utilizing backup resources when that platform is out of order. Finally, operational computer code should conform to rigorous standards of software engineering that may or may not apply to research codes. While many climate prediction research groups are shifting to a more formal software engineering approach (Chapter 10), primarily motivated by the need for including the input from a wide community of researchers and model developers, there remains a more informal methodology in most model development groups that enables and even encourages risk taking, as is appropriate in a research and development enterprise.
(System41), and other nations have similarly developed seasonal climate prediction systems that include a climate model developed specifically for this purpose.
There is a desire within the research community to migrate experimental climate prediction models into operational use (e.g., the National Oceanic and Atmospheric Administration [NOAA] Climate Test Bed effort to build a multimodel ensemble [NOAA, 2011]) and to improve on operational models by transitioning model components and/or parameterization schemes from experimental models developed in the broader community, motivated by the growing expectation for governments to provide climate services (e.g., Dr. Jane Lubchenco’s testimony before Congress during the hearings to confirm her as Undersecretary of Commerce for Oceans and Atmosphere and Administrator of the National Oceanic and Atmospheric Administration
___________________
1http://www.ecmwf.int/products/changes/system4/ (accessed October 11, 2012).
[Lubchenco, 2009]). This migration of experimental models into operational use has the potential of efficiently leveraging the U.S. climate research community to provide more skillful and comprehensive climate predictions. Effecting this transition is difficult, because of gaps between research goals and operational imperatives (e.g., that changing an operational model requires a more careful and elaborate process than for a research model) and mismatches between resource requirements needed to maintain an operational model and the current distribution of resources between research, development, and operations. There is clearly a need for adequate support for research on climate modeling, operational climate prediction, and an effective interface between the two.
Finding 9.1: Some operational seasonal-to-interannual prediction efforts are already under way, and there are archives of model output from research-oriented international climate model intercomparisons focused on multidecadal to centennial climate simulation; these archives do not cover all of the needs of climate information users.
ISSUES RELATED TO OPERATIONAL CLIMATE MODELING
The current practice of configuring and running climate models is primarily done by a relatively small number of developers and programmers with insufficient support for robust code development and support. Many aspects of climate model development and usage (e.g., setting up model experiments and “tuning” climate model parameterizations) cannot be made routine when model configuration and execution demands such large efforts from a small number of people. This practice also does not facilitate rigorous attention to reproducibility, which is needed to ensure credibility. Finally, singular efforts such as are the current practice do not adequately support the sustained two-way conversation that must take place between developers and user communities2 regarding requirements, expectations, use cases, etc. Interactions between model developers and other communities of researchers, practitioners, and decision makers are beginning to be encouraged; for example, the Community Earth System Model project recently added a working group on societal dimensions.3
Finding 9.2: The current collection of efforts for research in climate model development is not well positioned to perform operational climate modeling.
___________________
2 Individuals and groups interested in applying climate model outputs to the management of the societal effects of climate change.
3http://www.cesm.ucar.edu/working_groups/Societal/ (accessed October 11, 2012).
For decades there has been a mismatch between the expectations of the operational numerical weather prediction (NWP) and climate prediction community and the model research and development community. The principal measure of success of work that is supported by a typical short-term (e.g., 3-year) research grant is the number, quality, and impact of the research publications that result from any project. Researchers receive no reward for developments that become “operational,” so there is little incentive to do what is viewed as extra work to transform research results into operational methods or procedures. There is a view that the scholarly publications speak for themselves, which has been described as a “loading dock” approach—the research results are made available to the operational prediction community via peerreviewed publications (left on the loading dock), and it is up to users to figure out how to use the results. There are some nascent efforts in which the transition to operations is the objective rather than a by-product of research, e.g., the NOAA Climate Test Bed activity.4
From the operational community point of view, there are a great many constraints imposed by operations that should be taken into account by the researchers who seek to improve the operational predictions. In order to effect a transition from research to operations, they argue, the research community needs to modify its developments to conform to the constraints of operations so that their results can become useful, and the operational center needs to provide infrastructure support for the research community to use the operational model to conduct its research. The mismatch between the two communities’ expectations has been called the “valley of death,” that is, a communication and interaction gap. There is clearly a need to better align the two communities and provide adequate resources so that good ideas can be more rapidly and effectively transformed into operational practice.
Finding 9.3: The expectations of the research community and the operational prediction community are not well aligned.
As indicated throughout this report (Chapters 1 and 10), a market for climate model information already exists. Given the growing need for information about future climate from climate models, involvement of the private sector could be beneficial. The private sector is already engaged through consulting companies that provide customized and downscaled climate information. A number of private companies successfully sell climate information that depends on climate models. Examples include Prescient Weather, Ltd.,5 Atmospheric and Environmental Research Inc.,6 Risk Management
___________________
4http://www.cpc.ncep.noaa.gov/products/ctb/ (accessed October 11, 2012).
5http://www.prescientweather.com/ (accessed October 11, 2012).
6http://www.aer.com/ (accessed October 11, 2012).
Solutions Inc.,7 Stratus Consulting,8 and ICF International.9 An important question to resolve is the appropriate balance between the private sector and government organizational structures, such as a national climate services operation.
Is there a potential benefit of involving the private sector more directly in climate modeling? Could a market be created for climate model information? How much is happening already?
Such a balance has been struck in the weather community. Since the inception of the weather enterprise in the 1800s, it evolved to include three sectors: the National Weather Service, academia, and the private sector. Each plays a vital and unique role in weather services, and the competition between the sectors led to a flourishing and extensive set of valuable weather services. However, friction and conflict also abounded. Policies were initiated in the latter part of the 20th century to try to identify the roles and missions of the various sectors, but the boundaries were never as crisp as some wanted and the conflicts continued.
A 2003 NRC report, Fair Weather: Effective Partnerships in Weather and Climate Services, concluded that it is more effective to define a process for evaluating and adjusting the roles of the private, public, and academic sectors than to rigidly define such roles. Such a process entails the American Meteorological Society (AMS)—a neutral party—hosting a forum for civil interactions and discussion among the three sectors. AMS formed the Commission on the Weather and Climate Enterprise, which consists of three boards and several committees devoted to the various aspects of the partnership. The existence of the commission has resulted in a significantly reduced atmosphere of conflict among the three sectors and has helped introduce an era of cooperation. The joint participation of the private, public, and academic sectors has resulted in better data coverage, wider information dissemination, more realistic and scalable models, increased infusion of cutting-edge technology, and a greater number of specialized products (NRC, 2003).
Finding 9.4: The private sector already has at least some role in providing climate modeling information. There is precedent for this, for example, in how the National Weather Service interacts with the private sector.
As described in Chapter 10, standardized model outputs from the leading international climate models are routinely combined through the Coupled Model Intercomparison Project (CMIP) efforts. These CMIP outputs have heavily contributed to the
___________________
7http://www.rms.com/ (accessed October 11, 2012).
8http://www.stratusconsulting.com/ (accessed October 11, 2012).
9http://www.icfi.com/markets/climate (accessed October 11, 2012).
IPCC assessments, as well as provided accessible climate model simulations to a wider community of users and researchers. A centralized data archive was initially developed by the Program on Climate Model Diagnostics and Intercomparison at Lawrence Livermore National Laboratory in the United States, but as the volume of model output has grown, an internationally coordinated collaboration was developed, referred to as the Earth System Grid. As described in Chapter 2, self-organized grassroots efforts such as the Global Organization of Earth System Science Portals (GO-ESSP)10 and the Earth Systems Grid Federation (ESGF),11 as well as short-term grant-funded projects such as Metafor12 and ExArch,13 are responsible in large part for building the data infrastructure underlying CMIP5. These CMIP efforts are a vital backbone for efficiently providing current climate model output to diverse user communities. However, despite these efforts, the data sets are growing in size and complexity, and users require even more sophisticated access methods than are currently available. The demands being placed on these networks far exceed the capacity of volunteer energy: it is high time that the global data infrastructure was recognized as “operational” and resourced as such.
Finding 9.5: The global infrastructure for modeling and data distribution is currently a community-owned federation without formal governance. The climate modeling community increasingly recognizes the need to focus on the reproducibility of results, and of traceability from results back to the methods and models used to produce them. For instance, the CMIP5 project records model provenance using a questionnaire developed by the Metafor project. It is in fact quite possible to record provenance, provide codes that produce specific results, etc., such that third parties can in fact attempt to reproduce or vary them (Balaji and Langenhorst, 2012), aided by software tools such as OLEX,14 but these methods still need enthusiastic adoption by the community.
Finding 9.6: Climate modeling groups have not universally put a high priority on workflow provenance, i.e., documented steps from model configuration to a given output data set or graphic that make the process transparent and reproducible.
___________________
10http://go-essp.gfdl.noaa.gov/ (accessed October 11, 2012).
11http://esgf.org/ (accessed October 11, 2012).
12http://metaforclimate.eu/trac (accessed October 11, 2012).
13http://proj.badc.rl.ac.uk/exarch (accessed October 11, 2012).
14http://olex.openlogic.com (accessed October 11, 2012).
THE WAY FORWARD
Taking the best advantage of research findings for operational climate prediction requires a tight linkage, with shared goals, shared decision making, and shared resource allocation, between the research and development community and the operational prediction community. The transition of research advances to operations requires dedicated resources for external scientists to work on operational models and for the operational center to accelerate the transition to operations. There is a large benefit of confronting models with observations (e.g., through operational data assimilation and prediction) for advancing and potentially transforming climate model development.
A strategy for enabling a more rapid and effective transition from research to operations that can take best advantage of recent research advances and model developments in the academic community will require more sophisticated interactions among climate model developers, climate simulators, data assimilation experts, and climate analysts. This strategy should include a closer alignment of the goals and expectations of the research and development community with the goals and expectations of the operational prediction community and changes in the rewards system that recognize the value of contributions to operational climate prediction. A systems approach is needed that takes into account (1) the rigor needed for scientific advancement, (2) society’s needs for information from climate models, (3) the complexity and volume of data generated by climate models, and (4) the complex relationships among government laboratories, university research groups, the private sector, and potential users of climate model information. A meaningful and robust personnel exchange, whereby research scientists spend a significant period of time visiting operational centers to help advance prediction science, would be a beneficial part of such a strategy. Operational centers would best promote these advances by providing operational models, supporting data sets, and a user-friendly model testing environment that allows external researchers to test experimental parameterizations and/or model components in an operational setting.
Closing the gap between research and development and operational prediction will require the capability to establish workflow provenance and automate analysis where feasible and reasonable, for which research and development are needed.
This committee judges that data management of the large and complex data sets that are regularly produced by climate modeling research and operations should be viewed as on par, in terms of importance and resource provisioning, with the research and development, simulation, and prediction efforts themselves. Data distribution, including robust technical support for remote analysis to the large and diverse stake-
holder community, needs to be viewed as an operational imperative rather than a research project, with appropriate management and resource allocation.
The vision of the committee is that over the next 20 years climate modeling for interannual to decadal to centennial time scales will develop a stronger operational component. The model predictions will be substantially more robust and provide a considerably richer and more comprehensive set of products that are closely tied to recent scientific research developments in climate modeling. Although there have been previous calls for the United States to commit to the production of operational climate data products for model-based global climate projections (Chapter 2), the committee feels that it is too soon to make such a commitment for decadal to centennial prediction. Considerable research and dialogue among stakeholder communities is needed to determine if there is any overlap between what can be predicted and what needs to be predicted, in particular for decadal time scales.
Recommendation 9.1: To better address user needs for short-range climate predictions, the U.S. and international modeling communities should continue to push toward a stronger operational component for prediction of seasonal climate and regular experimental simulation of climate change and variability on decadal time scales.