III
USEFUL FRAMEWORK FOR UNDERSTANDING FORECASTING EFFORTS
As input to the meeting, the workshop organizers drafted a background paper for discussion purposes that identified several themes that were thought to be constructive in thinking about the challenges of connecting research about climate to action.5 The concepts were discussed by participants and the two items described below distill what about the themes resonated most with participants’ experience. They are presented as a scaffolding of ideas, or way of thinking about the challenges that confront scientists, decision makers, and program managers in producing and using climate forecasting information.
1. Systems for Linking Knowledge to Action
As an analytical tool, it can be useful to think about linking research and decision making from a systems perspective. One term that many participants found useful in describing the assemblage of institutions and activities that produce and utilize climate information is knowledge-action system.6 These systems are generally viewed as organized efforts to harness science and technology in support of social goals. Such systems have been developed at the national and international levels for social goals ranging from food production and health to manufacturing competitiveness, and defense. In general, they encompass the set of relationships, actors, institutions, and organizations that set priorities, mobilize funds, do the R&D, review publications/promotions, facilitate practical application and reinvention, and provide evaluative feedback on performance. Such systems are not generally designed from scratch, but rather evolve through time as a result of multiple and only partially integrated interventions. This is seen, for example, in the evolving role of FUNCEME in Brazil, from primarily a meteorology agency to one that integrates climate and weather information with hydrology, soil sciences, other natural resource sciences, and social responses. It does so with broadening relationships with international, national, state, and local partners. While far from complete, the efforts in Colombia to link climate forecasting to public health efforts demonstrate the emergence of a knowledge-action system and the mobilization of both existing and new resources to solve a variety of social health challenges.
2. The Challenge of Producing Information That Is Salient, Credible, and Legitimate
Workshop participants used a pragmatic working definition of what constituted an effective system for linking knowledge to action: Such systems may be considered more “effective” to the extent that a greater proportion of the potentially relevant knowledge that science has to offer is actually taken into account by users when making their decisions about which actions to take.
5 |
The discussion paper is available on the workshop Web site <http://www7.nationalacademies.org/sustainabilityroundtable/Decision_Support_Rountable_Main.html> and is an unpublished document that was provided to participants as a possible framework for discussions. |
6 |
The formal title of the Workshop referred to “Decision Support Systems,” reflecting current usage in the United States (e.g., in the core documents of the Climate Change Science Plan). Participants pointed out that this term has a substantial and not particularly distinguished history of use in which it implies a rather narrow and technocratic approach to computer-assisted decision making. (See, e.g., the special issue of Agricultural Systems Vol. 74(1), Oct 2002, on “Probing the enigma of the decision support system for farmers: Learning from experience and theory.” See also “A brief history of decision support systems” by DJ Power, Editor, DSS Resources.com <http://www.dssresources.com/history/dsshistory.html>. Participants preferred to use the full phrase “systems for linking knowledge to action” or, for short, “knowledge-action systems.” This phrase is not without its own problems, but it is adopted here as preferable to “decision support systems” for use, at least in international contexts. |
What are the general attributes of such relatively effective systems? Workshop participants discussed a hypothesis based on earlier work that systems for linking knowledge to action are more likely to be effective to the extent that they produce information that is perceived by users to be simultaneously salient, credible, and legitimate (Cash et al., 2003; Clark et al., in review). These three terms are not precisely defined but have been differentiated as follows:
-
Salience relates to the perceived relevance of information: Does the system provide information that decision makers think they need, in a form and at a time that they can use it? For example, farmers in some areas need to know something about the timing of first rains, as opposed to average expected precipitation over a season; or emergency preparedness managers need to have forecasts early enough to start preparations for potential natural hazards.
-
Credibility addresses the perceived technical quality of information. Does the system provide information that is perceived to be valid, accurate, tested, or, more generally, at least as likely as alternative views to be “true”? For example, do other scientists agree with underlying assumptions of a model? Or does ground-truthing of general research reveal that the general findings hold in a specific place?
-
Legitimacy concerns the perception that the system has the interests of the user in mind or, at a minimum, is not simply a vehicle for pushing the agendas and interests of other actors. The term fairness has been used to characterize legitimacy, but was felt by some workshop participants to convey an overly negative or suspicious view of the system. Questions about legitimacy often take the form of concerns about process or the peer group that the forecaster belongs to. For example: Who is involved in producing the knowledge? How were those involved selected? When and how are stakeholders engaged? How are R&D agendas set?
As described in the paper, the challenge of designing effective systems for linking knowledge and action—systems that produce information that is perceived to be salient, credible, and legitimate—is complicated by two aspects of the linkages among these criteria (or dimensions). First, it appears that if a system is perceived to be seriously lacking on any one of these dimensions, its likelihood of producing influential information falls significantly. (In other words, no amount of investment in, say, credibility, will make up for a serious shortfall in salience.) Second, it appears that the attributes of salience, credibility, and legitimacy are tightly linked: efforts to enhance one may either enhance or degrade another, depending on the circumstances and the strategies used. For example, greater involvement of stakeholders may increase salience (the right questions are asked) and legitimacy (a more transparent process ensues), but credibility might decrease (the science may appear to be politicized).
The art of designing effective systems for linking knowledge to action thus can usefully be viewed as the art of designing institutions (processes, organizations, norms) that balance such tradeoffs in ways that produce information perceived by users to meet simultaneously at least minimum standards of saliency, credibility, and legitimacy. How this has been done in efforts to bring knowledge about climate and its variability to bear on practical problems around the world was the focus of most of the comparative analysis conducted at the Workshop. Lessons learned emerging from those comparisons are summarized in the following chapter “Components of Effective Systems.”