National Academies Press: OpenBook

Decision Making for the Environment: Social and Behavioral Science Research Priorities (2005)

Chapter: 6 Decision-Relevant Science for Evidence-Based Environmental Policy

« Previous: 5 Environmentally Significant Individual Behavior
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

6
Decision-Relevant Science for Evidence-Based Environmental Policy

To strengthen the scientific infrastructure for evidence-based environmental policy, the federal government should pursue a research strategy that emphasizes decision relevance. Such a strategy should include four substantive research elements: (1) developing decision-relevant indicators for environmental quality, including pressures on the environment, environmental states, and human responses and consequences; (2) making concerted efforts to evaluate environmental policies; (3) developing better methods for identifying the trends that will determine environmental quality in the future; and (4) improving methods for determining the distributional impacts of environmental policies and programs. These research elements require contributions from both the social sciences and the natural sciences, as well as communication across scientific communities.

Because scientific information is critically important for environmental decision making, major research efforts in environmental science are often justified by their value to society. These efforts typically produce high-quality science, but they have repeatedly fallen short in addressing the questions most important to societal decision makers (Oversight Review Board of the National Acid Precipitation Action Program, 1991; National Research Council, 1996, 1999b, 1999d, 2001a, 2004c): they have failed to produce the right science for decision making. By pursuing this science priority, federal agencies would greatly improve the infrastructure of scientific information and methods relevant to addressing questions of importance to the potential beneficiaries of environmental science. The recommended research would allow environmental decision makers better to understand the nature, severity, and causes of environmental degradation,

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

to learn from the substantial experience of designing and implementing environmental policy in the United States, and to anticipate environmental trends and future policy needs. By focusing scientific efforts increasingly on decision relevance, such a program of measurement, evaluation, and analysis would increase the influence of empirical evidence and empirically supported theory in environmental decisions relative to the influences of politics and ideology. It would integrate the social sciences and the natural sciences of the environment and build a knowledge base that would better inform practical decision making while also informing scientific research.

Processes for determining which research is most decision-relevant should be participatory. Choices about how to construct indicators, evaluate policies, and so forth should be made with the participation of the full range of likely users of measures, evaluations, and analyses. This approach has previously been recommended for shaping scientific research agendas in a number of disparate areas of environmental science (e.g., National Research Council, 1996, 1999d, 2003, 2004c), and we state it here as a general principle. Broad involvement is essential to enhance the decision relevance and credibility of measures, evaluations, and analyses. Choices about measurement are not purely technical, so they are not appropriately left to analysts alone. Measurement focuses attention on what has been measured and away from what is unmeasured, thus embodying values about what is most worthy of attention. The affected parties in environmental decisions often disagree about what is most worth measuring, which outcomes of policies are most important, and the like, and the range of measures needed to make an analysis credible may not be obvious to scientists or government officials unless the various potential users of indicators are involved. Choices about what evidence to collect for policy are probably most appropriately made through broad-based analytic-deliberative processes, such as described in Chapter 2.

BUILDING THE INFRASTRUCTURE

Government should implement the strategy of seeking decision relevance in each area of environmental policy. This will require four kinds of research activities: (1) developing decision-relevant indicators, (2) evaluating past policies and programs, (3) improving the scientific capability to anticipate future environmental conditions and problems, and (4) measuring and monitoring the distribution of environmental impacts in the population in relation to issues of environmental inequities and their abatement. Within each of these activities, decisions about research priorities should be informed by dialogue between the potential producers and the potential users of the research.

Evidence-based environmental policy depends on having measures and

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

analyses of all the important decision-relevant aspects of the systems that policy affects. Meeting this need may require departures from usual routines of environmental measurement and analysis in at least three ways. First, the focus of measurement must encompass pressures on the environment, environmental states and conditions, and the human consequences of and responses to those conditions. Measuring conditions of the biophysical environment in isolation from its human interactions is insufficient because to anticipate the need for policy action or to anticipate or assess the results of any policy choice, it is necessary to consider the conditions of both the human and nonhuman parts of the system. Thus, it is important to measure and analyze the environmental implications of human actions that are taken for nonenvironmental reasons (e.g., trade, technological and economic development, national security, and so forth), which can exert major pressures on environmental systems or shape human responses to environmental conditions.

Second, evidence-based policy depends on an appropriately linking the social sciences and the natural sciences of the environment to provide the needed measures and analyses. The social and behavioral sciences are critical for developing measures of human influences, consequences, and responses, as well as for developing and refining processes for selecting useful measurements.

Third, evidence-based policy requires the participation of both the likely producers and the likely users of the evidence in deciding which measures and analyses are needed. This is so because for information to be decision-relevant, it must serve the needs of a variety of decision participants outside the scientific community, as well as those of scientists. In some environmental decision contexts, the potential information users differ considerably in the issues that concern them and in the kinds of information they want. Because there are often too many issues of concern to measure and analyze them all, difficult choices must be made about what, when, and how to measure. If these choices are not informed by the perspectives of a sufficient variety of information users, then environmental analysis is likely to fail to provide an adequate evidence base for decisions. Thus, the purpose of engaging information users is to promote the accuracy, rigor, decision relevance, transparency, and credibility of environmental information and analysis. The roles of the various participants should be defined to promote the achievement of these goals.

DEVELOPING DECISION-RELEVANT INDICATORS

Social science and natural science research should be integrated in a comprehensive approach to developing indicators that are relevant and usable for environmental policy. These indicators should cover not only

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

states of the biophysical environment but also human influences on nature and the impact of the physical world on humans. People involved in decisions that affect the environment need information to help them understand the ways in which possible decisions may affect pressures on the environment and for anticipating and assessing the effects of decisions on things people value. To produce such information requires integrating natural science and social science approaches, involving both the potential producers and users of information and engaging the relevant government agencies.

Measurement is the heart and soul of scientific inquiry and is also essential in defining, understanding, and managing human and environmental affairs. It is necessary for forecasting and planning for environmental situations and for assessing the results of decisions taken. Therefore, an integrated approach to measurement recognizes that environmental measurement has important users outside the scientific community and that systems of environmental measurement must serve the needs of those users as well as those of scientists. Because these various constituencies differ in the environmental situations that concern them and in the kinds of information they want, and because there are too many aspects of human-environment interaction and too many things potentially affected by policy to measure them all, difficult choices must be made about what, when, and how to measure. It is important to involve both the producers and the various kinds of users of environmental information in making these choices.

Indicators are quantitative measures, collected and reported on a regular basis, that convey useful information. Environmental indicators are based on data, such as from environmental monitoring systems, but not all data convey useful information. Thus, the minute-by-minute readings from a continuous air pollution monitor are data that can be aggregated into indicators, such as a daily average pollution level. Indices also convey useful information but are often more widely aggregated to represent broad conditions at the moment and through time. For example, the Dow-Jones Industrial Average on the New York Stock Exchange is an economic index having enormous general decision-making influence, although no one claims that it represents the precise details of any firm or sector of the economy. It is a useful composite or proxy when there is too much information to understand. An air quality index such as the U.S. Environmental Protection Agency (EPA) has developed may combine information on various pollutants measured at different locations. There is some overlap among terms; the same numbers may be data, indicators, and indices.

Indicators developed for decision relevance would enable major advances in fundamental understanding of the dynamics of human-environment interaction by vastly increasing the possibility of analyzing

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

these relationships quantitatively. They would also greatly increase the decision relevance of environmental analyses that use them by providing credible measures of variables of critical concern to both decision makers and scientists.

The Research Need

Indicators are necessary for rationally formulating, implementing, and evaluating environmental policy. In most situations, indicators are the only environmental reality that decision makers see because the environment itself is too vast and intricate to be perceived directly. Indicators are the institutional sense perceptions that tell us about the environment. The essential function of indicators is reflected in the fact that almost all environmental agencies conduct some kind of monitoring on which to base indicators. The federal government is estimated to spend $500-600 million annually on environmental monitoring. Unless the monitoring data are incorporated into indicators, they are not likely to be useful. There is no environmental equivalent to the Dow Jones; however, it is possible to go well beyond the current measures of biophysical phenomena by adding information about critical human dimensions seldom taken into account in conventional measures.

Environmental indicators have a variety of uses and users. They may be used for research, policy formulation, enforcement, management, evaluation, and public information. Users can include scientists, policy researchers, public officials at all governmental levels, representatives of affected parties, the mass media, and the public. Until now, approaches to developing indicators have not followed a comprehensive, integrated approach organized by the need to inform decisions. Rather, they have been fragmented by academic discipline, government agency, geographical and temporal scale, and in other ways. They have been driven by the perceived needs of specific groups, such as scientists and government officials responding to legislative mandates. Partly as a consequence, no set of environmental indicators in the United States commands the respect and attention of the public or policy makers.

The users of indicators typically want to know not only about environmental conditions, but also about their human connections. The Organisation for Economic Co-operation and Development (OECD) accordingly distinguishes three types of indicators: pressures (e.g., population, technology, consumption, pollutant emissions); states (e.g., the condition of ecological areas and biota); and responses (public and private actions taken to reduce pressures, protect states, and adapt to environmental changes). An indicator system might also include measures of the human consequences of environmental events that take responses into ac-

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

count. EPA’s 2003 Draft Report on the Environment (p. viii) proposes a six-level hierarchy that roughly parallels the OECD framework, although it differs by making it a hierarchy rather than a process with feedback loops. Also, the EPA effort was based on a narrower definition of environment that neglected such matters as resources and energy.

Environmental indicators in the United States have usually been framed narrowly. Initially, the effort to develop indicators was dominated by statisticians. The result was statistically sophisticated indicators that paid little attention to usefulness, either in terms of communicability or the needs of decision makers. Indices that combined diverse types of data were generally frowned on because, in the view of many statisticians, too much detailed information was lost when the data were combined into indices.

Ecologists have dominated the two most recent indicators efforts, the Committee to Evaluate Indicators for Monitoring Aquatic and Terrestrial Environments (National Research Council, 2000) and the Heinz Center group that received a mandate from the White House to develop an “environmental report card” (Heinz Center, 1999). These efforts have proposed indicators of environmental states but have omitted indicators of pressures and responses, even though, as the National Research Council report noted, indicators of pressures are “no less important” than indicators of states (p. 2).

Such efforts have not dealt adequately with measures of pressures or responses, although there are ample data that could be used to produce indicators of them. For example, data on materials flows (National Research Council, 2004a) can contribute to indicators of environmental consumption (see Chapter 5), a major pressure variable. On the response side, federal agencies have developed ample data because of the requirement under the Government Performance and Results Act of 1993 that all federal agencies measure how their missions are being accomplished. Most of these data lack any connection with measures of pressure or state, and so they are much less useful for evaluation purposes than they might be. Health data may provide another source of environmental indicators on the response side (see U.S. Environmental Protection Agency, 2003:4-1 to 4-24). The role that social scientists have played in risk assessment shows that they may make useful contributions to delineating the environment-health link (e.g., Krimsky and Golding, 1992). Efforts to modify economic indicators to take environmental factors into account have the potential to yield useful indicators related both to pressures and responses (e.g., National Research Council, 1999c). Economists have contributed much to this effort and can contribute more.

An integrated approach to decision-relevant environmental indicators will benefit greatly from the involvement of social scientists. They can bring expertise regarding the measurement of pressures, responses, and the hu-

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

man consequences of environmental events, as suggested above, as well as techniques and experience for modeling the linkages in complex systems. They thus can contribute significantly to an integrated system that includes pressure, state, and response measures. Such a system may require new mixes of disciplines, likely to include natural scientists, social scientists, and engineers.

Integrating social scientists will take special efforts. Until now, they have played almost no role in the development of environmental indicators, with a few exceptions, such as efforts to develop a green gross domestic product (National Research Council, 1999c), quality-of-life indicators (National Research Council, 2002c), and indicators of environmental sustainability. Despite considerable interest in results-based or outcomes-based environmental governance, a new book on Environmental Governance Reconsidered (Durant, Fiorino, and O’Leary, 2004), with essays by the leading social scientists in the field, does not contain any discussion of indicators to measure results or outcomes.

To ensure the decision relevance and comprehensibility of indicators, government agencies involved in developing them should create them in collaboration with the producers and potential users of the information, including a variety of nonscientists. The environmental indicator movement has run into trouble by asking scientists and almost no one else what is important to measure. This narrow approach is both common and problematic with science-based efforts to inform decisions, such as risk assessment (National Research Council, 1996), climate forecasting (National Research Council, 1999a), radioactive waste management (National Research Council, 1995), and valuation of biodiversity (National Research Council, 1999b). Indicators need to be built on an understanding of what matters most to the interested and affected parties to decisions (National Research Council, 1996), not just what matters most to scientists. Indicator development needs also to recognize that information that might be meaningful to one individual in given circumstances may be irrelevant or incomprehensible to another person elsewhere. For example, indicators that are meaningful to an ecologist studying an aquatic environment may be incomprehensible or of questionable relevance to a fishing boat operator.

Decision-relevant indicators must be developed with due attention to the variety of decision participants and the range of values that matter in specific decision-making settings or contexts. Indicators need to be understandable, quantifiable to the greatest extent possible, applicable to the realistic setting or circumstances at hand, and relevant to users’ concerns. They must portray information about the past and present of valuable ecosystems and include relevant and significant human dimensions all at the same time. Good indicators give a picture or map that reveals fundamental issues and suggests possibilities. They allow one to extend past

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

trends (what forecasters and planners call the “null projection”) to generate a plausible scenario for the future. They make problems stand out in strong relief and suggest where analytic and managerial attention might best be focused.

Indicators often reflect values in the sense that people make judgments about which direction they would like the indicator to go (e.g., less air pollution is better, fewer species is worse). This value component makes it important to show that the link between the indicator and the relevant value is really valid. For human health indicators, the validity problem involves finding the right weights to give to diverse health threats so they can be combined into a single scale. For ecological indicators, a problem is that concepts of ecosystem health are very subjective (National Research Council, 2000:24). Social scientists can contribute much to the clarification and amelioration of these problems, for example, by studying people’s understandings of proposed indicators. Decision scientists can help by encouraging other scientists to distinguish the variables that may be elements of human or ecosystem health in ways that promote empirical analysis.

Decision participants’ values also reflect their geographic, socioeconomic, cultural, occupational, and ethnic positions. Because of these different positions, some individuals face greater risks than others from particular sources of pollution or types and locations of resource degradation, and their exposures to risks and opportunities affect the kinds of information they need most for their decisions and the kinds of indicators they find useful and credible. Spatial decision support tools are being developed that can help illuminate the spatial and temporal aspects of particular environmental pressures or conditions, such as climate change, flooding, habitat degradation, and the transport of pollutants, but far less is known about how to illuminate the socioeconomic or cultural aspects of the uneven distribution of environmental risks and benefits across human populations. The development of decision-relevant indicators should take into account diversity among information users with regard to which information they need most.

For such reasons, choices about what to measure and how to aggregate data into indicators raise important and researchable questions about strategy and procedure. The answers will be contingent on the nature of the problem, decisions to be informed, the scale of those decisions, and other factors. Because the users of indicators normally have various perspectives on these matters, the answers may also be contingent on who asks the questions.

These choices should therefore be informed by the needs of potential users of indicators. The costs of environmental measurements need to be balanced against the value of information they provide. Scientists have a

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

hearty appetite for data, but from a decision-making perspective, it is important to strive to make indices few, inexpensive, comprehensible, and decision-relevant. Using and modifying the data collection and analysis efforts of others is one tactic to attain this end. Assessing the value and utility that decision makers and the public place on various indicators and discarding the least effective is another.

Users’ concerns have led to interest in issues of environmental justice, especially in the context of evaluation of environmental policies. Although there is considerable debate over what constitutes environmental injustice (Bowen, 2002), such debate can be informed by indicators that represent the consequences of environmental decisions at sufficiently fine-grained scales to identify how environmental risks are distributed among different populations. We return to this issue later in the chapter.

Good indicators must be not only technically valid and decision relevant, but also comprehensible to the potential users, who include a variety of nonscientists. Broad indices that combine several different types of data have especially strong potential for making information useful to the public and the mass media. Little work has been done on indices, however. There is a need to balance the concerns of technical specialists familiar with the environmental data, who often feel that indices distort the individual data sets, and the need to make information broadly meaningful. The EPA’s effort to develop air pollution indices (e.g., U.S. Environmental Protection Agency, 2003:1-4) is important but exceptional. There is, for example, no equivalent effort to develop water pollution indices.

Social scientists have a good deal of expertise that can be brought to the problem of index construction, including expertise in communicating information and in combining data from diverse sources. They also can help in analyzing the strengths and weaknesses of indices, such as the sustainability indices that have been proposed.

The objectivity or disinterestedness of the parties collecting and presenting indicators cannot be overemphasized. Transparency of collection, measurement, and presentation is crucial, particularly when trying to earn and maintain public trust and confidence. The essential questions here are: Who should provide the information? How should objectivity and transparency of the work be ensured? How should the information be made accessible to a full range of participants?

Good indicators require close collaboration among existing organizations and may require the creation of new ones. Many data problems arise from a lack of coordination among different institutions. Federal agencies often do not share data with each other, data collected by states may not be in a form useful to the federal government and vice-versa, and data sharing and review between the public and private sectors may be inadequate.

Data improvement also may depend on the creation of new institu-

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

tions. There have been repeated attempts to create a federal Bureau of Environmental Statistics but none have succeeded. Environmental policy is thus deprived of a source of impartial data and indicators. Almost all other major policy areas have an institution that supplies this need, such as the Bureau of Labor Statistics, the Center for Health Statistics, and the Energy Information Administration. Recommendations to create a federal Bureau of Environmental Statistics deserve serious attention because of the need for collaboration. There is also a need for stronger institutions to provide international environmental indicators.

Social scientists know a good deal about improving coordination and about creating and strengthening governmental institutions. They could thus help to strengthen the environmental data base and lay the groundwork for institutions that will provide good environmental data on a continuing basis.

Special efforts may be required to enable rapid development of indicators under conditions of surprise or disaster. It is impossible to develop indicators for every environmental policy situation that may arise. It may be possible, however, to build the capability for developing indicators quickly when there is a new need. Consider the Exxon Valdez disaster. Even though the accident affected a highly valued and well-studied ecosystem, there were unanticipated needs for measurement and for rapid scientific judgments about what to measure. Existing environmental indicators certainly helped those making decisions about minimizing the impacts of the spill, but much more was needed—and fast. Had the Valdez simply sunk in the middle of the ocean, the need for environmental measurement would have been far less, but also far different.

For such unexpected environmental events—oil spills, natural disasters, nuclear accidents, terrorist attacks, and so forth—a measurement strategy of rapid assessment makes sense. In these situations there may be preexisting environmental data and information, but probably not enough and not in forms that decision makers need or can use. The scientific challenge is thus to bring existing general knowledge about processes and interactions to bear as a means of selecting and sorting through the entire array of possible things to measure in the specific circumstances.

The interesting scientific tasks here involve anticipating how the surprising events might develop and then imagining and deducing what information would be required to cope. In the Valdez disaster, ecological and economic consequences were among those needing measurement. In the wake of the destruction of the World Trade Center on September 11, 2001, the public health effects of exposure to tons of toxic dust were among the effects to be monitored. The medical concept of triage is relevant as a reminder to decide what is important and manageable. Of course, these decisions call attention to other questions: importance to whom and manageable at what costs?

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

Rapid assessment efforts can draw on the knowledge and experience of social scientists, particularly disaster researchers, in rapid mobilization of scientific efforts in disasters and in understanding the ways crisis scenarios typically proceed. Rapid assessment would require teams with expertise across the sciences to consider the full range of consequences to be monitored and indicators to be developed.

Efforts to develop indicators should include the following steps regardless of the environmental system or problem for which indicators are needed. In general such efforts should entail the following steps: (1) clear identification of the user audience and the uses to which the indicators will be put; (2) an inventory and evaluation of existing efforts and indicators; (3) development of new methods for indicator construction and new indicators, if necessary; (4) identification of the data needed for the indicators and an evaluation of the availability of the data; and (5) pilot testing of each indicator to analyze how well it meets the specified uses. Extensive trial-and-error testing of different indicators will usually be necessary to see which ones best met the needs they were intended to fill.

Rationale for the Activity

Environmental indicators that are designed as recommended, to be decision relevant, would clearly have great potential value for environmental decision making. They would also have a strong likelihood of being used, despite some resistance that might be anticipated. The federal government makes large investments in indicators now, and these may well increase because of growing emphasis on assessing the results of government policies (see the section below on evaluation). Thus, improved indicators probably would be welcomed by a wide variety of potential users, and particularly by environmental policy makers.

Less certain is the issue of whether the field is ripe for making significant advances. Part of the problem is that the kind of transdisciplinary research community that is needed for developing the needed kinds of indicators for decisions at the national, state, and local levels is not yet organized. There is no academic discipline or set of journals associated with developing the kinds of comprehensive environmental indicators needed for policy decisions in the United States. There are social scientists working on related kinds of indicators, for example indicators of sustainable development and of vulnerability to climate change, but relatively little of this work has been directed to developing indicators to address salient environmental policy questions in the United States. Thus, this field cannot be judged ripe based on preexisting research. But the existence of research communities working in closely related areas suggests that the area is ripe for development by attracting researchers from those areas.

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

An effort to develop the recommended kinds of indicators might be sufficient to attract those social scientists and create the kind of community of scholars that exists in other research areas. The scholars are there. The research questions exist. What is needed is the spark to bring them together, and a governmental effort to develop better environmental indicators, even for a single important environmental system, could provide such a spark. There is a wealth of environmental indicator material available for social science researchers to use, and the field is virgin territory. If policy makers convert their demand for answers into a demand for comprehensive indicators, this demand could mobilize a new set of researchers who have the necessary skills but lack only the recognition of the contribution their skills could make.

EVALUATING ENVIRONMENTAL POLICIES

Federal agencies should support a concerted research effort to evaluate the effectiveness of environmental policies established by public and private actors at the international, national, state, and local levels. This research would analyze the evidence of the effects of past policies by applying and adapting techniques of evaluation research that have been used to assess the effectiveness of social welfare policies to the domain of environmental protection. It would examine the outcomes of environmental policies and their alternatives in terms of such policy goals as effectiveness, efficiency, fairness, and public acceptability; strengthen methods and capacity for determining the results of environmental policies; and thereby help to answer the call for results-based management in government.

The Research Need

Due in large measure to the series of sweeping environmental statutes signed into law in the 1970s and 1980s, the quality of the nation’s air, water, and land has improved dramatically over the past several decades. But even while these laws have led to substantial gains, many in government, business, and environmental advocacy organizations maintain that environmental policies could be more effective in achieving their goals (National Academy of Public Administration, 1995; Ruckelshaus and Hausker, 1998; Portney, 1990). Evaluation of environmental policies is a necessary step to improving policy effectiveness.

Using the tools of evaluation research, those in government and the private sector can begin to answer a range of pressing questions concerning the appropriateness and effectiveness of environmental policies. Policies may be evaluated before they are implemented (ex ante) to determine the “best” option among alternatives. “Best” may be determined on the basis

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

of efficiency, as in the case of benefit-cost analysis, or by risk reduction, fairness, or other environmental, social, political, or economic criteria. Policies may be assessed after implementation (ex post) to determine whether any of a number of conditions has changed: public health conditions, environmental quality, environmental performance of regulated entities, or managers’ perceptions about the costs, benefits, and effectiveness of the new rules. Evaluation can answer whether programs are accomplishing what they set out to do, where additional resources are likely to advance policy objectives, and where continued efforts are likely to prove fruitless. Evaluation allows policy makers to learn from experience: to identify the lessons of public and private experience developing and implementing environmental policy and to use these lessons to make policies more effective.

Despite the substantial potential value of rigorous evaluations of environmental policies, such ex post efforts are relatively rare (see Appendix D). Most evaluations of environmental policies are conducted ex ante and focus on whether the costs, measured in dollars, will outweigh the benefits (Knaap and Kim, 1998). Such assessments have the limitation of judging the appropriateness of a proposed policy on the basis of a single measure. Also, because they are essentially projections, they do not draw on evidence from actual experience.

Several factors make it difficult to evaluate environmental programs rigorously (Appendix D; Harrison, 2002). One is the need to judge the outcomes of a policy for the group or region that is subject to it against the outcomes for a suitable comparison group. When environmental policies are national in scope, finding such a comparison group is especially difficult and it is necessary to construct plausible counterfactual scenarios. Another obstacle is the lack of suitable outcome indicators against which effectiveness can be measured. The need for such indicators is discussed elsewhere in this chapter. It is worth noting that indicators developed for different purposes may not be right for policy evaluation (Gormley and Weimer, 1999). Also, when indicators are used for management, they have the potential to distort incentives and encourage managed entities to “teach to the test,” as the phenomenon is called in educational evaluation.

A major technical obstacle to rigorous evaluation is the necessity and difficulty of controlling extraneous variables. To establish a causal connection between a policy action and change in the natural environment or human outcomes, it is necessary to extract from consideration major influences that are independent of the policy, such as fluctuations in the economic cycle, developments of new technology, changes in land use and development, meteorological conditions affecting pollutants and natural resources, and so forth (Powell, 1997). Although there are ways to address this problem technically, the extraneous variable problem, especially the problem of economic effects, has generally been ignored in environmental

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

policy evaluation. Social scientists could make a major contribution by researching how to control for extraneous variables that muddy the interpretation of the effects of environmental policy decisions.

Evaluation of policies is also made difficult by the number of variables that shape how the targeted entities perceive and respond to policies. In firms, these variables include a facility’s history of environmental management, size, customers, and numerous other factors (see Appendix D). With individuals, they include attention to messages, trust in the sources of information, and perceived difficulties of engaging in the behaviors the policies are promoting (National Research Council, 1984; Stern et al., 1986; Gardner and Stern, 2002). Research needs related to understanding the factors that shape environmental performance are discussed for firms in Chapter 4 and for individuals and households in Chapter 5.

Finally, there is the potential for evaluation research to be used to justify or delegitimate existing policies according to a government’s political agenda. This potential underlines the importance of establishing effective quality control and review procedures for environmental policy evaluations.

Areas of Research

One useful area for evaluation research concerns the ongoing debate about the effectiveness of environmental regulations. For example, technology-based regulatory standards that specify how regulated entities must act and performance-based standards that specify the outcomes they must achieve have been credited with substantial improvements in environmental quality (Ashford et al., 1985; Houck, 1994; Shapiro and McGarity, 1991). Yet these approaches are also criticized for being rigid, fragmented, complex, costly, and failing to accommodate or motivate innovation (Ackerman and Stewart, 1985; Ayres and Braithwaite, 1992; Chertow and Esty, 1997; Fiorino, 1996; Gunningham and Grabowsky, 1998; Hahn and Stavins, 1991; Orts, 1995; Pildes and Sunstein, 1995; Stewart, 2001; Teuber, 1983). Furthermore, such regulatory approaches may be unsuitable for addressing environmental problems created bydisbursed, or nonpoint sources; environmental impacts from consumer, service, and agricultural sectors; or problems whose source is distant in terms of time or place. Other styles of regulation, such as by risk, exposure, licensing, and so forth, also have strengths and limitations. These issues raise various evaluation questions: What are the accomplishments and limitations of particular styles of environmental regulation at international, federal, state, and local levels? Do they address the most pressing environmental problems? What does each style of regulation do best, and what does it fail to do? What, in other

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

words, are the comparative advantages of different regulatory approaches and the needs for innovation in environmental policy?

Evaluation can also focus on a variety of policy reforms that EPA and state agencies have initiated to address the judged deficiencies of past regulations. These reforms have been categorized into three broad types: informal rules, economic incentive systems, and reflexive law (Stewart, 2001). Informal rules allow regulators leeway in how they interpret environmental laws. For example, EPA has tailored regulations to fit the specific circumstances of a firm (e.g., Project XL) or an industry (e.g., Strategic Goals Initiative for the Metal Finishing Sector). Economic incentive systems impose a price on pollution and allow individual firms discretion in the level of pollution they generate (e.g., sulfur dioxide emissions trading programs). Reflexive law attempts to create conditions under which facility managers are exposed to new sources of information about their environmental conduct, so that they reflect self-critically about their performance (an example is the Toxics Release Inventory, which requires firms to disclose environmental performance information and better enables them to monitor and manage their own emissions). These new approaches have assumed substantial significance in parts of EPA and state and local environmental protection offices.

Should the existing legal structure change to accommodate alternatives to conventional environmental regulation? How do these innovations stack up against the status quo? To what degree do they address the deficiencies critics have noted? What are the conditions under which they seem most suitable? Answering such questions will require developing a clearer understanding of how environmental innovations have worked in practice. Some innovative policies have been subject to systematic evaluation, including market-based instruments (e.g., Tietenberg, 2002) and negotiated rule-making (Langbein, 2002). But many innovative approaches, even some that are mature programs, have so far not had the benefit of serious assessment. Such evaluation would be valuable for informed decisions about institutional and legal change.

Evaluation research should also focus on the impact of environmental policies on the behavior of firms (see Appendix D and Chapter 4). While EPA has been experimenting with innovative programs, many firms and trade associations have apparently moved from fighting external pressures for better environmental protection to incorporating environmental concerns into internal decisions and normal management practices. What is the link between environmental policies promulgated by agencies and these changes in the ways managers define their responsibilities? To what degree do the environmental policies of Europe and international organizations affect firm behavior in the United States? Fruitful areas for evaluation include the effects of the Coalition for Environmentally Responsible Econo-

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

mies, the American Chemistry Council’s Responsible Care initiative, and the ISO 14001 system of third-party certification. The lessons of such efforts have not been learned or incorporated into policy.

Learning from evaluation can help public and private policy makers better determine the appropriate roles of various policy approaches. What shape should environmental policy take based on what we have learned from experiences with various styles of regulation and alternative policies? How are various approaches most appropriately combined in an overall policy strategy?

Establishing environmental policy evaluation as a research priority will encourage researchers to answer such questions. It will also strengthen methods for untangling the causal relationships between policy implementation, behavioral change, and environmental outcomes. These causal relationships are difficult to discern, whether the question at hand is the impact of a local land use policy on wetlands protection or the effectiveness of a federal initiative to conserve energy. Determining such causal relationships requires sophisticated understanding of how best to design research, utilize available data and overcome data gaps, and identify appropriate comparison groups. A concerted research effort on environmental policy evaluation will lead researchers to address these methodological challenges and advance the field. Such advancement should include expansion of evaluation criteria to include criteria of substantial importance to segments of the public, such as fairness and inclusiveness.

Rationale for the Activity

Evaluation research has a history extending back to assessment of the effectiveness of the New Deal programs of the 1930s (Rossi and Freeman, 1993). It became much more widespread after President Lyndon Johnson signed an executive order in 1965 establishing the Planning-Programming-Budgeting System (PPBS) as a requirement for federal program managers, leading to substantial increases in public spending on evaluation and the establishment of offices of program planning and evaluation in federal agencies (Haveman, 1970). Policy evaluation came to involve educators, sociologists, public health scholars, and psychologists, all bringing distinctive and useful measurement tools from their disciplines (Suchman, 1967). Later, researchers joined from economics and political science and then from various management disciplines. During the 1960s and 1970s, most policy evaluations focused on the effectiveness of human service programs to determine ex post whether efforts were achieving their intended results (Caro, 1971). The early generation of evaluation research, though often academically rigorous, was not necessarily intelligible, timely, or useful for decision makers. The simple question “Is this program working?” often

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

was ignored or could not be answered (Szanton, 1981). The mismatch between analytic standards and client acceptance has long been a focus of thinking and writing about evaluation (Cronbach et al., 1980).

During this period environmental policies were subject to ex post review only rarely, due in part to the conceptual and methodological challenges already noted. Environmental policy evaluation was also impeded by the expense and difficulty of collecting outcome data, which often could be interpreted only through models that took into account meteorological conditions, human and environmental exposures, uptake mechanisms, and so forth.

Relevant environmental data are more readily available now than previously. For example, researchers can access the substantial information EPA and states collect from regulated plants to evaluate the effectiveness of regulations and alternative policies. This information is a valuable resource for program evaluation (Metzenbaum, 2003). Compliance and enforcement data about facilities, available in on-line databases such as EPA’s Integrated Data for Enforcement Analysis (IDEA), Enforcement and Compliance History Online (ECHO), and the Sector Facility Indexing Project (SFIP), can be used to study trends in regulatory compliance and the relationships between compliance and interventions, such as facility inspections, technical assistance, and introduction of a voluntary program. These data could also be used to test the effects on compliance of environmental programs initiated by business, such as ISO 14001, the international environmental management standard, and responsible care. EPA and states could use the results of such evaluations to strengthen enforcement strategies.

Compliance and enforcement data bases also contain valuable information about facilities’ environmental releases. The IDEA database includes data on facility releases to air, land, and water. EPA’s Office of Pollution Prevention and Toxics has recently created the Risk Screening Environmental Indicators Model, which allows researchers to calculate the risks associated with facility releases. The model estimates the toxicity of chemicals that facilities report to EPA’s Toxics Release Inventory and models their human exposure. It covers virtually all facilities reporting the data since 1990.

The EPA’s Office of Environmental Information, created in 1998 to fill gaps in health and environmental data, has helped to make these data available, develop common standards for data from different geographic locations and environmental media, and ensure data accuracy. EPA assigns every facility required to report release information an identification number so that information can be integrated and accessed. Similar initiatives to improve the availability and utility of facility performance data have taken root at the state level. For example, the Environmental Compliance Con-

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

sortium, a voluntary collaboration among state environmental agencies, uses publicly available performance data to identify strong programs and showcase them as national models.

The availability of such data on facility environmental releases makes it possible to compare the environmental performance of facilities that do and do not participate in specific policy interventions and to explore other determinants of firms’ environmental performance, such as management commitment to environmental excellence or participation in EPA’s National Environmental Performance Track, a program for “top” environmental performers. Understanding the effects of such organizational variables can help explain variation among facilities and suggest more effective policy approaches.

Private industry has recently been developing analytic tools for evaluating the performance of its own policies and programs. For example, in the late 1990s the International Organization for Standardization published ISO 14030, the environmental performance evaluation standard, a tool managers can use to compare their actual environmental performance to what they intended. Many individual firms devote substantial resources to evaluating their own environmental performance and that of their peers (see, e.g., http://www.intel.com/intel/other/ehs/perform.htm). Research could help strengthen these efforts by documenting and comparing the aspects of performance companies are evaluating.

Much more work still needs to be done to develop reliable and comprehensive outcome metrics, as noted in the discussion of environmental indicators above. For example, available data still tell little about the impact of policies geared to improving land use management, natural resource utilization, or pollutant emissions from nonpoint sources. They do not summarize consumption of energy, water, or other environmental inputs. Non-manufacturing firms may not be required to report releases to government at all, even though their impacts may be significant. Improved data in such areas would strengthen researchers’ capabilities to do sound evaluations.

Environmental policy evaluation can make a difference in policy by distinguishing promising policy approaches from those that are unlikely to lead to substantial advances. It can lead to adjustments in policy emphasis to achieve desired objectives, and improve decision makers’ ability to create, compare, and implement more attractive policy alternatives.

Such research has a strong likelihood of being used because of increased emphasis on results-oriented government, especially in the federal government. Following the Government Performance and Results Act of 1993, a Performance Assessment Rating Tool (PART) was developed by the Office of Management and Budget “to systematically and routinely assess program performance” (Johnson, 2003a). Using PART, agencies are specifically asked whether they regularly conduct independent evaluations

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

of their programs (Johnson, 2003b). In August 2001, President George W. Bush proclaimed results-oriented government as one of three overriding principles (U.S. Office of Management and Budget, 2002). Evaluation is central to the implementation of a results-oriented approach to government.

High-level, nonpartisan groups, such as Enterprise for the Environment and the National Academy of Public Administration, have called for results-based and information-driven environmental policy (Ruckelshaus and Hausker, 1998; National Academy of Public Administration, 1995, 1997). EPA’s strategic plan for 2003-2008 lists focusing on results as a primary objective and its innovation strategy calls on environmental agencies to “emphasize results more than the means to achieve them, using regulatory and non-regulatory tools” (U.S. Environmental Protection Agency, 2002b). EPA has pledged to “evaluat[e] innovations results to make strategic decisions about those that can and should be applied on a broader scale” (U.S. Environmental Protection Agency, 2002b). This strategy acknowledges the critical role evaluation plays in determining which innovations are ready to be scaled up to full-blown programs, which require refinement, and which are unworkable.

Notwithstanding the demand for evaluation, it will be important for research to explore ways to design program evaluations so as to ensure they will be used. The barriers to use are fairly well documented (Solomon, 1998; Kraft, 1998). When evaluators do not understand the political contexts of the programs they study, they are apt to produce results that are not relevant to policy makers’ questions. Evaluators may present results in complex and technical language or may fail to disseminate their findings widely. The structure of environmental agencies—organized by environmental media, staffed by regulatory experts, and reliant on outside contractors—may make them particularly resistant to utilizing evaluations. These barriers to use are not insurmountable, however.

In summary, environmental program evaluation is critical for improving the effectiveness of public and private environmental policies, and the need for measures of policy results has been recognized at the highest levels of the federal government. In some areas, particularly pollutant emissions from the manufacturing and utility sectors, useful data are now available on compliance and environmental releases. Evaluation is particularly salient in the context of widespread claims that traditional regulatory approaches to environmental protection are reaching the limits of their effectiveness. Many innovative approaches at the federal, state, and local levels have now been in use long enough to have established substantial track records. An ambitious agenda of environmental policy evaluation research could help fill the need for better understanding of what has worked and what has not and can thus provide better decision-relevant information for policy uses.

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

IMPROVING ENVIRONMENTAL FORECASTING

Federal environmental agencies should undertake an assortment of research initiatives to collect, appraise, develop, and extend analytic activities related to forecasting in order to improve environmental understanding and decision making.

Forecasting refers to a diverse collection of tools, methods, and practical approaches all generally intended to clarify past trends by their extension into the future, as well as to reveal and explore possibilities by postulating likely and desired changes in the present and projecting their consequences. Both forecasting and prediction aim to the future, creating a potential confusion between the terms. Forecasts may be assessed for their accuracy as predictions, but they should also be appraised and valued for their ability to reveal and discover phenomena, relationships, and the implications of existing trends. None of these important purposes operates in the same way when prediction is the goal (Brewer, 1992).

Many reasons exist to motivate forecasting activities. Among them are the desires to increase the available lead time until decisions must be made to allow more careful analysis of various options and associated outcomes and to increase the chance for broad public participation in decision making. Short lead times make for poorly considered decisions and limit potential participation (Anderson, 1997). To illuminate and secure the common interest and achieve the widest possible shaping and sharing of human values, time is needed to define and analyze problems, synthesize and communicate diverse and complex information, and weigh the legitimate interests of numerous stakeholders and participants. Forecasting, in its most general sense, has the potential to contribute significantly to all of these objectives.

The Research Need

Environmental forecasting, like the creation of indicators, has often suffered from inadequate appreciation of the relevant social science. For example, forecasting tools are often developed without taking into account the diverse needs of forecast users for information depending on their decision situations. Researchers often “do all the science” first, usually in search of some point prediction of likely events, and then “add on” the human dimensions almost as an afterthought. Consequently, the resulting forecasts often fail to connect to the needs of decision makers (Sarewitz and Pielke, 1999).

In addition, forecasts are often treated mechanistically. Climate change forecasting, for example, has predominantly been based on models of how greenhouse gases, once emitted, propagate through the global environment

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

and bring about changes in global mean temperature, glacier thickness, and other physical variables. A mechanistic approach is appropriate so long as the forecasting task is limited to physical processes governed by universal laws. It has the advantage of being readily defensible as rational and objective (Ascher, 1987). Because the predictions are thought to be objective, controversy over policy is believed to be confined to settling differences in normative judgments (Friedman, 1953).

This approach has serious limitations when used to inform the decisions of human beings engaged in realistic policy decision making (Hammond, 1996). Clear testimony of its limits comes in the consistently poor predictive accuracy of forecasts using complex, computer-based economic and behavioral models (Ascher, 1978; Greenberger, Brewer, Hogan, and Russell, 1983; Craig, Gadgil, and Coomey, 2002; also see Appendix E). Although such models often have heuristic value, there has been no improvement since the early 1950s in the predictive accuracy of forecasts based on such models, which perform no better than judgmental forecasts. Complex models are less satisfactory than judgmental or scenario-based forecasts in terms of transparency, because they are commonly fitted to data by adjusting model assumptions and specifications. Among the beneficial roles of simple models is their capacity to explore alternative assumptions in a relatively straightforward and transparent manner—tasks quite different from simply predicting.

Forecasting efforts could be improved by focusing from the start on the human setting of environmental decision making, which should be the starting point and the framework within which past trends and possible futures are forecast. Forecasting efforts should encompass human influences on the environment as well as biophysical processes. They should be directed at decision-relevant outcomes, including environmental, health, and socioeconomic outcomes and the distribution of these outcomes across segments of the population. As with other aspects of environmental measurement, the development of forecasting methods should be guided by input from the potential users.

A human-centered approach stresses the concepts of intentionality and choice. Because people can invent and alter the future (sometimes influenced by forecasts), a simple “null projection” of past trends provides only a first-order approximation of future events or problems—one possible future out of many. The idea of “null” conveys the sense of what might happen if nothing changes—the status quo conditions of rules and relationships thrust into the future. Numerous plausible decisions need to be considered and analyzed to see which among them offer advantages compared with the null projection in terms of particular sets of values (Bobbitt, 2003; Hawken et al., 1982).

Forecasting, like the development of environmental indicators, presents

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

substantial challenges in terms of acknowledging and consulting a diversity of information sources. These include applying the appropriate analytic tools to assemble, organize, integrate, and synthesize the available information into a useful whole and the need to present the results in meaningful and intelligible forms that can be readily and reliably communicated to potential users.

Environmental forecasting can benefit by drawing on experience with related kinds of forecasting (Armstrong, 2001). For example, models, simulations, games, and other closely related tools and methods have long been commonplace in the national security realm, but there has been little transfer or crossover of these tools to environmental forecasting. Efforts to conduct integrated environmental assessments are somewhat encouraging (Brewer, 1986), although comparisons of them to comparable military systems analyses show ample room for improvement. Among other things, because environmental assessments and forecasts typically offer a single-minded vision of the future, likely changes and plausible surprises and the consequent need for forecasting to be creative and adaptive are given short shrift. What is needed are multiple scenarios based on the perspectives and decision needs of many different participants and stakeholders that allow the exploration of many possible futures and conditions. The so-called Total Systems Performance Assessment (TSPA) for the proposed nuclear waste repository at Yucca Mountain, Nevada, is illustrative of the problem and this need (U.S. Nuclear Waste Technical Review Board, 1995).

Sophisticated private-sector environmental forecasting, such as in the energy area, may have limited social or public utility for proprietary reasons (Schwartz, 1996). The experiences of Stanford University’s Energy Modeling Forum (an activity funded in large part by the Electric Power Research Institute) provide one strong example of positive developments from the private sector that can offer insights for public-sector environmental forecasting (Weyant et al., 1996).

Research Areas

Identifying Best Practices

We recommend a wide-ranging stock taking and appraisal of forecasting tools, methods, and experiences from a variety of different fields and subject matters to identify general best practices and to highlight common problems—the former to serve as exemplars and the latter to be avoided. Such an effort would go a long way toward enriching and improving forecasting in general practice and forecasts in the specific environmental realm. Taking stock and then setting and enforcing standards are desirable outcomes. Creating courses and curricula to emphasize best practices will

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

contribute to improvements with longer term and longer lasting effects. As essential as forecast methods and tools are for integration and synthesis, it is remarkable to realize that they are not regularly taught in graduate educational or professional training programs.

Environmental Modeling Forums

To open access and to make forecasting more transparent and thus credible, we recommend sponsorship of one or a few continuing environmental modeling forums patterned on the long-running and successful Energy Modeling Forum at Stanford University. Providing ongoing and continuing appraisal is a desirable outcome here. The longerterm effects might include increased numbers of qualified professionals and improved understanding of environmental forecasts by government officials and the publics they serve. Enlarging the circle of those who understand the strengths and limitations of environmental models and analyses may also help to raise the credibility of forecasts and forecasters from their current low levels.

Improving Characterization of Uncertainty

At the heart of any forecast are difficult matters related to uncertainties. Uncertainties appear in the relevance and reliability of data, in the appropriateness of theories used to structure analyses, in the completeness of the specification of the problem, and in the “fit” between a forecast and the social and political matters of fact on the ground. Moreover, the characterization of uncertainty should consider the decision relevance of different aspects of the uncertainties. Failure to appreciate such uncertainties results in poor decisions, misinterpretation of forecasts, and to diminished trust of analysts among the potential users of forecasts. Considerable past work on uncertainty in environmental assessments and models make this topic ripe for progress (e.g., Morgan and Henrion, 1990; Rotman and van Asselt, 2001; National Research Council, 1997a). Improved ways to describe uncertainties in forecasts would provide widespread benefits.

Rationale for the Activity

The demand for environmental forecasts from public policy makers is evident in their continued reference to expected futures as providing the rationale for policy. The question is whether this field is ready to make significant scientific progress. The modest research efforts recommended here are highly leveraged because of past efforts in other problem areas and policy arenas and are likely to yield cost-effective results in terms of establishing best practices and standards for including and highlighting essential

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

human aspects to be addressed in forecasting efforts. New technical and professional courses of study are an expected outcome that would increase the number of competent environmental analysts.

Another useful product could well be independent appraisal and certification of environmental forecasts, analyses, and analysts. Such are likely goals for the recommended Environmental Modeling Forums. Certification processes are likely to create incentives for improved clarity and transparency of forecast models and, in time, improve the low levels of trust and confidence the public generally has for environmental forecasts and those who use them.

DETERMINING DISTRIBUTIONAL IMPACTS

Federal agencies should support concerted efforts to improve the data, methods, and analytical techniques for determining the distributional impacts of environmental policies and programs related to issues of environmental inequities and their abatement. These efforts should include the determination of the most appropriate levels of social, spatial, and temporal aggregation of measurement for environmental monitoring and indicator development.

The Research Need

Concern about the distributional impacts of environmental risks has a long history in the United States and in EPA. Continuing concerns and controversy about claims of environmental inequity and injustice underline the importance of developing an adequate evidence base for addressing the issues.

Empirical research on these issues has been fragmented, inconclusive, and inconsistent in its results. The pioneering empirical work on distributional impacts in the 1970s focused on pollution in cities (Kruvant, 1974; Berry, 1977). Later, interest was driven by claims of environmental injustices based on documentation of the disproportionate burden of toxic waste on minority communities (U.S. General Accounting Office, 1983; United Church of Christ, 1987). Much of the recent literature on environmental inequity and injustice consists of activism and advocacy, analysis of the legal and civil rights aspects of environmental justice, and theoretical discussions of the meaning of equity (Bowen, 2001, 2002; English, 2004; Liu, 2001; Szasz and Meuser, 1997).

In recent years, there has been a marked increase in the number of methodologically based studies, especially those employing spatial analytical techniques (Stockwell, Sorenson, Eckert, and Carreras, 1993; Chakraborty and Armstrong, 1997; McMaster, Leitner, and Sheppard,

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

1997; Cutter et al., 2002; Mennis, 2002; Pine, Marx, and Lakshmanan, 2002) or historical demographic methods for measuring the evolution of inequities (Oakes, Anderton, and Anderson, 1996; Yandle and Burton, 1996; Been and Gupta, 1997; Mitchell, Thomas, and Cutter, 1999). Nevertheless, the measurement and modeling of environmental inequality and its causes are still in their infancy. There are also fundamental questions regarding the appropriate social and geographic scales for analyzing claims of inequity (Greenberg, 1993; Zimmerman, 1994; Cutter, Holm, and Clark, 1996) and the relationship of environmental justice issues to other issues of public policy decision making (Bowen and Wells, 2002; Sexton and Adgate, 1999; Margai, 2001; Miranda, Dolinoy, and Overstreet, 2002; Sexton, Waller, McMaster, Maldonado, and Adgate, 2002). Despite considerable research and policy interest during the past 20 years, fundamental questions remain concerning how to determine that environmental justice problems exist and, once determined, how to abate them. Four research themes seem most promising for addressing such questions.

Areas of Research

Defining Key Variables

Executive Order 12898 requires each federal agency to “make achieving environmental justice part of its mission,” defines that mission with reference to adverse effects on “minority populations and low-incom ity can make a significant contribution to defining and measuring these concepts, taking into account changes in definitions over time and across space (e.g., the contextual nature of the terms).

Social scientists can also contribute to a deeper understanding of environmental justice in other ways. They can analyze whether the most significant adverse impacts are best identified by analyzing only residential location, as is often done, or by also considering occupational categories, prior health condition, or combinations of these and other risk factors. They can also help understand cultural, socioeconomic, or other systemic differences in what people or communities see as unjust and the conditions under which individuals and communities judge that an environmental injustice has been done.

Analyzing the Dependence of Impacts on Spatial and Temporal Scale

Associations between human activities and environmental conditions are well known to appear different as a function of the scale of measurement (e.g., Wilbanks and Kates, 1999; Geist and Lambin, 2002; Association of American Geographers Global Change in Local Places Working

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

Group, 2003). The implications of this general finding for environmental justice issues remain unknown. At present, data and analytical methods are inadequate to examine equity problems and impacts at multiple scales (e.g., individuals to ecosystems to regions). The existing research base is restricted geographically (focused on a few individual cities with census tracts as the enumeration unit) and temporally, usually providing only a static view of current distributions of impacts. There is little comparative work between urban places or between cities, suburbs, and rural areas. In urban settings, different settlement histories in northeastern versus southern or western cities raise questions about whether it is more appropriate to look for distributional impacts in central cities or in larger scale entities such as metropolitan areas or watersheds. Shifts in impacts over time have not yet been the subject of much investigation, yet they are extremely important for forecasting the future outcomes of environmental policies and programs and addressing issues of generational equity.

Developing Integrated Biophysical and Social Models

Models relevant to assessing differential exposures to and impacts of environmental risks are being developed by specialists in various fields—environmental, natural, and health sciences, social sciences, and engineering sciences—without much dialogue among researchers or their models. A sustained effort to model environmental impacts in an integrated way, focusing on their distribution, can help instigate this dialogue and lead to significant scientific advances. Integrated models should include (1) multiple stressors (cumulative or simultaneous); (2) multiple pathways of exposure (air, water, land); and (3) social vulnerability metrics. Consequently, efforts to build the models will engage a broad cross-section of researchers with critical issues in developing indicators, improving the quantity and quality of georeferenced data, and considering issues of scale dependence.

Improving Visualization and Risk Communication

Many distributional impacts and equity considerations are inherently geographical and readily displayed using maps. However, more interactive and sophisticated approaches to visualization, for example, using animations and virtual reality, may be very useful in this application arena because they may be accessible to a wider range of nonexpert audiences and they may enhance decision makers’ understanding of the differential impact of risks and the variability in impacts from their management. The use of advanced geographic decision support tools may also result in a better informed next generation of American citizens. Examples of such tools now in use include the EPA’s Surf Your Watershed (see http://www.epa.gov/surf/)

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×

and Environmental Defense’s Scorecard see http://www.scorecard.org). Such tools need evaluation, of course, in terms of their ability to provide accurate information about the distribution of environmental impacts, their value in generating ideas about how to reduce inequities, and their accessibility to the various populations concerned with environmental justice issues.

Rationale for the Activity

A concerted effort to measure the distribution of environmental impacts would advance both social science and environmental decision making. The research community is now mature enough to move from the rhetoric of environmental justice to the scientific analysis of the underlying phenomena. The proposed research activities can significantly advance understanding by scientists and the public of the dimensions and underlying causes of situations judged as inequitable through developing improved methods of measuring and monitoring them and models for understanding them. This would represent a significant advance over the evidence base for policy at present, which often consists of perceptions of environmental inequities among stakeholder groups rather than any robust empirically based assessment of exposures and impacts.

The recommended research would produce results that are potentially useful to policy makers at various levels of government and to citizens concerned with environmental justice issues. Moreover, there appear to be ready users. An EPA advisory panel has recommended that the agency make environmental justice a core part of its policies and expand its risk framework for measuring the cumulative impacts of toxic chemicals on disadvantaged communities (Risk Policy Report, March 16, 2004:8). This research would provide a tool kit to environmental agencies for examining the impacts of federal and state policies on particular localities or affected groups and considering whether or not the policies are achieving their desired results. The tools developed could help monitor progress toward environmental goals, such as embodied in Executive Order 12898, and ultimately reduce the unanticipated consequences of environmental decision making at the federal and state levels.

Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 85
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 86
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 87
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 88
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 89
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 90
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 91
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 92
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 93
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 94
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 95
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 96
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 97
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 98
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 99
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 100
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 101
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 102
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 103
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 104
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 105
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 106
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 107
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 108
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 109
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 110
Suggested Citation:"6 Decision-Relevant Science for Evidence-Based Environmental Policy." National Research Council. 2005. Decision Making for the Environment: Social and Behavioral Science Research Priorities. Washington, DC: The National Academies Press. doi: 10.17226/11186.
×
Page 111
Next: References »
Decision Making for the Environment: Social and Behavioral Science Research Priorities Get This Book
×
Buy Paperback | $61.00 Buy Ebook | $48.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

With the growing number, complexity, and importance of environmental problems come demands to include a full range of intellectual disciplines and scholarly traditions to help define and eventually manage such problems more effectively. Decision Making for the Environment: Social and Behavioral Science Research Priorities is the result of a 2-year effort by 12 social and behavioral scientists, scholars, and practitioners. The report sets research priorities for the social and behavioral sciences as they relate to several different kinds of environmental problems.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!