Executive Summary
EPA's Environmental Monitoring and Assessment Program (EMAP) was established to provide a comprehensive report card on the condition of the nation's ecological resources and to detect trends in the condition of those resources. At EPA's request, the National Research Council's Board on Environmental Studies and Toxicology and Water Science and Technology Board established the Committee to Review EPA's Environmental Monitoring and Assessment Program. This fourth and final report is the committee's overall evaluation of the program.
In 1988, the Science Advisory Board of the U.S. Environmental Protection Agency recommended that EPA ''undertake research on techniques that can be used to help anticipate environmental problems," and that "an office be created within EPA for the purpose of evaluating environmental trends and assessing other predictors of potential environmental problems before they become acute".
Following the Science Advisory Board's advice, EPA established the Environmental Monitoring and Assessment Program (EMAP) "to monitor ecological status and trends, as well as to develop innovative methods to anticipate emerging environmental problems before they reach crisis proportions". In 1993 EMAP's stated goals were to:
- Estimate the current status, trends, and changes in selected indicators of condition of the nation's ecological resources on a regional basis with known confidence.
- Estimate the geographic coverage and extent of the nation's ecological resources with known confidence.
- Seek associations between selected indicators of natural and human stresses and indicators of the condition of ecological resources.
- Provide annual statistical summaries and periodic assessments of the nation's ecological resources.
As described by EPA, EMAP is unified by its approach to landscape characterization, the application of a coherent strategy for the choice and the development of indicators, and the use of a probability-based sampling approach that uses a hexagonal grid for identifying sampling sites. There are eight resource groups identified by the program: agroecosystems, arid (now rangeland) ecosystems, forests, the Great Lakes, estuaries, inland surface waters, wetlands (recently subsumed under surface waters and the Great Lakes), and landscape ecology. These resource groups are intended to represent ecosystem types or resources of national interest, and to provide a basis for incorporating ecological knowledge into the design of indicators and sampling programs.
The committee's reviews of other EMAP components such as forests and estuaries and surface waters were published as separate reports. The executive summaries of these reports are in Chapter 4.
After four-years of review, the committee retains its belief that EMAP's goals are laudable. However, because achieving the goals of this ambitious program will require that EMAP successfully meet many difficult scientific, practical, and management challenges, the committee continues to question whether and how well all these goals can be achieved. This final report reiterates that general assessment.
As first conceived and presented to the committee in 1991, EMAP was significantly different than it is today. Several of its central features and components seem to have less importance in mid-1994 than they did in 1991. The reverse is also true: the resource groups have become much more important and are leading the program. One of the major strengths of EMAP as initially presented was that it planned to integrate information across regions and across resource types, but the nature and extent of that integration is still not clear.
Given the need for 10 years or more of data to sample regions and distinguish trends, nobody—including the members of this committee—can be certain whether, or how fully, EMAP will achieve its stated goals. This is to be expected for a large, ambitious, and novel program like EMAP. However, the program-wide concerns expressed in the committee's previous reports, in Chapter 2 of this report, and summarized below, are so important that EMAP will have little chance of achieving its goals if they are not addressed. Concerns revolve around the following issues.
- The EMAP sampling program may operate at too coarse a scale in space and time to detect meaningful changes in the condition of ecological resources.
- EMAP's success will be diminished if it does not develop reliable, scientifically defensible indicators for measuring change. The development of indicators of ecological health or integrity and of aesthetic quality appear to be particularly challenging.
- EMAP's success will be diminished if it does not select the right assessment end points (i.e., the end effect that is the goal of the monitoring program), something it has not done so far.
- EMAP's success will be diminished if the retrospective or prospective monitoring approach does not match the assessment needs and the needs of policy makers.
- EMAP needs to incorporate the best scientific advice in the design, implementation, and review of its program.
- EMAP has not yet fulfilled its promise of innovation and national comprehensiveness because the programs to integrate
- information across space, time and resource types have not been developed. The most important of these are an indicator development strategy, information management, and landscape characterization.
- EMAP's information management system must support efficient access to a large, distributed database and application of an appropriate range of information processing tools.
- Lack of continuity in staffing at EMAP has inhibited development of the program. EMAP cannot succeed unless the government (i.e., the administration and the Congress) makes a sufficient financial commitment to EMAP to support administrative and technical excellence, continuity, and efficiency in program management. That commitment is necessary for EMAP to succeed, but is not sufficient by itself.
A September 1994 letter from EMAP director Edward Martinko (Appendix A) describes EMAP's recent responses to earlier NRC reports and provides additional updates about the program. Many of the changes described appear to be in line with the earlier committee recommendations. EMAP has not provided more detailed documentation of these encouraging changes, so this report has not been substantially altered. However, recommendations in this report that deal with matters directly addressed by Dr. Martinko's letter are indicated with an asterisk.
Recommendations
Statistics, Sampling, and Design
- EMAP should consider design changes that would increase the probability of detecting smaller scale ecological changes. Some possibilities include increasing revisitation rates at a subset of sample sites; inclusion of a set of nonrandomly selected sentinel-sites with intensive data-collection, such as the Long
- Term Ecological Research (LTER)/Land Margin Ecosystems Research (LMER) networks; and stratified random sampling by ecoregion with data-quality objectives specified for strata. If EMAP does not adopt these design changes itself, then it should become extremely closely and explicitly coordinated with a program that has these features.
- EMAP should consider further combining effects-oriented and stressor-oriented monitoring approaches. Predictive, or stressor-oriented, monitoring seeks to detect the cause of an undesirable effect (a stressor) before the effect occurs or becomes serious. Retrospective, or effects-oriented, monitoring seeks to detect the effect after it has occurred. EMAP has relied mostly on the latter. Stressor-oriented monitoring will increase the probability of detecting meaningful ecological changes. As in the above point, if EMAP does not adopt these changes, it should become closely coordinated with a program that monitors in this way.
- EMAP should undertake power analyses regarding the effectiveness of the sampling design for each resource group.* A power analysis is an analysis of the statistical strength of an approach to detect change if a change exists. Different resource groups have adopted different sampling approaches. All the resource groups should adopt the practice of the EMAP lakes component, which has assigned teams of statisticians to assess the effectiveness of EMAP for that particular resource.
- EMAP should reconsider its detection criterion of a 20 percent change over 10 years. In some systems, such a large change is unlikely to occur in nature, while in other systems, a much smaller change would elicit concern. EMAP should also consider systems or indicators for which a change in the variance, rather than mean or median, is important.
Indicators
- EMAP should initiate a major, focused research program on indicator development.* Indicator development is at the heart of the EMAP program. Without a well-considered set of indicators for each resource group, EMAP will not fulfill its goal of presenting an evaluation of the nation's ecological resources. The difficulty and importance of indicator development requires EPA to attract the highest quality researchers in the environmental sciences to this program. The program should include a combination of internal research (by EMAP scientists) and external research involving open announcements of funding availability with peer reviewed grants.
- Each EMAP resource group should develop one or more mechanistic conceptual models of its resource, based on current scientific knowledge. * These models should serve as explicit hypotheses about those aspects of ecosystem structure and functioning relevant to the assessment end points the group has chosen. The models must be detailed enough to include potential indicators, assessment end points, and key variables.
- EMAP should provide program-wide guidance for numerous evaluation issues if the indicator selection strategy is going to yield the nationally applicable set of indicators EMAP envisions. The committee recommends as a high priority the explicit and early evaluation of the statistical properties of all potential indicators. Such evaluations should include analyses of the properties of the mean, variance, and behavior of the index in power tests. If this cannot be done analytically, then simulation analyses should be performed.
- EMAP should carefully evaluate each potential indicator at incrementally larger spatial scales. EMAP needs to make sure that it has information on the domains of usefulness of its indicators—at what spatial and temporal scales are they reliable, and at what scales are they less reliable? The ways in which the various resource groups deal with this problem will have important consequences for the selection of nationally implemented
- indicator metrics. Program-wide strategies for dealing with this issue should be developed now, in time to be applied with some uniformity across the resource groups.
Integration
- EMAP should develop key integration and assessment questions that cross-resources. This would help focus the program and significantly extend its value nationwide.
- EMAP should designate resources for integration. As EMAP now stands, there are relatively few financial or other resources directed specifically at integration. Such resources could be directed in various ways, but several important needs must be met. Individual resource programs directed at integration must have access to the information management system, and must have computer and software resources to generate and test generalizations. One approach would support a team of individuals who focus on developing and addressing the integration and assessment questions, and who either work together at one physical location or were coordinated among resource groups by a central office. Key members of this group must be participants of the Landscape Characterization, Landscape Ecology, and Indicator Development groups. The new Integration and Assessment Program is a positive step in this direction, but we do not have a good description of the activities of this program.
- EMAP should develop coordinated sampling between terrestrial, aquatic, and atmospheric resources.1 Resources
- appearing to have very important ecological connections due to hydrologic linkages are now being sampled in separate locations. The design would be enhanced by a cooperative sampling scheme between resource groups in which lakes and streams were sampled in watersheds whose terrestrial systems (forests, agroecosystems, arid systems) also were being sampled. A stratified random system such as this would not compromise EMAP's ability to make regional scale generalizations based on probability-based samples. The data sets would be considerably stronger because the spatial covariance of the data sets could be used to test hypotheses related to cause and effect relationships.
Possible examples include indicators reflecting net primary productivity, biological diversity, and aesthetic value. At present, it is unclear whether or not the assessment questions in each resource group are similar enough to lead to parallel sets of indicators. Such symmetry among resource groups, while not essential to basic EMAP objectives, would greatly enhance the scientific and analytical value of the data collected.
Appropriate Scale and Boundaries of Regions
- EMAP should choose ecologically meaningful units as the primary scale for summarizing and reporting data. Ecologically meaningful units, such as Bailey's or Omernik's ecoregions, should be the primary objects of statistical analysis and data reporting rather than political units or EPA regions. In general,
|
member feels strongly that such inter- and indeed intra-agency interactions are essential for effective coordination of monitoring and assessment efforts involving the atmospheric transport, transformations, and deposition (as well as associated intermedia transport) of a wide range of harzardous gaseous and particulate air pollutants. |
- EMAP should reconsider the scale and boundaries of units for which the national program summarizes and reports data.
Coordination And Management
- 1994) is an excellent example of such coordination. The committee encourages further efforts with programs like the National Water Quality Assessment of the U.S. Geological Survey.
External Scientific Review
- The current external review structure of EMAP should be modified so that its core is a permanent panel, with rotating membership, to provide continuity. A permanent board of accomplished scientists may provide more expertise and consistency of viewpoint than EMAP has had access to heretofore. The panel should advise both at the level of resource groups, such as the forests or estuary resource level, and at the level of the entire EMAP program.
Information Management
- While top-down planning for EMAP's information system is important, EMAP should base such planning on the viewpoint that the information system is a scientific database system, rather than an information system focused on the needs of management if the Information Management System is to function and facilitate integration among research groups as envisioned by EMAP. In particular, the planning should focus on the design of an environment that is sensitive to user requirements and that provides excellent hardware, software, and support personnel.