Strengthening Science at the U.S. Environmental Protection Agency: Research-Management and Peer-Review Practices (2000)

IN the three decades since the U.S. Environmental Protection Agency (EPA) was created, great progress has been achieved in cleaning up the nation's worst and most obvious environmental pollution problems. Belching smokestacks and raw-sewage discharges are now scarce, and air pollution alerts and beach closings are more rare. EPA deserves a significant share of the credit for the accomplishments, but some of the most difficult and challenging tasks remain. Many past illusions about simple and easy solutions to environmental problems have been replaced by greater realization that environmental protection is a complicated and challenging mission.

Today, scientific knowledge and technical information are more important than ever for understanding and successfully addressing the increasingly complex environmental problems facing the nation. In the 1970s, environmental protection efforts and the associated demands for scientific knowledge were largely and appropriately focused on the manufacturing and transportation sectors and the problems associated with environmental releases at the sources. Today, there is a greater recognition that this is a limited view of the environmental problems challenging public health and the environment. Such problems can arise from environmental releases during all stages of a prod-

uct's life cycle, including manufacturing, distribution, use, and disposal. Further, the problems are not associated solely with the pollutants released directly, but are often the result of complex reactions and interactions occurring in the environment, such as those associated with the formation of ozone in the troposphere or the formation and bioaccumulation of methyl mercury. Such problems can only be addressed through an understanding of the complex interrelationships among environmental media (air, water, land, and biota), human health and ecology, and economic sectors.

Along with the growing need for scientific knowledge and technical information to understand these complex factors are the rapidly occurring scientific advances in fields as diverse as molecular biology, chemistry, medicine, information technology, and the social sciences. These advances and the knowledge and technology they create hold the key to our future ability to identify and understand the environmental problems that pose the greatest risks to human health, environmental quality, natural resources, the economy, and our quality of life. Such advances in scientific knowledge and technological capability are also critical in the development of strategies for reducing environmental risks. In addition, advances in the social and behavioral sciences, including qualitative-analysis, risk-communication, and stakeholder-engagement techniques, are increasingly recognized as critical components of risk-reduction strategies.

Scientific knowledge and technical information are also needed to set environmental priorities. In the absence of sound scientific information, high-risk problems might not be adequately addressed, while high-profile but lower-risk problems might be targeted wastefully. When scientific knowledge is unavailable or overlooked, regulations and policies may fail to address serious environmental problems or unnecessarily seek to overprotect every person or every ecosystem against hazards that are minor or that few will actually experience. This can carry serious implications for public health and the environment or impose a heavy burden on society and the economy without providing appreciably better protection for most people or ecosystems.

Scientific knowledge is also needed to help identify and prepare for emerging and future environmental problems, including problems not envisioned or addressed by current statutes and government programs. If scientists can identify emerging or future environmental

trends and their consequences, early steps can be designed to avoid or reduce the risks posed by those trends, thereby avoiding the much larger costs of addressing problems after they have grown to serious proportions.

As stated in Safeguarding the Future: Credible Science, Credible Decisions (EPA 1992),

[S]cience is one of the soundest investments the nation can make for the future. Strong science provides the foundation for credible environmental decision making. With a better understanding of environmental risks to people and ecosystems, EPA can target the hazards that pose the greatest risks, anticipate environmental problems before they reach a critical level, and develop strategies that use the nation's, and the world's, environmental protection dollars wisely.

EPA was created in 1970 by presidential executive order, not by legislation. Although a principal justification for creating the agency was the need for a unified environmental program for the nation (CEQ 1970), the programs and capabilities of EPA over the years have derived from a disparate collection of federal statutes that use varying approaches to address separate environmental problem areas but lack a unified approach to protect the environment and public health.

Although no formal overall mission has ever been enacted for EPA, the agency has published a strategic plan (EPA 1997b), developed in response to the 1993 Government Performance and Results Act. It states,

The mission of the U.S. Environmental Protection Agency is to protect human health and to safeguard the natural environment – air, water, and land – upon which life depends.

Thus, EPA does not primarily have a “science” mission in the same sense that the National Institutes of Health (NIH) or the National Science Foundation (NSF) have primary missions to advance scientific and technical knowledge through research. Traditionally, EPA has mainly been a “command-and-control” regulatory agency. Alternatives to regulatory action in some areas have been explored, but the agency's first priority has been to implement and enforce the statutes,

mostly regulatory, under which it operates. Science has been an important part of the basis for many agency decisions and actions, but it has not been the only basis, and in many cases it has not even been a major determinant of EPA's decisions. EPA's past decisions and actions have largely been driven by the requirements of regulatory statutes and the policies and priorities of each administration.

Nevertheless, EPA's strategic plan (EPA 1997b) states that one of the agency's seven overall purposes is to ensure that “National efforts to reduce environmental risk are based on the best available scientific information.” In addition, one of the agency 's 10 major goals, also stated in the strategic plan, is the following:

Sound Science, Improved Understanding of Environmental Risk, and Greater Innovation to Address Environmental Problems: EPA will develop and apply the best available science for addressing current and future environmental hazards, as well as new approaches toward improving environmental protection.

EPA's strategic plan also states,

Science enables us to identify the most important sources of risk to human health and the environment, and by so doing, informs our priority-setting, ensures credibility for our policies, and guides our deployment of resources. It gives us the understanding and technologies we need to detect, abate, and avoid environmental problems. This goal recognizes that science provides the crucial underpinning for EPA decisions and challenges us to apply the best available science and technical analysis to our environmental problems and to practice more integrated, more efficient, and more effective approaches to reducing environmental risks.

The agency promised (EPA 1997b) the following future scientific accomplishments over the next decade:

EPA's research program will measurably increase our understanding of environmental processes and our capability to respond to and solve environmental problems. During the past decade, significant concerns have been expressed about the adequacy of the Agency's ability to assess risks – not only to human health, but also to ecosystems. Research will

lead to greater certainty in assessing and comparing environmental risks. Our aim is to reduce major areas of uncertainty in our analyses of risk and to minimize reliance on default assumptions. In order to accomplish this, we will develop improved exposure assessments that identify environmental exposures posing the greatest environmental risks to the American public and will increasingly use biologically-based methodologies. We will demonstrate improved knowledge of current ecosystem conditions and the most critical stressors affecting these conditions, as well as deliver improved capabilities to interpret what these conditions imply in terms of immediate and future risks. This will provide strengthened capability to determine the condition of the environment and its responses to alternative management strategies at local, regional, and national scales. This will also lead to better technologies to manage and restore ecosystems.

We will also build institutional capacity to forecast and prepare for emerging problems. To prevent damage to both human and ecosystem health, it is critical to detect, describe, evaluate, and mitigate or eliminate stressors before damage occurs. We plan to improve capacity and technology to monitor and model stressors and effects. We plan to encourage the rapid acceptance and implementation of improved environmental technology by assessing and verifying the performance characteristics of commercially ready technologies and by making those assessments available for consideration by a variety of potential technology users. This will help provide proven, cost-effective technologies and approaches to prevent or manage environmental problems.

The Agency plans to strengthen the science base of the Regions by increasing their capacity to monitor and measure environmental conditions. We also plan to strengthen our overall quality of science by significantly enhancing peer review in the Agency and by seeking guidance from the Science Advisory Board, leading to more defensible environmental decisions.

Since scientific quality and cost-effectiveness are generally increased through collegial interaction, the Agency plans to increase its “partnering” with other Agencies and organizations, especially in joint efforts through the National Science and Technology Council, and in more frequent collaboration with NASA, NSF, and DOE. Similar synergistic benefits are sought through joint participation in the peer review of Agency documents and positions by advisory committees from different departments and agencies.

The following report examines EPA practices and the likelihood of achieving these worthy goals.

In the 30 years since EPA was created, the agency's scientific performance has been assessed many times in reports from the National Research Council (NRC), the EPA Science Advisory Board (SAB), the General Accounting Office (GAO), and many other organizations; in congressional oversight and judicial proceedings; and in countless crit icisms and lawsuits from stakeholders with interests in particular EPA regulatory decisions.

As early as 1974, concerns about EPA's scientific mission, research organization, and research planning process were expressed by an NRC committee (NRC 1974). Three years later, a set of important recommendations that eventually helped define some of the principal features of the agency's current research program were made in the NRC report Analytical Studies for the U.S. Environmental Protection Agency, Volume III: Research and Development in the Environmental Protection Agency (NRC 1977). Among the recommendations, the 1977 report stated that all EPA research should be centralized in the agency's Office of Research and Development (ORD); that ORD should conduct a mixture of fundamental, anticipatory, and regulatory support research; that an integrated risk-assessment office should be created in ORD; and that an extramural program of research grants and centers should be established. Although those particular recommendations were soon adopted by EPA, other recommendations of the 1977 report were not followed immediately but were adopted nearly 2 decades later (see Chapter 2). For instance, the 1977 report called for a clear definition of EPA's research mission, the development of a strategic plan for environmental research, and nonmanagerial career advancement paths for EPA research scientists.

EPA's own SAB has constructively criticized the agency's scientific performance and recommended changes in the agency's research program many times over the years. Especially important among the SAB reports of recent years were Future Risk: Research Strategies for the 1990s (EPASAB 1988) and Reducing Risk: Setting Priorities and Strategies for

Environmental Protection (EPASAB 1990). In the 1988 report, the SAB emphasized the need for EPA and its research program to shift emphasis from the traditional command-and-control and cleanup strategies to the anticipation and prevention of pollution problems. Among its recommendations, the 1988 report called for ORD to expand its long-term research program, emphasizing core research areas in which EPA has special capabilities and responsibilities. It also recommended that ORD place greater emphasis on anticipatory studies and monitoring, understanding human exposure to pollutants, and epidemiological research. The 1988 report also urged EPA to increase its efforts in public education, technology transfer, and education of environmental scientists. Reducing Risk: Setting Priorities and Strategies for Environmental Protection (EPASAB 1990), stressed the need for EPA and its research program to become more proactive. It argued that EPA and its research program, given limited resources, should move beyond the agency' s longstanding practice of fragmented regulatory program responses driven by individual statutory mandates, and move toward a cost-effective focus on the greatest health and environmental risks and the greatest opportunities for reducing those risks. Among its recommendations, the SAB's 1990 report also urged ORD to place greater emphasis on risks to ecosystems and on the development of better risk-assessment methods and data.

In Safeguarding the Future: Credible Science, Credible Decisions (EPA 1992), a panel of senior academicians, including two members of this NRC committee, concluded, “Currently, EPA science is of uneven quality, and the Agency's policies and regulations are frequently perceived as lacking a strong scientific foundation.” While acknowledging that EPA had a number of knowledgeable scientists on its staff, the panel reported that the science base at EPA was not perceived to be strong by the university community, and that many EPA scientists at all levels throughout the agency believed that EPA did not use their scientific knowledge and resources effectively.

The 1992 panel observed, “A perception exists that regulations based on unsound science have led to unneeded economic and social burdens, and that unsound science has sometimes led to decisions that expose people and ecosystems to avoidable risks.” The panel commented that EPA had not always ensured that contrasting, reputable scientific views were well-explored and well-documented from the

beginning to the end of the regulatory process. It pointed out that the agency was often perceived to have a conflict of interest because it needed science to support its regulatory activities, and the panel described a widely held perception by people both outside and inside the agency that EPA science was “adjusted” by EPA scientists or decision-makers, consciously or unconsciously, to fit policy.

The NRC's Board on Environmental Studies and Toxicology, which oversaw the study that produced this report, has issued many previous reports urging improvements in specific parts of EPA's research program and scientific performance. Examples of such reports are Rethinking the Ozone Problem in Urban and Regional Air Pollution (NRC 1991); Science and Judgment in Risk Assessment (NRC 1994c); Review of EPA's Environmental Monitoring and Assessment Program (NRC 1994a,b, 1995a); Research Priorities for Airborne Particulate Matter (NRC 1998a, 1999c); and Modeling Mobile-Source Emissions (NRC 2000).

In the fiscal year 1995 appropriations report for EPA, Congress directed the agency to obtain an independent assessment from the National Academy of Sciences regarding the overall structure and management of EPA's research program and an evaluation of scientific peer-review procedures used by the agency.

This report is the fourth prepared by two companion expert committees convened by the NRC, the principal operating arm of the National Academy of Sciences and the National Academy of Engineering, in response to that congressional request and to subsequent, related requests from EPA. To carry out the study, the NRC appointed the Committee on Research and Peer Review in EPA, which prepared an interim report (NRC 1995b) addressing the initial request from Congress, according to a short deadline set by Congress, as well as this report, the final report in the study. As part of the study, the NRC also appointed a companion committee – the Committee on Research Opportunities and Priorities for EPA – which issued an interim report in 1996 and the report Building a Foundation for Sound Environmental Decisions in 1997 (NRC 1997). This fourth and final report expands on issues discussed in the previous reports and addresses related questions.

The members of both committees were chosen by the NRC for their expertise in biology, chemistry, statistics, chemical engineering, environmental engineering, atmospheric sciences, toxicology, exposure assessment, public health, ecology, soil science, and other disciplines. Special emphasis was placed on selecting committee members with research-management experience and knowledge of the research and other scientific activities of EPA and other agencies. The chairman and two other members of the Committee on Research Opportunities and Priorities for EPA were also members of the Committee on Research and Peer Review in EPA.

The Committee on Research and Peer Review in EPA was charged to assess EPA's overall research-program structure, peer-review procedures, long-term research program, laboratory site-review procedures, and research-staff career-development and performance-evaluation procedures. It was asked to consider problems, issues, and recommendations contained in previous evaluations of EPA's research program and peer-review practices, as well as other relevant problems and issues that the committee might identify. In framing its approach to its task, the committee was asked to place particular emphasis on the aspect of its charge regarding previous evaluations of scientific practices and performance in EPA. Previous evaluations by committees of the NRC, EPA's SAB, and other independent groups are featured prominently in our committee's reports. Many of their findings and recommendations are cited and reviewed in some detail, and we have tried to build on their foundation and to indicate clearly where we did or did not agree with them.

In addition, our committee's approach to its task was strongly influenced by two important developments that occurred during the course of our study. First, in 1996 EPA 's ORD requested and obtained under the Federal Advisory Committee Act a charter for a new body, the Board of Scientific Counselors (BOSC), to advise EPA's Assistant Administrator for Research and Development. The BOSC is not part of the SAB but is organized and operates in a similar manner. Established with 15 senior expert members from universities and other organizations serving on an ongoing basis, including one member of our NRC committee, and augmented with temporary members appointed to serve on ad hoc subcommittees as needed, including additional members of our NRC committee, the BOSC is charged to evaluate the

management and operations of ORD's research programs and peer-review practices. The BOSC conducts site reviews of ORD's laboratories and centers and evaluations of research staff in ORD, and it has already issued its first round of reviews and evaluations. In Chapter 2 of this report, our committee discusses the results of the BOSC's reviews and evaluations and, for the most part, concludes that we concur with them and that the BOSC should continue to conduct site reviews and staff evaluations. Although additional findings and recommendations of our committee are included in Chapter 2, we mainly decided to endorse and defer to the BOSC on this aspect of the committee's task.

The second important development that affected our committee's approach to its task was the creation in 1996, at EPA's request, of the NRC's Committee on Research Opportunities and Priorities for EPA, our companion committee in this study, as stated above. That committee was charged to provide an overview of significant emerging environmental issues, identify and prioritize research themes most relevant to understanding and resolving those issues, and consider the role of EPA's research program in the context of research being conducted or sponsored by other organizations. In its 1997 report, Building a Foundation for Sound Environmental Decisions, our companion committee assessed and re-defined key aspects of the goals and structure of EPA's research program. It replaced previously used terms like “long-term” and “short-term” research, and “basic” and “applied” research, with the concepts of “core” and “problem-driven” research, as discussed in Chapter 2. As stated in that chapter, our committee fully concurs with our companion committee on these issues, so the component of our charge dealing with “long-term” research was re-oriented to be compatible with the new terms and concepts.

During the course of study leading to the preparation of this report, the Committee on Research and Peer Review in EPA drew on the expertise and experience of its members, considered more than 300 relevant documents obtained from EPA and other sources, and consulted with more than 200 scientists, engineers, managers, and other persons within and outside EPA to obtain relevant information and insights on research-program structure, planning, funding, and management; organizational matters; and scientific career development, performance evaluation, recruitment, and morale issues. The committee held seven 2-day plenary meetings, six of them at National Academies facilities in

Washington, DC; Woods Hole, MA; and Irvine, CA; and one meeting at the EPA laboratory facilities in Research Triangle Park, NC. In addition, teams of committee members and staff made one or more site visits to interview agency scientists, managers, and other staff at each of the following EPA laboratories, centers, headquarters offices, and regional offices: National Health and Environmental Effects Research Laboratory, Cincinnati, OH; Corvallis, OR; and Research Triangle Park, NC; National Exposure Research Laboratory, Cincinnati, OH; Las Vegas, NV; and Research Triangle Park, NC; National Risk Management Research Laboratory, Cincinnati, OH; and Research Triangle Park, NC; National Center for Environmental Assessment, Cincinnati, OH; Research Triangle Park, NC; and Washington, DC; National Center for Environmental Research, Washington, DC; Office of the Administrator, Washington, DC; Office of Air and Radiation, Research Triangle Park, NC; and Washington, DC; Office of Policy, Planning and Evaluation, Washington, DC; Office of Prevention, Pesticides, and Toxic Substances, Washington, DC; Office of Research and Development, Washington, DC; Office of Solid Waste and Emergency Response, Washington, DC; Office of Water, Washington, DC; and regional EPA offices in Boston, MA; Chicago, IL; Denver, CO; New York, NY; and San Francisco, CA. At those locations, committee members and staff interviewed a cross section of EPA personnel, including senior officials, middle managers, staff scientists and engineers, and support staff. At each location, the site-visit team posed a prepared list of questions pertaining to the committee's charge. Following the site visits, the responses to the questions were discussed by the committee. During the course of the study, committee members and staff also interviewed officials knowledgeable about EPA from Congress, the GAO, NSF, NIH, the Office of Management and Budget (OMB), and the Office of Science and Technology Policy (OSTP). In addition, some members of the committee have previously served on one or more groups that independently evaluated the research programs of EPA and other federal agencies under the auspices of the NRC, the Carnegie Commission, EPA's SAB, ORD's BOSC, or other organizations.

Shortly before our committee began its work, an EPA agency-wide committee was in the process of internally evaluating the agency's research program. In July 1994, that committee completed its report to the EPA administrator, Research, Development, and Technical Services at

EPA: A New Beginning (EPA 1994b). The report recommended major changes in the ORD program, and the agency began implementing them in 1995 (see Chapter 2).

Soon afterward, our committee's interim report (NRC 1995b) offered a preliminary endorsement of the general scope and direction of the changes then being made in ORD, as well as other steps being taken to strengthen agency-wide peer-review practices. A full assessment was not possible at that time, because the changes were a work in progress and needed time to take root. Now, the committee judges the 1995 reorganization of ORD and the agency's new peer-review practices to be essentially in place and, although still a work in progress, sufficiently mature to be assessed in this report.