National Academies Press: OpenBook

Models in Environmental Regulatory Decision Making (2007)

Chapter: 2 Model Use in the Environmental Regulatory Decision Process

« Previous: 1 Study Background
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

2
Model Use in the Environmental Regulatory Decision Process

Regulatory model use at EPA can be contentious. Decisions based on model results might have important public health or environmental consequences and impose substantial costs. Like other aspects of regulation, models are used and evaluated within an environment of legislative requirements, regulatory review, extensive comment by interest groups and other federal agencies, and legal challenge. Within this environment, the development, maintenance, and use of models diverge in important ways from research modeling in academia or nonregulatory modeling in the public and private sectors.

In spite of the challenges, the use of computational models within the regulatory decision process at EPA is a continually growing practice. This growth is in response to greater demands for quantitative assessment of regulatory activities, including analysis of how well environmental regulatory activities fulfill their objectives and at what cost. Models are essential for estimating a variety of relevant characteristics—including pollutant emissions, ambient conditions, and dose—when direct observation would be inaccessible, infeasible, or unethical. Finally, models allow regulators to move away from technology-based regulations that do not use quantitative analysis for assessing their benefits. This chapter describes the diversity of model use at EPA and the current integration of models into its regulatory policies. It highlights how EPA regulatory model use is influenced by legislative mandates and executive orders as well as oversight from the courts and outside participants.

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

REGULATING WITHOUT COMPUTATIONAL MODELS

Although models are essential tools if regulators are to be able to predict the risks or the effects of their regulations on the natural and human environment, models are neither necessary nor sufficient to produce the regulations themselves. In the 1970s, when the legislative framework underlying most of today’s environmental policy was first established, few sophisticated computational environmental models—models designed to predict the environmental consequences of human activity—existed. Moreover, the monitoring networks capable of quantitative description of the state of the environment were rudimentary, and the technology for measurement of pollutant discharges of various kinds and their environmental effects were much less developed than today’s technology. It was in this setting that most modern environmental regulatory statutes first appeared, including the Clean Air Act (CAA) of 1967, the Federal Water Pollution Control Act of 1972 and renamed the Clean Water Act (CWA) in 1977, and the Safe Drinking Water Act of 1976. Regulatory designs at the time necessarily minimized the use of computational models in the regulatory process.

The models that did exist played little role in that process because the new environmental statutes emphasized the use of technology-based pollution discharge regulation. Technology-based regulation requires polluters to adopt a particular technology (or, in some cases, achieve a level of performance associated with a particular technology) without regard to the potential or actual environmental improvements that would result.

Even before the implementation of the federal environmental statutes, technology-based regulation partly relied on there being some level of pollution abatement practiced by at least some plants in most industries. EPA was to find those plants and set a performance standard for all plants that was based in some way on what most plants were doing. Usually the congressional mandate involved the use of the words “best technology,” and it was left to EPA to interpret and give operational meaning to the various designations of “best.” For example, industrial water pollutant dischargers had to meet “best practicable treatment” (BPT) technology standards by 1977 and “best available treatment economically achievable” (BATEA or, more often, BAT) standards by 1983. In industries such as food processing and laundries that generated wastewater that resembled domestic waste (in constituents if not in strength), the usual interpretation of BPT was a performance standard that approximated what good secondary (biological) treatment could do. For other

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

industries, BPT was often defined as the “average of the best” plants in the industry identified as having wastewater treatment in place. For BAT, the standard was the “best of the best,” at least until the CWA Amendments of 1977, which redirected BAT toward the control of toxic pollutants in wastewater.

It should be noted that technology-based standards are not the only policy instrument that makes it possible to regulate without having to predict the environmental effects of environmental regulations. Indeed, the need to predict the consequences of regulations depends not on the policy instrument but on the policy goal. If the goal is to achieve a level of emissions reductions rather than environmental quality, there is no need to inquire into environmental effects, regardless of policy instrument. Other environmental policies proposed in the early 1970s shared that property, including several proposals using economic incentives.1 Like technology-based standards, none of these proposals had an environmental objective beyond the notion that a reduction in effluent discharges would be an improvement and that policies could be fine-tuned later, when scientists had collected more data and achieved a better understanding of environmental processes.

Although the CAA and CWA of the 1970s (as well as other environmental statutes) made extensive use of technology-based standards, it would be misleading to leave the impression that their regulatory arsenals were not limited to such standards. Both statutes also had explicit environmental goals, measurement criteria for determining when the goals were met, and timetables for meeting them. For example, the National Ambient Air Quality Standards (NAAQS) in the 1970 CAA focus on reducing air pollutant concentrations to levels that are protective of human health and public welfare. This legislation required states to develop state implementation plans (SIPs), which are subject to EPA approval. Such approval was contingent on whether the plans, when implemented, would reduce emissions enough to allow the ambient standards to be met. EPA would come to base these SIP approval decisions on emission-inventory models linked to air quality models. In a similar manner, the CWA specified further regulatory action in “water-quality-limited” waters, where the imposition of the technology-based

1

In 1970, a tax on sulfur emissions as a partial alternative to some of the air quality regulation then under consideration in Congress was proposed by President Nixon. In November 1971, an effluent-charge amendment to clean water legislation then under consideration was offered and debated in the Senate (Kelman 1982; Kneese and Schultze 1975).

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

standards was considered insufficient to achieve water quality standards. Eventually, that section of the CWA gave rise to the “total maximum daily load” program.

Technology-based regulation proved to be a crude approach to pollution abatement policy. Moreover, it did not ultimately relieve Congress and EPA of the need for models to assess whether abatement policies were sufficient to achieve ambient goals. However, at a time when few models were available for linking pollution abatement to environmental improvement, technology-based standards provided a basis for regulating pollutant discharges that did not require knowledge of what the effects of such regulation would be. Today technology-based regulations are still in use, primarily in circumstances in which data and models do not yet permit an adequate assessment of the effects of regulation on environmental or health end points and in which other approaches have failed to generate regulations (these two situations overlap substantially). For example, Title III of the 1990 CAA Amendments changed the primary focus of hazardous air pollutant (HAP) regulation from a risk-based approach to a two-step process, where the primary focus has been on a technology-based approach to mandate promulgation of emissions standards for sources based to some extent on maximum achievable control technologies (MACT), followed by a residual risk assessment. In the preceding regime, regulators made little progress in producing regulations, largely because the inadequacies of data and models linking emissions of HAPs to adverse health effects. The current approach directs EPA to develop a MACT standard for each industrial source category, defined in part by high emissions of listed pollutants. Since 1993, EPA has promulgated over 100 MACT regulations (for the list, see EPA 2006b). After a MACT has been applied, EPA is to perform a residual risk assessment to evaluate the adequacy of the MACT, which might require additional controls if significant risks still exist.

REGULATORY MODEL CLASSIFICATIONS

There are many ways to classify the regulatory models used by EPA, each with its own perspectives and particular advantages and disadvantages. Two broad categorizations are used here: (1) a functional perspective that categorizes models based on their representation of scientific and other processes that translate human activities and natural systems interactions into environmental impacts and (2) a regulatory perspective that categorizes models based on how they are used in environ-

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

mental regulation. In short, we see these as attempting to represent how an environmental scientist, engineer, or economist might see model use and how a regulator or stakeholder might see model use. In presenting a science perspective and a regulatory perspective, the committee acknowledges that the user community for environmental regulatory models is diverse, and a single perspective on model classification is not possible. More perspectives provide insights into model use, insights that are not possible from a single perspective. Looking at the functions of models as representing different environmental and human processes helps to emphasize the role of individual models and the need to integrate across multiple models for many regulatory activities. Looking at models from the perspective of their role in a complex regulatory setting helps to make clear the role of legislation and regulation in determining modeling objectives and the separate modeling responsibilities for EPA, state, and local governments.

Given the wide range of model applications and large number of models used in environmental regulation, the committee does not attempt to present an inventory of models used by EPA. The most exhaustive inventory with descriptions of individual models is EPA’s Council on Regulatory Environmental Models (CREM) (EPA 2006c), although many other web sites are devoted to describing various programs’ modeling initiatives (see Table 2-1). CREM’s knowledge-base documents more than 100 models used by various offices at the agency. It is the single best, although incomplete, inventory of models at EPA. The information available on each model includes user information on obtaining and running the model and model documentation, including conceptual basis, scientific details, and results of evaluation studies. A full review of the knowledge base has recently been completed by EPA Science Advisory Board and is beyond the committee’s charge (EPA 2006d). However, we note that additions to the CREM knowledge base have ceased since 2004, with the exception of several climate change models that were added in 2006. For the knowledge base to reach its full capability, it needs to be updated continually and to include all types of models used at EPA, including those in the health risk assessment field.

Regulatory Models from a Functional Perspective

In this section, we discuss models categorized according to how they fit into a description of the processes that translate human activities

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

and natural systems interactions into environmental impacts. Figure 2-1 shows an illustration of the pathways from activities to emissions to impacts. In the figure, individual components simulate the relationships between human activities and emissions, emissions and concentrations, concentrations and exposures, and exposures and impacts. It also indicates the feedback of impacts on human activities and natural processes. Appendix C provides examples of specific models from the model categories. The figure provides an approximate categorization of how computational models used in environmental analysis have historically been grouped, in particular, in economic, environmental, and human health models. This perspective allows for the identification of particular types of models and the linkages among these models. Each box is highly aggregated and could be expanded into a diagram of sub-boxes. An example of how this aggregate representation might be represented in more detail will be discussed with respect to human health risk assessment in a later section.

The categories of models that are integral to environmental regulation include activity models, natural and anthropogenic emissions models, fate and transport models, exposure models, dose models, human health models, environmental and ecosystem impact models, and economic impact models. Although the categories of models shown in

TABLE 2-1 Examples of EPA’s Web Sites Containing Model Descriptions for Individual Programs

National Exposure Research Laboratory Models Web Site

http://www.epa.gov/nerl/topics/models.html

Atmospheric Sciences Modeling Division Web Site

http://www.epa.gov/asmdnerl/index.html

Office of Water’s Water Quality Modeling Web Site

http://www.epa.gov/waterscience/wqm

Center for Subsurface Modeling Support Web Site

http://www.epa.gov/ada/csmos.html

National Center for Environmental Assessment’s Risk Assessment Web Site

http://cfpub.epa.gov/ncea/cfm/nceariskassess.cfm?ActType=RiskAssess

National Center for Computational Toxicology Web Site

http://www.epa.gov/ncct

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

FIGURE 2-1 Basic modeling elements relating human activities and natural systems to environmental impacts.

Figure 2-1 are not specific to environmental media, the models that fit into each category tend to be further subdivided by media. For example, the generic category of environmental fate and transport models can be subdivided further into various types of subsurface containment transport models, surface-water quality models, and air quality models (Schnoor 1996; Ramasawami et al. 2005).

Scope of Regulatory Model Applications

Table 2-2 contains short descriptions of some of EPA’s regulatory activities that rely on modeling. These environmental regulatory modeling activities typically occur as a subset of the full system summarized in Figure 2-1. The underlying statutory requirements, the regulations implementing the statutory requirements, and the importance of the activity dictate the nature of the modeling analysis. For example, assessing the toxicity of new pesticides and other chemicals in the environment may focus on just the fate and transport or toxicity portion of the system. Assessing the risks from leaking underground petroleum storage tanks,

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

TABLE 2-2 Examples of Major EPA Documents That Incorporate a Substantial Amount of Computational Modeling Activities

Air Quality

Criteria Documents and Staff Papers for Establishing NAAQS

Summarize and assess exposures and health impacts for the criteria air pollutants (ozone, particulate matter, carbon monoxide, lead, nitrogen dioxide, and sulfur dioxide). Criteria documents include results from exposure and health modeling studies, focusing on describing exposure-response relationships. For example, the particulate matter criteria document placed emphasis on epidemiological models of morbidity and mortality (EPA 2004c). The Staff Paper takes this scientific foundation a step further by identifying the crucial health information and using exposure modeling to characterize risks that serve as the basis for the staff recommendation of the standards to the EPA Administrator. For example, models of the number of children exercising outdoors during those parts of the day when ozone is elevated had a major influence on decisions about the 8-hour ozone national ambient air quality standard (EPA 1996).

State Implementation Plan (SIP) Amendments

A detailed description of the scientific methods and emissions reduction programs a state will use to carry out its responsibilities under the CAA for complying with NAAQS. A SIP typically relies on results from activity, emissions, and air quality modeling. Model-generated emissions inventories serve as input to regional air quality models and are used to test alternative emission-reduction schemes to see whether they will result in air quality standards being met (e.g., ADEC 2001; TCEQ 2004). Regional scale modeling has become an integral part of developing state implementation plans for new 8-hour ozone and fine particulate matter standards. States, local governments, and their consultants do this analysis.

Regulatory Impact Assessments for Air Quality Rules

RIAs for air quality regulations document the costs and benefits of major emission-control regulations. Recent RIAs have included emissions, air quality, exposure, and health and economic impacts modeling results (e.g., EPA 2004b). See Box 2-3 for a further discussion of the RIA.

Water Regulations

Total Maximum Daily Load (TMDL) Determinations

For each impaired water body, a TMDL documents a state-designated water quality standard need to meet a designated use for that water body and the amount by which pollutant loads need to be reduced to meet the standard. TMDLs utilize water quality and/or nutrient loading models. States and their consultants do the majority of this modeling, with EPA occasionally doing the modeling for particularly contentious TMDLs (EPA 2002b; George 2004; Shoemaker 2004; Wool 2004).

Leaking Underground Storage Tank Program

Assesses the potential risks associated with leaking underground gasoline storage tanks. At an initial screening level, it may assess only one-dimensional transport of a conservative contaminant using an analytical model (Weaver 2004).

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

Development of Maximum Contaminant Level for Drinking Water

Assess drinking water standards for public water supply systems. Such assessments can include exposure, epidemiology, and dose-response modeling. (EPA 2002c; NRC 2001b, 2005b).

Pesticides and Toxic Substances Programs

Pre-manufacturing Notice Decisions

Assess risks associated with new manufactured chemicals entering the market. Most chemicals are screened initially as to their environmental and human health risks using structure-activity relationship models.

Pesticide Reassessments

Requires that all existing pesticides undergo a reassessment based on cumulative (from multiple pesticides) and aggregate (exposure from multiple pathways) health risk. This includes the use of pesticide exposure models.

Solid and Hazardous Wastes Regulations

Superfund Site Decision Documents

Includes the remedial investigation, proposed plan, and record of decision documents that detail the characteristics and cleanup of Superfund sites. For many hazardous waste sites, a primary modeling task is utilizing groundwater modeling to assess the movement of toxic substances through the substrate (Burden 2004). The remedial investigation for a mining megasite might include water quality, environmental chemistry, human health risk, and ecological risk assessment modeling (NRC 2005a).

Human Health Risk Assessment

Benchmark Dose (BMD) Technical Guidance Document

EPA relies on both laboratory animal and epidemiologic studies in assessing the noncancer effects of chronic exposure to pollutants (that is, the reference dose [RfD] and the inhalation reference concentration, [RfC]). These data are modeled to estimate the human dose-response. EPA recommends the use of BMD modeling, which essentially fits the experimental data to use as much as the available data as possible (EPA 2000).

Guidelines for Carcinogen Risk Assessment

The cancer guidelines set forth a revised set of recommended principles and procedures to guide EPA scientists and others in assessing the cancer risks resulting from exposure to chemicals or other agents in the environment. One of the principal advancements was to describe approaches that consider mode-of-action data, if available, in the quantitative assessment. The guidelines are also used to inform agency decision makers and the public about risk assessment procedures (EPA 2005a).

Ecological Risk Assessment

Guidelines for Ecological Risk Assessment

The ecological risk assessment guidelines provide general principles and give examples to show how ecological risk assessment can be applied to a wide range of systems, stressors, and biological, spatial, and temporal scales. They describe the strengths and limitations of alternative approaches and emphasize processes and approaches for analyzing data rather than specifying data collection techniques, methods, or models (EPA 1998).

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

especially during initial assessments, focuses solely on the fate-transport component. The SIP process, which involves extensive emissions and air quality modeling, stops at simulating atmospheric concentrations of air pollutants. Ideally, regulations would be informed by understanding the whole of the paradigm, shown in Figure 2-1, from human activities through adverse outcomes. However, only the most important regulatory assessments, such as some of those done for federal rules that have major economic impacts, include a simulation of processes from activity to health impacts. These are the rules that generate most of the benefits and costs of environmental regulation, and the modeling effort can be enormous. A recent example of such an analysis is the regulatory impact assessment (RIA) for the control of air pollutant emissions from nonroad diesel engines (EPA 2004b). Even the extensive modeling that accompanied this rule cannot quantitatively consider all aspects of the problem. For example, in discussing behavioral responses to increasing costs for nonroad diesel engines, stakeholders suggested that equipment users may substitute different equipment (gasoline engines) or even labor (the use of a laborer and shovel instead of a backhoe) for more expensive diesel engines (EPA 2004b). Such behavioral aspects were only discussed qualitatively in the report. Incorporating behavior into environmental regulatory models is discussed more generally in Box 2-1.

Linkages among the different processes are not seamless. Each category often is represented by a separate model and regulatory analyses often require that inputs and outputs of one model interface with other models in separate categories. Sometimes temporal or spatial scales do not line up and results from one model may not have natural counterparts in the models with which it interfaces. An example is from the air quality analysis in which emissions from vehicles and other sources that are estimated at the regional level must be allocated spatially and estimates of aggregated hydrocarbon emissions must be disaggregated by species for input into the air quality model. More fundamentally, the linking of these different categories means the linking of separate disciplines. To properly link different modeling categories requires the building of interdisciplinary bridges, which is an ongoing effort at EPA. Although there are software tools and integrated models that allow multiple processes to be combined into a single modeling framework as discussed in a subsequent section, such a model still faces the difficulty of needing to rely on the expertise from multiple disciplines.

The level of effort dedicated to environmental regulatory applications varies greatly. This variation is a critical consideration when

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

BOX 2-1

Incorporating Human Behavior into Environmental Models

For regulatory purposes it is important to not only model natural systems but also human activities and their interactions with natural systems. These interactions, which can always be found at either end of the causal chain, shown in Figure 2-1, and often in the middle as well, require models from the social sciences, usually economics. A key modeling consideration is the extent to which such models incorporate human behavior. The earliest models used for environmental regulatory purposes had little if any behavioral content. The effects of both regulations and environmental changes were estimated without considering the full range of responses available to economic agents—individuals, households, and firms. One of the first models to demonstrate that possible behavioral responses could affect the costs or effectiveness of regulations was developed by Gruenspecht (1982), who pointed out that the common regulatory practice of requiring more stringent and more costly abatement for new sources of pollutants than for existing sources could retard the turnover of existing equipment. Behavioral responses are sensitive to the details of regulatory design, and numerous models appeared in the economics literature describing the unintended consequences of such real-world policies as CAFE (Kwoka 1983) and vehicle inspection and maintenance (Hubbard 1998). Behavioral responses also affect other outcomes of interest to EPA, including regulatory enforcement (Harrington 1989), pollution abatement subsidies (Freeman 1978; Rubin 1985). Behavioral responses to adverse environmental consequences, such as private defensive expenditures, have also been analyzed.

For many years, EPA made frequent use of behavioral models for policy analysis and regulatory impact analysis. In cases involving economic incentives, behavioral models are essential because the behavioral response is what drives the policy outcome. For example, analysis of proposed emissions cap-and-trade policies to control airborne sulfur dioxide emissions from the electric power industry requires the agency to predict the behavior of utilities in the permit market. For this task, EPA uses the integrated planning model, a proprietary dynamic linear programming model that determines the least-cost loading of generating capacity to meet electricity demand. The optimization simulates the expected outcome in the permit market.

Not all of EPA’s regulatory models that could incorporate behavioral responses to regulation do. For example, the MOBILE model, which projects average regional or national motor vehicle-emission rates under a variety of regulatory design parameters, does not consider the effects that regulatory alternatives might have on fleet composition or vehicle use through their effects on vehicle or fuel prices. MOBILE’s failure to anticipate behavioral responses to regulation has been most noticeable in the motor vehicle emissions inspection and maintenance program (I/M) component, which has underestimated the ability of motorists to avoid I/M tests altogether and overestimated the ability of those tests to identify high-emitting vehicles as well as the effectiveness of vehicle repair (e.g., NRC 2001a; Holmes and Cicerone 2002, and references therein).

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

developing recommendations, later in the report, related to model development, evaluation, and application. At one end of the spectrum are applications that involve a small investment in resources and modeling effort. Leaking underground petroleum storage tanks number in the hundreds of thousands, and preliminary screening for EPA’s leaking underground storage tank program typically relies on the application of an analytical model with assumed parameters (Weaver 2004). These state-run programs may spend as little as $500 for site assessments. The new chemicals program under the Toxic Substances Control Act (TSCA) requires EPA to review approximately 2,000 new chemicals per year and issue decisions on up to 20-30 chemicals per day (C. Fehrenbacher personal commun., EPA Office of Pollution Prevention and Toxics, February 23, 2006). Because of these demands, the agency relies on the quantitative structure-activity relationships (QSARs) model that uses basic knowledge of a chemical’s structure to predict physical and chemical properties and environmental fate and transport when data are not available. At the other end of the spectrum, EPA may spend years or even a decade assessing the health and environmental consequences of other environmental pollutants, making their modeling efforts extremely complex. Under the CAA, EPA is required to review NAAQS every 5 years. This requires major investments of resources and may take many years of assembling background information and performing analyses, including modeling analyses. Somewhere between these two extremes are the water quality management TMDL and the air quality management SIP analyses. EPA estimates 3,000-4,000 TMDLs, with a wide array of resource requirements, will be needed annually for the next 8 to 13 years to meet current deadlines (NRC 2001c). While some TMDLs require extensive data collection and modeling, at least one state has proposed using a nonmodeling approach for catchments with little or no data (George 2004). The SIP process can be a major undertaking requiring development of emissions inventories and analysis of control options. Each local area out of attainment must submit a plan for each pollutant. For example, there are currently 116 counties out of attainment with the current 24-hour PM2.52 standard (Bachman 2006).

2

PM2.5 refers to a subset of particulate matter collected by a sampling device with a size-selective inlet that has a 50% collection efficiency for particles with an aerodynamic diameter of 2.5 µm.

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Evolution of Regulatory Modeling

The elements that are included in modeling may change over time for a given type of assessment, typically adding complexity to the modeling process. This is a result of changes to regulatory requirements, scientific understandings, and modeling capabilities. A potential example is in the health risk assessment paradigm. Fundamentally, a health risk assessment developed today is conceptually consistent with what is discussed in the NRC “Red Book” (Risk Assessment in the Federal Government: Managing the Process, NRC 1983) and laid out in Figure 2-2. A major modeling component is the development of dose-response relationships through analysis of epidemiological or toxicological studies (Setzer 2005). The NRC reports on toxicological effects of arsenic from drinking water provide a prime example of many of the issues associated with developing dose-response modeling for a contaminant (NRC 1999a, 2001b). Box 2-2 described this case study in more detail.

FIGURE 2-2 Basic elements of risk assessment from the National Research Council’s Red Book. Source: NRC 1983.

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

BOX 2-2

Risk Assessment for Arsenic in Drinking Water

EPA’s 2001 risk assessment for arsenic in drinking water provides a rich case study that illustrates many of the challenges associated with using models to inform environmental regulation. Establishing a U.S. standard for arsenic in drinking water has been a source of controversy for many decades. From the perspective of environmental regulation, the arsenic story is an interesting one for a number of reasons. First, exposures arise from natural sources and some have even argued that at very low doses it is an essential element for human health. Second, arsenic is not directly carcinogenic in animals; hence, all evidence for human health effects arises from epidemiological studies.

Two National Research Council committees (NRC 1999a, 2001b) convened to advise EPA on this matter suggested that regulation be based on data from 42 villages in southwestern Taiwan, which showed increased rates of bladder and lung cancer as a function of arsenic levels measured in village wells. While it was originally hoped that the arsenic might provide an opportunity for using EPA’s then new guidance on carcinogen risk assessment that allowed the use of biologically based models, the first NRC committee found that there was so much controversy over underlying mechanisms that it was not possible to identify a suitable biologically based model. Instead, the committee recommended reliance on more empirically based statistical models. Although the dose-response modeling was based on human data, which removed the uncertainty associated with extrapolation of results from animals to humans, the inherent variability associated with human data introduced other sources of uncertainty. There were many concerns expressed about the appropriateness of relying on the Taiwanese data for the purpose of setting regulations in the U.S. context. Some cited differences in dietary patterns between the United States and Taiwan, particularly in this relatively poor rural area of Taiwan. Others were concerned that the Taiwanese study used cancer incidence data extracted from population records and exposure crudely assessed based on the median levels of arsenic measured in villages wells. Indeed, Morales et al. (2000) fit the data using a series of relatively simple empirical models that differed according to how age and exposure were incorporated and compared the results obtained from the multistage Weibull model, which had been classically suggested for the analysis of time-to-event data of the type encountered in the Taiwanese data set. As shown in Figure 1-5, these various models differed substantially in their fitted values, especially in the critical low-dose area that is so important for establishing the benchmark dose used to set a reference dose (RfD).

Rule-making was able to move forward, despite the uncertainty, since all the models supported the conclusion that risk levels at the then standard of 50 micrograms per liter (µg/L) were unacceptably high. Based on the first NRC review, EPA lowered the standard for drinking water from 50 µg/L to 10 µg/L in January 2001. This standard was initially delayed so that the EPA Science Advisory Board, the National Drinking Water Advisory Council, and a second NRC committee could further examine benefits, costs, and health risks. These reviews supported the proposed 10 µg/L standard, which was subsequently finalized by EPA.

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

However, research and practice has enabled major changes since 1983 in how the risk components are developed. As the black box between exposure and effect gathered light, improvements in risk assessment practice (Reddy et al. 2005), toxicological testing technologies (NRC 2006b), biomonitoring (NRC 2006a), and understanding of the modes of action, to name a few allow for a more mechanistic modeling approach to relating exposures to health outcomes (see Figure 2-3). For example, analysts understand that even if humans and rats received the same external exposure, they did not receive the same dose of active chemical to the target tissue. To understand these events, data on basic physiological and pharmacokinetic processes and resultant physiologically based pharmacokinetic (PBPK) modeling can integrate specific properties of chemicals with age- and organ-specific physiological processes in different species to relate an effective experimental exposure to an effective environmental exposure (Clewell 2005). PBPK models offer the possibility that doses of chemicals delivered to target cells of a rat could be quantitatively extrapolated to target cells of humans. These models also offer the possibility that differences in age or sensitivity (for example, polymorphisms in metabolism) in the human population could be incorporated in models. However, the use of a PBPK model in a risk assessment can be a time- and cost-intensive undertaking requiring expertise (Clewell 2005). It must be accompanied by a thorough evaluation that includes the following:

  • Evaluation of biological plausibility of model structure and parameters.

  • Verification of model code (equations and logic).

  • Validation of model’s region of applicability.

  • Sensitivity and uncertainty analysis.

These models may also bring with them many technical and science policy challenges. One outcome of more mechanistic approaches to health risk assessment modeling is that regulatory end points might be based on an upstream biochemical precursor event instead of observed adverse health outcomes. The challenge (and controversy) then becomes selecting the appropriate point between an innocuous molecular change and frank disease to use in the assessment. The risk assessment of perchlorate (NRC 2005b) offers an example of how this can be addressed by selecting a nonadverse effect (the inhibition of iodide uptake by the

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

FIGURE 2-3 Elements of advanced mechanistic approaches to health risk assessment. This figure illustrates the fundamental elements of assessment and the models that link the elements. PBPK refers to physiologically based pharmacokinetic models and BBDR refers to biologically based dose-response models. Source: EPA 2003b.

thyroid gland) as a point of departure for adding uncertainty factors. More fundamentally, as understanding increases, so do options and questions about the most appropriate approaches to assess risks. For example, in a particular scenario, judgments may be needed as to whether EPA should give preference to empirical models using human epidemiology or mechanistic rodent-based models (Preuss 2006).

Integrated Models and Modeling Frameworks

Some models tend to fit into a single category, while other regulatory models represent multiple categories of processes, such as modeling emissions and fate and transport together. For example, the integrated planning model produces estimates of electricity sector activity, including fuel demands, prices, and emission-control decisions for given levels of emissions (Napolitano and Lieberman 2004). Models that represent

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

pesticide exposure, such as the CARES, DEEM/Calendex, and LifeLine models, simulate activities that expose humans to pesticides and the residues that different pesticides produce in food and the residential environment to simulate exposure profiles (EPA 2004d).

More recently, the movement toward integration has utilized advances in software to develop modeling frameworks that allow user flexibility to use a combination of compatible models, facilitate multiple simulations, and facilitate output analysis. Examples are CMAQ/Models-3, FRAMES, 3MRA, and BASINS. For example, the Better Assessment Science Integrating Point and Nonpoint Sources (BASINS) is a multipurpose environmental analysis system that integrates a geographical information system, national watershed data, and state-of-the-science environmental assessment and modeling tools into one modeling package (EPA 2006e). The model integrates individual stand-alone models that simulate pollutant loadings from point and nonpoint sources and instream water quality models for performing watershed- and water-quality-based studies. It is intended to make watershed and water quality studies easier by bringing key data and analytical components “under one roof.” A further discussion of improvements in integrated model methods is contained in Chapter 6.

Regulatory Models from a Regulatory Perspective

In this section, we describe the use of models in six phases of the regulatory process. Strategic planning identifies environmental problems of present and future importance and assembles data and constructs modeling tools to permit analysis. Rule-making translates congressional directives into specific regulations. Delegation has states and localities given responsibilities for developing plans to achieve environmental goals locally and writing regulations to achieve those goals. Permitting, licensing, registration is where these rules are applied to govern the behavior of polluting individuals, firms, or other entities. The last two phases are enforcement and ex post facto analysis.

Strategic Planning

The first element in the regulatory sequence above involves the strategic use of models to inform Congress and decision makers within

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

EPA in deciding whether or how to legislate or regulate. “Strategic” implies a thoughtful, informed, and priority-set analysis that identifies goals and major approaches to achieve the goals. Because strategies are inherently predictive, models are crucial. They can inform the identification of goals that are important to achieve (for example, whether a certain air pollutant already regulated is still an important public health risk requiring additional legislation or regulations), and they can characterize approaches to achieving them (for example, whether the predominant source of this air pollutant is stationary, mobile, or personal identifies optimal regulatory targets). Examples include congressional requests to assess alternative legislative proposals for controlling multiple pollutants from power plants (EPA 2001a,b) and EPA’s internal use of modeling to identify the population at risk from ozone exposure that guided decisions on changing the NAAQS for this pollutant (EPA 1996). The use of modeling in strategic planning can become part of the debate between Congress and EPA over environmental policy. An example of this is a May 13, 2004, letter from Congressman Thomas Allen to EPA Administrator Michael Leavitt concerning delays of model runs assessing control options for electric power-plant emissions of mercury (Allen 2004).

One of the broadest uses of models in environmental strategic planning was by the congressionally mandated National Acid Precipitation Assessment Program (NAPAP), which was directed to perform research to inform decisions on regulations of acid rain. The interagency program (EPA and 11 other federal agencies) was funded for 10 years in the 1980s and produced 27 state-of-the-science and -technology reports on all aspects of the acid rain issue. One of the primary products was the air quality models that are precursors to the models used at EPA today. Information developed by NAPAP was useful for changing the understanding of the scientific-information related to acid rain and informing Congress in its development of the parts of the CAA Amendments of 1990 that dealt with acid rain. However, this legislation was enacted before NAPAP completed its integrated assessment report of its activities. This report was intended to synthesize the science for policy makers (NAPAP 1990). In the end, NAPAP was criticized on a number of different levels by both the participants and the observers (Roberts 1991; Rubin 1992; Herrick 2000). Global warming modeling provides a contemporary example of the strategic use of models. The U.S. Climate Change Science Program (CCSP) has developed a strategic plan for attempting to coordinate research, including modeling research, being done by 13 agencies and departments in the government (CCSP 2003; NRC 2004b).

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Rule-making

Rule-making encompasses the tasks of regulatory design and promulgation. The goal of regulatory design is to produce a proposed rule that complies with the legislative requirements set down by Congress and that provides sufficient support and analysis of the rule. EPA’s modeling activities at the rule-making stage can be extensive. For example, the non-road diesel RIA included the use of activity models, emissions models, air quality models, engineering cost models, energy forecasting models, petroleum refinery models, and human health and agricultural impacts models to assess the benefits and costs of the proposed regulation (EPA 2004b). Other rules incorporate less modeling. However, at this point in the regulatory process, EPA is responsible for performing the model analysis, although other stakeholders may submit model analysis and comments on the agency’s modeling analysis during the public comment period. The external review of EPA’s modeling in support of rule-making, including the role of the public comment period and interagency review, is discussed in a later section of this report.

Delegation

Many environmental statutes, including the CAA and CWA, delegate important roles for compliance, which includes implementation and enforcement, to states. States may further delegate some responsibilities to local agencies. Delegation of authority for implementation and enforcement is also given to tribal governments. Modeling analysis is part of the delegated responsibility. The roles of EPA and the state and local agencies vary by the statutes and within statutes. Under the SIP process, states or local governments must prepare a plan for each area that does not meet NAAQS, describing how that area will be brought into attainment. This process includes the modeling analysis described in Table 2-2. For large urban areas, typically a metropolitan planning or other local air quality agency prepares the SIP that must be then approved by the state and eventually by EPA. In the case of the Los Angeles area, it is the South Coast Air Quality Management District that prepares the SIP. For areas with smaller populations, such as the Missoula, Montana, area, the state prepares any SIPs submitted for approval to EPA. For the TMDL program, states are primarily responsible for carrying out the program, including the modeling described in Table 2-2. However, EPA will carry

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

out some TMDL for particularly contentious settings, such as the establishment of a TMDL for limiting mercury in fish tissue residual in the Ochlockonee River Watershed in Georgia (EPA 2002b). Thirty-five states run their own programs for dealing with leaking underground storage tanks, including assessments of subsurface containment transport and risk assessment modeling. As mentioned, tribal governments have the option of running their own environmental programs, and some tribes have received authorization to run air quality and water quality programs. Private consultants often are engaged to perform part of the modeling analysis required under state delegated programs.

State-generated source-specific regulations, required by both SIP and TMDL, are based on the effects of air and water pollutants on environmental quality. This requirement raises a host of technical, economic, and political issues that are sometimes not sufficiently covered in the writing of federal standards. The issues include the following:

  • Interdependence. The environmental effects of emissions from any one source depend on the emissions from numerous other sources.

  • Nonpoint sources. Emissions from sources that are difficult to monitor and regulate at the individual level either because the sources are numerous and diffuse or because the emissions are episodic and dependent on natural processes.

  • Distributional asymmetry. The sources responsible for pollutant discharges are located in a different area from where the environmental damages are suffered. For example, states and cities may have no control over air pollutants that have blown in from afar.

Permitting

Other statutes, such as the Toxic Substances Control Act, Safe Drinking Water Act, and Food Quality Protection Act, require EPA or the state to permit an activity. This activity might be required for the construction and operation of a point emissions source or the introduction and continued use of a chemical in the market. The statues vary in what role modeling plays and which entities perform the modeling. For licensing new pesticides, manufacturers supply a substantial amount of modeling of environmental and human health risks to EPA that might be supplemented by additional agency analysis. For the relicensing of pesticides, which is carried out under the Food Quality Protection Act’s man-

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

date to assess cumulative and aggregate risks, EPA performs the modeling analysis. For the premanufacturing determination that must be made before new chemicals can enter the market under the Toxic Substances Control Act, EPA is responsible for assessing risks. The initial screening is done using structure-activity models, and the results of such modeling determine whether a more thorough assessment is needed and whether manufacturers will be required to submit more test data. Programs that permit discharges into water, controlled under the National Pollution Discharge Elimination System (NPDES), are primarily run by the states, although some states have only partial authority. Although many of the requirements under the NPDES program are still driven by technology-based standards, increasingly state and federal permit writers must take into account water quality standards and watershed considerations, which increases modeling needs. The CAA mandates that the states implement and that EPA oversee permit programs to control and regulate pollutant emissions from major stationary sources. Under these programs, each new major stationary source of air pollutants must apply for a permit before construction and provide modeling to help to demonstrate that the new facility will meet appropriate emission-control standards. Permittee modeling is subsequently reviewed by state regulators.

Compliance and Enforcement

Models are used in compliance and enforcement in several ways. For enforcing some regulations, EPA uses models to estimate the benefit to the regulated party—usually cost savings—from delaying or avoiding pollution control expenditures. For example, the BEN model (see EPA 2007) calculates a violator’s economic savings from delaying or avoiding pollution control expenditures. This estimate is then used as a basis for setting the penalty, which ensures that the violation will not be to the regulated party’s advantage. Other models assess a regulated party’s ability to afford such costs as civil penalties, Superfund cleanup costs, and pollution control expenditures. An example is the MUNIPAY model (EPA 2006f), which evaluates a municipality’s or regional utility’s ability to afford compliance costs, cleanup costs, or civil penalties. EPA may also use models to estimate “natural resource damages” from private actions that damage natural resources. These natural resource damage actions arise out of legislative liability schemes under the CWA, the Oil Pollution Act, and the Comprehensive Environmental Response, Com-

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

pensation and Liability Act. The damage estimates are generally based on contingent valuation surveys, as well as models that attempt to estimate the costs of restoring or replacing the damaged resources.

Ex Post Facto Auditing and Accounting of Impacts

Like strategic planning, assessment of the performance and costs of regulations after they have been implemented is relatively rare within EPA, although it is often carried out by other parties. The Office of Management and Budget (OMB 2005) reviewed recent ex post facto analyses of regulations, including environmental regulations. EPA has also received periodic requests from Congress to report on the aggregate costs and benefits of its regulations. In the past, for example, Congress has required EPA to periodically estimate the total costs of the CAA (under section 812) and CWA (under section 512). One of the main sources of data for these reports is the Pollution Abatement and Control Expenditure (PACE) Survey conducted annually from 1978 to 1994 and conducted in 1999 by the U.S. Census Bureau on a sample of manufacturing establishments.3 In the PACE Survey, individual establishments report total expenditures on pollution abatement, separated by receiving medium (air, water, or land). EPA uses the survey results to estimate expenditures for all manufacturing plants and adds information on expenditures in other sectors to produce the report. These analyses are of limited use for policy assessment because they report only on the aggregate costs of regulation rather than the costs of specific regulations.

Similarly, EPA also occasionally conducts ex post facto studies of benefits. The most prominent example is the ongoing study of the benefits and costs of the CAA—a study required by section 812 of the 1990 CAA. The first major report was a retrospective study of the benefits and costs of the CAA from 1970 to 1990 (EPA 1997a). This report was followed by a prospective study of the benefits and costs of the CAA from 1990 to 2010 (EPA 1999a). A second prospective study is in progress for the period from 2000 to 2020. The retrospective study is best known for its controversial benefit estimate of $6 to $50 trillion in benefits over the period. It illustrates the difficulty of estimating benefits and costs of massive, aggregate programs such as the CAA. All benefit and cost estimates require comparison to a “without-regulation” scenario. For very large changes, determining an appropriate without-regulation scenario be-

3

A new PACE survey is in development at the EPA and may soon resume.

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

comes a matter of achieving consensus rather than analysis and is riddled with uncertainty. Modelers outside of federal agencies also contribute post hoc analysis of environmental regulatory activities. The literature is vast. Some particular examples include the assessments of compliance costs and other impacts of the sulfur dioxide emissions trading programs (e.g., Ellerman et al. 1997; Stavins 1998; Burtraw and Mansur 1999) and the effects of “corporate average fuel efficiency” standards on energy consumption and emissions from motor vehicles (e.g., Kliet 1990; Harrington 1997; Greene et al. 1999; Portney et al. 2003).

CONGRESSIONAL AND EXECUTIVE BRANCH INFLUENCES

There are some particular influences and constraints on the regulatory process resulting from the enabling statutes passed by Congress and from a series of executive orders that over time have given OMB oversight responsibility over regulations and imposed specific requirements on how regulatory decisions are supported through modeling. It is essential to understand these influences.

Congressional Influence

Federal environmental statutes, such as the CAA and CWA, usually contain statements of health and welfare goals, schedules and deadlines for meeting them, and, often, criteria for determining whether the goal has been met. Table 2-3 contains a sample of some of the general and specific directives found in several important environmental statutes. To write regulations to meet these requirements, EPA produces much analysis to justify its decisions and show how its actions meet the congressional directives, which can sometimes require the agency to do the following:

  • Explain quantitatively the magnitudes as well as the spatial and temporal patterns of present and projected contamination.

  • Trace the contaminant back to the human activities that contribute to the contamination and trace the contaminant forward to its health impacts.

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

TABLE 2-3 Examples of Substantive Legislative Directions for EPA Models

General Directions

 

Toxic Substances Control Act, 15 U.S.C. § 2605(a)

Authorizing regulatory action on existing toxic substances “if the administrator finds that there is a reasonable basis to conclude that the manufacture, processing, distribution in commerce, use, or disposal of a chemical substance or mixture, or that any combination of such activities presents or will present an unreasonable risk of injury to health or the environment”) .

Clean Air Act, 42 U.S.C. § 7409(b)(1)

NAAQS for criteria pollutants must “protect the public health,” “allowing an adequate margin of safety.”

Federal Insecticide, Fungicide, and Rodenticide Act, 7 U.S.C. § 136a(c) (5)(D)

Allows pesticides to be registered only if the administrator finds that “when used in accordance with widespread and commonly recognized practice it will not generally cause unreasonable adverse effects on the environment.”

Federal Food, Drug, and Cosmetic Act, 21 U.S.C. § 346a

A protective standard for pesticide residues is rebutted only once “there is a reasonable certainty that no harm will result from aggregate exposure to these residues.”

Safe Drinking Water Act, 42 U.S.C. § 1412(b)(4)

Maximum drinking-water contaminants are “set at the level at which no known or anticipated adverse effects on the health of persons occur and which allows an adequate margin of safety.”

Clean Water Act, 33 U.S.C. § 1313(c)(2)(A)

The objective of the Act is to “restore and maintain the chemical, physical, and biological integrity of our Nation’s waters.” Water quality standards set by statute “shall be such as to protect the public health or welfare….”

Resource Conservation and Recovery Act, 42 U.S.C. § 6924(m)

Standards for treatment of hazardous wastes disposed onto land shall specify “those levels or methods of treatment, if any, which substantially diminish the toxicity of the waste or substantially reduce the likelihood of migration of hazardous constituents from the waste so that short-term and long-term threats to human health and the environment are minimized.”

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

General Directions

 

Comprehensive Environmental Response, Compensation, and Liability Act, 42 U.S.C. § 9621(b)

“The President shall select a remedial action thatis protective of human health and the environment, that is cost effective, and that utilizes permanent solutions and alternative treatment technologies or resource recovery technologies to the maximum extent practicable” and specifying additional criteria the President must consider in selecting a remedial action.

Specific Directions

 

Food Quality Protection Act of 1996, 21 U.S.C. 346a(b)(2)(C) and (D)

“In the case of threshold effects … an additional ten-fold margin of safety for the pesticide chemical residue shall be applied for infants and children” … with additioaal legislative specifications for the types ofinformation that must be used in conducting the risk assessment.

Safe Drinking Water Act, 42 U.S.C. § 300g-1 (b)(3)(B)

“The Administrator shall, in a document made available to the public in support of a regulation promulgated under this section, specify, to the extent practicable: i) each population addressed by any estimate of public health effects; ii) the expected risk or central estimate of risk for the specific populations; iii) each appropriate upper-bound or lower-bound estimate of risk….”

Resource Conservation and Recovery Act, 42 U.S.C. § 6924(g)(10).

Requiring (for example) the Administrator to “complete a study of hazardous waste managed [with specific types of treatment processes] … to characterize the risks to human health or the environment associated with such management … [n]ot later than five yearsafter March 26, 1996.”

Source: EPA 2004a.

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
  • Project patterns of contamination and their impacts under various regulatory proposals (including no regulation and, in some cases, deregulation).

To produce the kind of regulations authorized by such health- or welfare-oriented legislation, therefore, requires the use of the types of models discussed in the preceding section and displayed in Figure 2-3. The figure, to be sure, suggests a degree of simplicity that EPA does not necessarily enjoy in its regulatory activities. EPA must translate general and sometimes vague statutory prescriptions into specific rules governing the behavior of individuals, firms, and state and local governments; pollutant sources must be identified and brought into compliance with the rules; and periodic assessments must be undertaken to ensure that satisfactory progress is being made to meet the statutory goals. Notions like “restore and maintain the chemical, physical, and biological integrity of our Nation’s waters” must be put into regulatory practice. Such legislative mandates often require EPA to develop or use models despite substantial data gaps and minimal supporting theory. For example, besides requiring the use of MACT standards for HAPs, the 1990 CAA Amendments also required a secondary regulatory phase when EPA is instructed to assess the “residual risk” due to a HAP that remains after compliance with the standards. Besides the need to interpret the meaning of the term “residual risk,” there are many technical difficulties associated with assessing such a risk, including the methods of calculating risks and data limitations (NRC 2004a). A similar modeling challenge occurs with the mandates in the Food Quality Protection Act that requires EPA to assess aggregate health risks from exposure to one chemical from multiple pathways and cumulative health risks from aggregate exposure to multiple pesticides (EPA 2001c, 2002d). Though it may pose difficulties for modelers, the agency’s priority must be to ensure that its regulations meet the requirements set forth under the legislation, not whether the regulations fit model capabilities.

Legislation also affects how EPA uses model assumptions. For example, under the CAA, EPA is instructed to set NAAQS for criteria pollutants that are “requisite to protect the public health” with an “adequate margin of safety.”4 This mandate has been interpreted by the

4

Criteria pollutants are air pollutants emitted from numerous or diverse stationary or mobile sources for which NAAQS have been set to protect human health and public welfare. The criteria pollutants are ozone, particulate matter, carbon monoxide, sulfur dioxide, nitrogen dioxide, and lead.

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

courts to require EPA to use models in such a way that their results are more likely to err on the side of caution with respect to protecting public health and to prohibit the agency from taking economic costs into account in setting standards.5 The adequate margin of safety is a policy choice of the EPA administrator’s intended to protect sensitive groups from adverse health effects (Murphy and Richmond 2004). The impact on regulatory modeling is that control costs, technological feasibility, and cost-benefit comparisons are not included in the analysis used to set NAAQS. It also causes EPA to consider a variety of sources of modeling evidence as shown in Figure 2-4.

Executive Branch Oversight

While being overseen by the White House, Congress, and others, EPA exercises substantial discretionary authority to implement and enforce environmental laws. With respect to models, EPA makes the vast majority of decisions about whether a model is needed to implement or enforce a legislative mandate, how to select and review models to carry out its authorities, and when it is time to replace one model with another.

The executive branch has provided oversight of the regulatory process through analytical requirements for the review of the costs, benefits, and effects of all major regulations. This factor has produced extensive modeling requirements for major regulatory actions overseen primarily by the Office of Information and Regulatory Affairs within OMB. One requirement is for an assessment of benefits and costs for major regulations through an RIA. Box 2-3 discusses the history of the RIA requirement.

For an RIA to be required, a regulation must have estimated economic effects that exceed $100 million annually or must have important adverse effects on prices, employment, productivity, or other economic consequences. Few regulations issued by EPA or other agencies require an RIA; in FY2004, for example, 4,088 rules were published in the Federal Register, but only 11 had RIAs. Of those 11, 6 were issued by EPA, 4 were issued by the Office of Air and Radiation, and 2 were issued by the Office of Water. Despite the small numbers, OMB estimates that the rules requiring RIAs “capture the vast majority of total costs and benefits

5

Lead Indus. Ass’n v. EPA, 647 F.2d 1148 (D.C. Cir. 1980); American Trucking Ass’ns v. EPA, 175 F.3d 1027, 1034 (D.C. Cir. 1999).

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

FIGURE 2-4 Sources of information for setting various NAAQS. Source: Murphy and Richmond 2004.

BOX 2-3

The Development of the Requirement for Regulatory Impact Analysis for Major Federal Rules

RIAs are required currently for any regulation whose estimated economic effects (costs) exceed $100 million annually or have important adverse effects on prices, employment, productivity, or other economic consequences. The requirement for an RIA came about as Presidents sought to have more influence over the agendas of executive agencies by requiring a review of the costs, benefits, and effects of all major regulations. The key event was Executive Order 12291 (EO12291), issued on February 17, 1981, announcing new rules governing the issuance of regulations by federal agencies. EO12291 introduced two important innovations into federal rule-making. First, it required federal agencies to produce, before certain “major” proposed regulation could appear in the Federal Register, an assessment of the benefits and costs of the proposal and alternatives to it. Before this executive order, economic assessment of regulations was concerned not with benefits and costs but with “economic impacts,” which included the effect of the regulation on inflation, employment, and the profits of affected industries.6 In addition, EO12291 required centralized review of regulations and the accompanying RIA by an oversight group, the Office of Information and Regulatory Affairs (OIRA) housed in OMB.

Each President since has either issued his own executive order affirming the RIA requirement and the OMB review or accepted that of his predecessor. For example, EO12866, issued on September 30, 1993, changed the procedure to increase the public’s accessibility, added requirements to specifically address

6

See Magat et al. (1986) for a discussion of the preparation and use of such studies in the Effluent Guidelines rule-making process.

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

the problem to be addressed by the regulation (usually a market failure) and examine distributional consequences of new rules, and require only that the benefits of proposed regulations have to “justify” the costs, not “outweigh” the costs as it had been in EO12291. For the most part, recent Presidents of both parties have retained support for regulatory review requirements, including the RIA. The implication of the RIAs for EPA modeling is that where possible, all the effects of a proposed regulation, positive and negative, must be expressed in monetary terms. Since most of the benefits—and many of the costs—of environmental regulation are not traded in markets, econometric models are needed to estimate individuals’ willingness to pay for the predicted physical effects of regulations, such as improved air quality.7 The RIAs could result in the estimation of regulatory benefits and costs even for rules where the enabling legislation has expressly forbidden the use of costs to make regulatory decisions. For example, as noted in the preceding section, the CAA prohibits cost to be a criterion in the setting of NAAQS. However, that did not prevent a very extensive and thorough RIA from being prepared to support the 1997 revision of the ambient standards for ozone and fine particulates (EPA 2006g). The RIA found very large positive net benefits for both standards, so there was no actual conflict between the RIA requirement that the costs be justified by the benefits and the legislative conflict that costs not be considered. The most recent OMB guidance on the preparation of RIAs is in OMB Circular A-4 (OMB 2003), which has expanded the requirements for uncertainty analysis.

of all rules subject to OMB review” (OMB 2005). In addition, rules exceeding $1 billion per year in economic effects are subject to a further requirement to include a formal analysis of uncertainty. Only the non-road diesel rule in FY 2004 was subject to this requirement (OMB 2005). As discussed in Chapter 4, uncertainty analysis adds considerably to the analytical burden of producing and comparing alternative regulations with unclear benefits.

In addition, the executive branch has been interested in the quality of information and peer review practices used by federal agencies, including EPA. One set of guidelines developed by OMB is Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies (OMB 2001). These guidelines, which were mandated by the Information Quality Act (IQA) (Treasury and General Government Appropriations Act for Fiscal Year 2001, Pub. L. No. 106-554, § 515, 114 Stat. 2763 [2000]), called for

7

A snapshot of the state of the art in valuing mortality and morbidity reductions (by far the most important source of monetizable environmental benefits) can be found in the proceedings of a workshop sponsored by EPA’s National Center for Environmental Economics (EPA 2006h).

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

agencies to issue information-quality guidelines to ensure the quality, objectivity, utility, and integrity of information. Recognizing the critical roles that models have in developing information, EPA has developed its own guidelines for data use to ensure that the models used in regulatory proceedings are objective, transparent, and reproducible (EPA 2002a). In addition, as discussed in a later section, OMB has released guidance on peer review (OMB 2004).

OVERSIGHT PROCESSES GOVERNING REGULATORY MODELS AT EPA

After Congress or EPA has decided to use a model for one or more regulation-relevant purposes, the model normally goes through some internal and external oversight to ensure that it meets scientific, stakeholder, and public approval. Although these oversight processes are not perfect and run the risk of introducing their own sources of error or complication, they nevertheless exert an important and independent pressure on regulatory models that is generally not present when models are developed and used in nonregulatory settings.

Because the results of models can impose important costs on regulated parties and the public at large, EPA’s evaluation of models used for regulatory design and promulgation (the rule-making phase from above) is the most heavily constrained by legislative requirements, regulatory review, and legal challenges. Figure 2-5 shows a schematic of the regulatory requirements placed on the regulatory design and promulgation phase. Models used for other regulatory purposes—outside of rule-makings—are generally not subjected to these extensive internal and external review requirements. Models used at the enforcement stage, for example, are generally not required to go through peer review or even notice and comment, but they are required to at least gain judicial acceptance before a court will enter penalties against a violator based on the model. Models used in environmental regulatory programs delegated to the states, such as models used to develop SIPs and TMDLs, can be subjected to public comments and debate, but independent peer reviews of individual model applications are not required. Models used in providing guidance may be subjected to scientific and public review, but generally, this review is done at the agency’s discretion. The Science Advisory Board and Science Advisory Panel, described in a subsequent section, are two sources for peer reviews. Models used for strategic

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

FIGURE 2-5 Flow chart of general regulatory requirements for models used at the regulatory design and promulgation stage.

planning or priority setting within the agency involve even fewer mandatory oversight processes. Yet in these cases, EPA still develops guidelines for internal peer review and may voluntarily subject these models to sources of external review as well.

Because regulatory design models encounter the most extensive oversight requirements and also tend to be an important modeling activity at EPA, regulatory design models are the focus of the remaining discussion. In general, these models require multiple layers of review, including formal scientific peer review, notice and comment processes, and intra-agency review. Interested parties are also provided with an opportunity to challenge the model to the agency and in court to ensure that the model is reliable.

External Review of EPA’s Models

The first and perhaps most important set of requirements involves subjecting regulatory decisions, including the models underlying them, to review by three layers of outside reviewers. This external review is thus conducted independently of the authors of the model or the users for a specific application. This section summarizes the current state of EPA

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

review activities, recognizing that there is no single approach. It depends on the nature of the model, its application, the needs of the model developers and users, the peer review guidance being followed, and the requirement of the specific regulatory environment statutes. For the purposes of this section, external reviews are categorized as peer review, public review, and interagency review.

Peer Review

This category refers to technical experts reviewing the model and its application for scientific merit. Although it is expected that key elements of models will be published in the peer reviewed literature, this discussion does not address journal reviews. Peer review is embedded in the history of science because of its value in improving the quality of a technical product and providing assurance to nonexperts that the product is of adequate quality. These values are so important that attention must be paid to the quality of the peer review itself and to whether the comments were addressed and appropriately incorporated into the final product. All peer reviews are not equivalent. A peer review on model code, for example, will be useful, but inadequate to evaluate the utility of the model for a specific application. Thus, the charge to each peer review for a model and its application needs to be considered relative to the criteria for model evaluation and where the model is in its life cycle, as described in Chapter 4.

In July 1994, EPA published Guidance for Conducting External Peer Review of Environmental Regulatory Modeling (EPA 1994c), which was a prelude to broader peer review guidance published in 2006 (EPA 2006a). The 2006 guidance is very comprehensive and detailed, describing such elements as matching the kind and degree of peer review to the impact of the work product (product of influential scientific information, very influential scientific information, or other) or rule (Tier 1, 2, or 3 rule), determining resources needed for peer review, selecting peer reviewers, documenting the review, and so forth. EPA has also created an “action development process” for regulations and other decisions. OMB also has published the Final Information Quality Bulletin for Peer Review (70 Fed. Reg. 2664 [2005]). Although these three documents differ in details, they are conceptually similar because they require peer review of models or their applications that are most likely to have “major” or “substantial” impacts. They also describe the need for peer reviewers to have the necessary technical expertise and to be free of con-

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

flicts of interest and the need for a panel to balance biases. OMB’s guidance has greater emphasis on the need to make key elements of the review available to the public. The EPA Science Inventory keeps a list of the different science activities and their required levels of peer review. Its activities are broad and described at http://cfpub.epa.gov/si/si_pr_agenda.cfm.

The guidance on regulatory models calls for reviews with the goals of “judging the scientific credibility of the model including applicability, uncertainty, and utility (including the potential for misuse) of results, and not for directly advising the agency on specific regulatory decisions stemming in part from consideration of the model output” (EPA 1994c). Box 2-4 lists elements of peer review described by EPA for use with regulatory models. This guidance also offers a framework for reviewing model development, model application, and environmental regulatory decision making. It explains that policy decisions resulting from the science and other factors are required by law to be made by EPA decision makers. The policy decisions are often subject to public comment.

BOX 2-4

Elements of External Peer Review for Environmental Regulatory Models

Model Purpose/Objectives

  • What is the regulatory context in which the model will be used and what broad scientific question is the model intended to answer?

  • What is the model’s application niche?

  • What are the model's strengths and weaknesses?

Major Defining and Limiting Considerations

  • Which processes are characterized by the model?

  • What are the important temporal and spatial scales?

  • What is the level of aggregation?

Theoretical Basis for the Model—formulating the basis for problem solution

  • What algorithms are used within the model and how were they derived?

  • What is the method of solution?

  • What are the shortcomings of the modeling approach?

Parameter Estimation

  • What methods and data were used for parameter estimation?

  • What methods were used to estimate parameters for which there were no data?

  • What are the boundary conditions and are they appropriate?

Data Quality/Quantity

Questions related to model design include:

  • What data were utilized in the design of the model?

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
  • How can the adequacy of the data be defined taking into account the regulatory objectives of the model?

Questions related to model application include:

  • To what extent are these data available and what are the key data gaps?

  • Do additional data need to be collected and for what purpose?

Key Assumptions

  • What are the key assumptions?

  • What is the basis for each key assumption and what is the range of possible alternatives?

  • How sensitive is the model toward modifying key assumptions?

Model Performance Measures

  • What criteria have been used to assess model performance?

  • Did the data bases used in the performance evaluation provide an adequate test of the model?

  • How does the model perform relative to other models in this application niche?

Model Documentation and Users Guide

  • Does the documentation cover model applicability and limitations, data input, and interpretation of results?

Retrospective

  • Does the model satisfy its intended scientific and regulatory objectives?

  • How robust are the model predictions?

  • How well does the model output quantify the overall uncertainty?

Source: EPA 1994c.

EPA has several forums to conduct peer reviews: the EPA Science Advisory Board (SAB), the EPA Clean Air Science Advisory Committee (CASAC), the EPA Science Advisory Panel (SAP), or ad hoc committees. They are described in more detail in Box 2-5. The first three organizations are convened under the Federal Advisory Committee Act and are subject to requirements of that act, including that all meetings and deliberations must be public. Major ad hoc committees also hold open meetings. Typically, the charges to SAB, CASAC, and SAP are broad. Ad hoc committees are often used for more in-depth reviews. All types of peer review are of substantial value, but the adequacy of peer review of a model must be judged in context with the need for evaluation of each major step from model conception to application. Major reviews, such as those performed by SAB, besides providing valuable input to agency scientists and managers, can become a part of the administrative record and can be used in court challenges. Examples of model peer reviews are the SAB reviews of the 3MRA model (EPA 2004e), the SAB review of the EPA Region 5 critical ecosystem assessment model (EPA

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

BOX 2-5

The Different Types of Science Advisory Panels at EPA

The CASAC was established under the CAA to review EPA’s NAAQS and report to the EPA administrator. It is administratively housed in SAB. This group reviews the “criteria documents” of the criteria air pollutants to evaluate whether the information contained is adequate to support a decision. They also review the staff paper that has the EPA staff’s recommendations for the standard. Both documents rely on models.

SAB traces its history to 1978. Its charge is to provide independent science and technical advice, consultation, and recommendations to the EPA administrator on the technical bases for agency positions and regulations. Most of its activities involve reviewing technical documents, including numerous model reviews (e.g., EPA 2004e, 2005b). SAB also produced the Resolution on the Use of Mathematical Models by EPA for Regulatory Assessment and Decision-Making (EPA 1989).

The federal Insecticide, Fungicide, and Rodenticide Act established SAP in 1975. The Food Quality Protection Act mandated a science review board of scientists who would be available to SAP on an ad hoc basis. SAP provides scientific advice, information, and recommendations to the EPA administrator on pesticides and pesticide-related issues as to the impact of regulatory actions on health and the environment. Several SAP panels have focused on models to predict exposures to pesticides or on pesticide health assessments that were partly based on models. SAP panels summarize their discussions and issue recommendations in the minutes of the meetings (e.g., EPA 2005c).

Ad hoc committees are often used by EPA when the document being reviewed does not have the impact that invokes the need for SAB, CASAC, or SAP. As related to models, they might involve highly technical reviews before the SAB-level stage or might be for risk assessments that include some degree of reliance upon models.

2005b), and the SAP preliminary evaluation of physiologically based pharmacokinetic and pharmacodynamic modeling for the N-methyl carbamate pesticides (EPA 2005c).

Public Review

Public review of a regulatory model concerns review and comments by stakeholders during the public comment periods of external peer review activities or during the “notice and comment” period that accompanies rule-making activities. Herein, “stakeholder” is defined as a person or nonfederal entity and external to the agency not involved in the above-described peer review. They include members of the general public. Thus, many individuals and entities are stakeholders and have different interests, capabilities, and capacities to perform this role. For

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

example, consider the different capabilities to generate comments on models and model results between a member of the general public with limited abilities to perform computational analysis and a corporation or other organization with a substantial scientific staff. These differences need to be understood and accommodated when fulfilling the intent and actual requirements for public review. When EPA requests a peer review by CASAC, SAB, or SAP, the document is made public, and the public is able to comment at the public meetings of these organizations as per the Federal Advisory Committee Act. Furthermore, EPA is required by statute to solicit comments from affected parties and the public at large on all final proposals for agency action (5 U.S.C. § 553). A mandatory “notice and comment” process is intended to ensure that the agency informs the public of its activities and takes their concerns and input into account. According to statute, EPA must also make all relevant documents in the record supporting its decision available to the public for viewing during the comment process.

Interagency Review

EPA’s regulations are developed and implemented as part of a larger federal fabric. For example, some of EPA’s regulations affect other agencies directly (for example, Department of Defense Superfund sites) and indirectly (for example, economic consequences to policies of other agencies). A example of an EPA model that plays a critical role in another agency’s activities is the motor vehicle emissions factor model, (MOBILE), which plays an important role in the Department of Transportation (DOT) transportation planning activities (Ho 2004). This has inspired DOT to evaluate aspects of MOBILE directly (Tang et al. 2003a,b). Thus, there is a variety of both formal and informal processes for interagency review of regulatory models and analysis based on these models. The majority of interagency reviews involve mandatory oversight by OMB, although other agencies may also engage in more informal review and comment. Under various executive directives, OMB review is generally cursory unless the regulatory program, which the model informs, is deemed to be “significant” with respect to its economic implications (Graham 2004). OMB oversees these process requirements and will work with the agencies to ensure that their regulatory analyses are satisfactory. OMB review of other agencies’ rule-makings is generally established through executive order and, while these presidential

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

directives are mandatory, agency violations cannot be enforced through the courts.

Completing the Review Cycle

Several of the processes of external reviews are still not transparent in regard to the disposition of the comments. In some instances, the effect of comments on the regulatory process is not clear. It is understood that not all comments are appropriate or useful, even though all need to be carefully considered. Thus, the issue is transparency—those commenting, from prominent scientists on the SAB to members of the general public, need to understand how their comments were considered. EPA’s Peer Review Handbook (EPA 2006a) discusses this issue and calls for a written record of response to comments. EPA has an exemplary process in terms of transparency for the NAAQS where a public docket contains both the original comment and the agency’s responses.

Legal Challenges to EPA’s Models

Laws and executive orders not only provide a mechanism for increased external inputs to EPA’s models but also provide opportunities for adversarial challenge. There are two formal opportunities for interested parties to challenge EPA’s models. The first and most established is the ability of interested parties to challenge agency action in court. If the model supports a regulation and has been subject to notice and comment, the courts give EPA considerable deference. Thus, challenges to EPA models are successful only when the regulation (and/or underlying model) is in conflict with EPA’s statutory mandate, has been determined to be inconsistent with Administrative Procedure Act requirements, or is “arbitrary and capricious” (5 U.S.C. § 706). As one court summarized in reviewing a model: “This Court must not undertake an independent review of EPA’s scientific judgments; our inquiry focuses only on whether the agency has met the statutory requirement for ‘sufficient evidence.’” (National Oilseed Processors Ass’n v. Browner, 924 F. Supp. 1193, 1209 [D.D.C. 1996], affirmed in part and reversed in part on other grounds, Troy v. Browner, 120 F.3d 227 [D.C. Cir. 1997]). If the model has not been subject to notice and comment but creates obligations for private parties—for example, at the permitting or enforcement stage—those af-

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

fected by the model can typically challenge either the model or its application in court. In some of the cases, the agency may receive much less deference from the courts compared to the situation where the model has been subject to notice and comment (for example, see United States v. Plaza Health Laboratories, Inc., 3 F.3d 643 [2d Cir. 1993]) (applying the rule of lenity, rather than deferring to EPA, in interpreting “point source” in a criminal CWA prosecution). Generally, a complete model history documenting the justification for various decisions related to model design and development may help the agency defend a model against formal challenges.

EPA’s models have sometimes been challenged, and in some cases, challengers have been successful in forcing the model or its application back to EPA to correct what the courts view as fundamental flaws. Some of this judicial activity may be a result of EPA’s past, ad hoc approach to developing and using models; a more rigorous and formalized approach might ward off some of these challenges by instituting more rigorous modeling practices in the agency. For example, when EPA declines to explain its decision or revise a supporting model even after receiving comments refuting one of the model’s critical assumptions, the courts have invalidated and remanded the model back to EPA. Challengers have also been successful when they establish that EPA’s model is not applicable to a particular subset of industries, activities, or locations. If EPA applies a generic air dispersion model to a large power plant located in a meteorologically unusual setting, such as the shores of Lake Erie, EPA might have to test the location to establish that the model provides some reliability in that setting, or it must be prepared to explain why its model should be accepted as is (for example, State of Ohio v. EPA, 784 F.2d 224 [6th Cir. 1986] and 798 F.2d 880 [6th Cir. 1986]).8 Finally, if challengers disagree with embedded policy judgments, such as the risk adversity of assumptions built into a risk assessment, courts will sometimes invalidate a model and not defer to the agency (Gulf South Insulation v. Consumer Product Safety Commission, 701 F.2d 1137 [5th Cir. 1983]). However, this line of cases is more complex and unpredictable (Pierce

8

Remanding EPA’s air dispersion model because EPA had not adequately demonstrated that its CRSTER model took into account the “specific meteorological and geographic problems” of the modeled large power plants situated on the shores of Lake Erie. It was therefore arbitrary and capricious for EPA to allow a 400% increase in emissions “without evaluation, validation, or empirical testing of the model at the site.”

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

1988).9 Because these legal challenges are time-consuming and costly, they are typically mounted only when an affected or interested party stands to gain something important—whether it is gaining less stringent regulatory requirements or positive publicity for members—from a challenge.

A second, more recent opportunity for external challenge to model use in the regulatory process is through the Information Quality Act (Treasury and General Government Appropriations Act for Fiscal Year 2001, Pub. L. No. 106-554, § 515, 114 Stat. 2763 [2000]), which is implemented through OMB’s Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies (OMB 2001). Some of challenges under the Information Quality Act result from EPA’s occasional ad hoc approach to developing and using models. This statutory provision allows any interested person to file “requests for correction” on “information” that is “unreliable” or lacks other qualities, such as objectivity or integrity. To date, courts have refused to review these challenges, but the challenges can be appealed inside the agency and the agency must respond to complaints that the information, including information used in models or the models themselves, is unreliable. However, there are continued efforts to make challenges under the Information Quality Act reviewable by the judiciary (Shapiro et al. 2006).

Challenges filed under the Information Quality Act to date generally target technical decisions within EPA that have important economic consequences (EPA 2006i). In at least one instance—the Competitive Enterprise Institute’s (CEI’s) challenge to the climate change models used in the National Assessment on Climate Change—the challenge has been directed specifically at agency models (EPA 2003c). In the case, CEI argued that the models were not reliable and had not been adequately peer reviewed. The agencies denied the petitions and CEI’s internal appeals. CEI then appealed its case to the D.C. District Court where CEI ultimately withdrew its case. Information Quality Act challenges brought by affected parties sometimes seek correction of flaws or technical misstatements in agency documents, but in other instances, as in the Competitive Enterprise Institute’s challenge, they have requested that the agencies cease dissemination of the information. If an agency

9

Arguing that judges on the D.C. Circuit may be substituting their own interpretations of ambiguous statutes for agencies and randomly reversing agency policy making in rule-makings.

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

denies a petition on appeal, as has been the case for most IQA challenges filed to date, the challenge fails.

THE CHALLENGES OF MODELING IN A REGULATORY ENVIRONMENT

This chapter has described the types of models used in EPA’s regulatory activities, how models fit into the regulatory process, and legal and other constraints governing their use. Modeling is a difficult enterprise even when it is not being conducted in an adversarial regulatory environment. Further, the range of model applications is vast, and many agencies and stakeholders are involved in producing analysis. When the demands of regulatory accountability, transparency, public accessibility, and technical rigor are added to the challenges typically encountered in modeling, the task becomes much more complex.

Although improvements to EPA regulatory modeling efforts are possible, EPA clearly has made important advances in the science of environmental modeling and has been a global leader in using models in the environmental regulatory decision process. However, future regulatory modeling activities will be challenged by new scientific understandings, expanding sources of environmental and human observations, and new issues. To meet the challenges, continued improvement in model practices will be required. In this chapter, the committee offers recommendations related to continuing improvements to the accessibility of regulatory modeling. Later in this report, we offer recommendations related to model evaluation; principles for model development, selection, and application; and model management.

Model Goals

Models are used in regulatory settings when EPA determines that a model will be useful in reaching or enforcing a regulatory decision. Given the diversity of regulatory aims and targets, however, a wide variety of models and modeling goals can exist. At one extreme, the agency can use a model that provides the best technical analysis of the concentrations of ambient air pollutants and resulting health and environmental impacts most likely to result from combined industrial and nonindustrial emissions controls. This precision is desired because of the enormous compliance costs associated with emissions controls and the enormous

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

health costs if air pollutants are not correctly estimated and exceed allowable levels. At the other extreme, EPA might want to use a model that provides only a crude and inexpensive prediction for a system.

The regulatory environment also creates the opportunity for many different types of legal constraints on modeling that are foreign in non-regulatory settings. Congress may instruct, for example, that a regulation err on the side of over-predicting public health harms. Other constraints might result from legislative mandates that EPA develop and use models in situations where resources, including both time and financial support, are scarce.

Time and resource limitations can also lead EPA to use existing models outside their “application niche,” a set of conditions for which the model is designed to be useful. There is some evidence, for example, that EPA and other agencies have sometimes used a model in a setting where the model no longer provides useful guidance. For example, EPA’s generic test to predict the toxicity of wastes in landfill settings (the Toxicity Characteristic Leaching Potential Test) generally adopts worst-case assumptions. Yet, in some disposal settings, the worst-case assumptions have been challenged successfully as inapplicable for specific types of disposal operations, such as for the disposal of a particular type of waste (potliner waste) in a monofill (for example, see Columbia Falls Aluminum Co. v. U.S. Environmental Protection Agency, 139 F.3d 914 [D.C. Cir. 1998]; Edison Electric Institute v. U.S. Environmental Protection Agency, 2 F.3d 438 [D.C. Cir. 1993]; and Association of Battery Recyclers, Inc. v. U.S. Environmental Protection Agency, 208 F.3d 1047 [D.C. Cir. 2000]).

Technical Reliability

The sometimes contentious environment for regulatory models also creates important impediments for ensuring the technical reliability of EPA’s models. Formal evaluation processes required by administrative law may deter meaningful model reevaluation and adjustment over time. Once a regulatory action has survived the multilayered review and challenge processes, it may remain in place for some time. Indeed, rule-making requirements can be read to require that the agency undergo notice and comment and the risk of judicial review every time it revises a model that supports a rule-making, since it must ensure that there has been “meaningful public comment” on all aspects of its final rule (for

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

example, see Small Refiner Lead Phasedown Task Force v. EPA, 705 F.2d 506, 540-41[D.C. Cir. 1983]). This inertia is not ideal for any regulatory decision, but it is particularly unfortunate for models. The cumbersome regulatory procedures and the finality of the rules that survive them are directly at odds with the dynamic nature of modeling and the goal of improving models in response to experience. Although some stakeholders may prefer a constant model because of the stability it provides, this model might not reflect the most updated science.

Transparency and Accountability

In the regulatory environment, EPA has the responsibility to ensure that a model’s development and use is transparent. Because modeling is often a very technical exercise, EPA faces a challenge in making all of the underlying decisions intertwined within a model intellectually accessible to a nontechnical audience. A model that attempts to determine the fate of a chemical in soil, for example, may involve choices between competing assumptions, such as the percolation rate of a chemical at a particular location. Selection of the most appropriate assumption in some cases may depend not only on technical judgment but also on the policy goals of the modeling effort. A recent EPA report documents how science mingles with policy in health risk assessment (EPA 2004a). If the model is supposed to err on the side of protecting health and the environment, the model may need to err on the side of quicker percolation rates when several choices are plausible. Making these choices explicit and accessible is a challenge because policy judgments can be numerous and varied in their importance. Nevertheless, administrative processes expect EPA to make many of these types of judgments and technical decisions transparent so that affected stakeholders and the general public can comment on the model and its regulatory implications.

Because models are uncertain and are used to make policy, stakeholders necessarily play a vital role in EPA’s development, use, and evaluation of models. Differing interpretations of risk, risk preferences, and a range of other values and understandings mean that a broad array of participants will have much to add to the modeling exercise. As a result, these various constituencies and individuals must be able to participate in the model evaluation process through various activities, including producing their own supporting or conflicting model results, and challenging the legitimacy or accuracy of a model in public comments or judicial actions.

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

Clearly, EPA faces many difficult challenges in making its models, particularly its complex models, accessible to the diverse interests. Nevertheless, EPA has taken a major step in the right direction through the CREM database of models. This information further enhances the transparency and understandability of models to a wide array of interested participants. Despite these efforts, however, stakeholders with limited resources or technical expertise still face substantial barriers to being able to evaluate EPA’s models, comment on important model assumptions, or use the models in their own work.

Recommendations

EPA should place a high priority on ensuring that stakeholders and others have access to models for regulatory decision making. To ensure that its models database contains all actively used models, EPA should continue its support for the intra-agency efforts of CREM. A more formal process may be needed to ensure that CREM’s models database is complete and updated with information that is at least equivalent to information provided for models currently contained in the database.

Yet, even with a high-quality models database, EPA should continue to develop initiatives to ensure that its regulatory models are as accessible as possible to the broader public and stakeholder community. The level of effort should be commensurate with the impact of the model use. It is most important to highlight the critical model assumptions, particularly the conceptual basis for a model and the sources of significant uncertainty. Meaningful stakeholder involvement should be solicited at the model development and model application stages of regulatory activity, when appropriate. EPA could improve model accessibility through a variety of activities, such as requiring an additional interface for each model to help to identify the assumptions and sources of parameters and other uncertainties and providing additional user and stakeholder training.

However, even if full information on a model is available, technical expertise will still be required to judge independently its quality and suitability for regulatory application. Each of these recommendations requires staff time and resources, which may be considerable. Thus, EPA’s efforts to enhance opportunities for public participation in any particular case must be balanced against other agency priorities.

Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 40
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 41
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 42
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 43
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 44
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 45
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 46
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 47
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 48
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 49
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 50
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 51
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 52
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 53
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 54
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 55
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 56
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 57
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 58
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 59
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 60
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 61
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 62
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 63
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 64
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 65
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 66
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 67
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 68
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 69
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 70
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 71
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 72
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 73
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 74
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 75
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 76
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 77
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 78
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 79
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 80
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 81
Suggested Citation:"2 Model Use in the Environmental Regulatory Decision Process." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 82
Next: 3 Model Development »
Models in Environmental Regulatory Decision Making Get This Book
×
Buy Paperback | $65.00 Buy Ebook | $54.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Many regulations issued by the U.S. Environmental Protection Agency (EPA) are based on the results of computer models. Models help EPA explain environmental phenomena in settings where direct observations are limited or unavailable, and anticipate the effects of agency policies on the environment, human health and the economy. Given the critical role played by models, the EPA asked the National Research Council to assess scientific issues related to the agency's selection and use of models in its decisions. The book recommends a series of guidelines and principles for improving agency models and decision-making processes. The centerpiece of the book's recommended vision is a life-cycle approach to model evaluation which includes peer review, corroboration of results, and other activities. This will enhance the agency's ability to respond to requirements from a 2001 law on information quality and improve policy development and implementation.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!