National Academies Press: OpenBook

Models in Environmental Regulatory Decision Making (2007)

Chapter: 3 Model Development

« Previous: 2 Model Use in the Environmental Regulatory Decision Process
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

3
Model Development

INTRODUCTION

Models are a simplification of reality that can be compared to maps. Road maps indicate certain aspects of reality (for example, roads of a certain size) and not others (for example, sewer lines, power lines, and buildings). No one map can include all aspects of reality and, similarly, all models, no matter how complex, are constrained by basic assumptions, structure, and uncertainties. Model development involves the definition of model objectives, conceptualization of the problem, translation into a computational model, and model testing, revision, and application. Although almost all model development follows these general steps, models designed for regulatory purposes are subject to constraints in addition to those for models developed strictly for research. This chapter focuses on how model development might best proceed toward regulatory objectives, although there is no one route for successful model development. Our objective is not to provide a treatise on model development. Many other references in an array of disciplines provide comprehensive descriptions of model development for various types of models (Starfield and Bleloch 1991; Clemen 1995; Mesterton-Gibbons 1995; Beck 2002a; Bassetti and Woodward 2005; Ramaswami et al. 2005). This chapter discusses the major steps in regulatory model development focusing on the main lessons learned from previous efforts in EPA and other organizations. It is intended to discuss some of the literature on model development and provide a general framework for EPA as it goes about its business. The wide range of environmental model types makes

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

our effort prone to both overgeneralization and oversimplification. To reduce such difficulties, we will often refer to examples of regulatory model development, especially those from air quality modeling. Box 1-1 in Chapter 1 contains a brief history of EPA’s effort to model tropospheric ozone.

ALTERNATIVE MODEL DEVELOPMENT PATHWAYS

Some regulatory models arise from those developed as general research tools. Others were developed specifically for addressing regulatory issues. They have been developed by EPA scientists, academia, national laboratories, or the private sector. Some of the most complex models have benefited from contributions by almost all of the above. For example, complex regional chemical transport models for simulating air quality usually include components contributed by multiple parties. The urban airshed model (UAM), heavily used for the design of ozone control strategies in the 1980s and 1990s, was developed by a private company (Systems Applications International) relying on contributions of academia and on support from public and private organizations. The major air quality model developed for use in-house by EPA is the community multiscale air quality (CMAQ) model (EPA 1999b). EPA and NOAA scientists developed the most recent CMAQ model in partnership with a nonprofit organization (Microelectronics Center of North Carolina) and contributions by academia funded by EPA, the National Science Foundation and state authorities, most prominently California authorities (CMAS 2006). A variation of CMAQ, called CMAQ-MADRID, has been developed by a private company (Atmospheric and Environmental Research, Inc) using the CMAQ model as a starting point and adding components developed by academic researchers or by company scientists (Zhang et al. 2004). A private organization, Electric Power Research Institute, provided funding for the CMAQ-MADRID development. All the above codes are in the public domain.

Under the Toxic Substances Control Act, EPA must make individual pre-manufacturing decisions on 2,000 new chemicals per year before a new chemical can enter the market. Because of the large number of decisions, the agency has had to rely on screening tools that predict properties from chemical structure. EPA uses EPI (Estimation Programs Interface) Suite, which consists of several quantitative structure-activity relationships (QSARs) models that are available in the public domain.

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

However, the set of models began as a few proprietary models developed by Syracuse Research Corporation. Later some of the models were developed in collaboration with EPA, and then all the models were sold to EPA. QSARs are able to take complex chemical structures and predict physical properties, behavior in the environment, and toxicity (Jaworska et al. 2003; Tunkel et al. 2005). The EPI Suite is used by EPA’s Office of Pollution Prevention and Toxics to predict physical-chemical properties, environmental fate and transport, and aquatic toxicity for regulatory decisions on new chemicals when data are not available. The models are also used by industry for pollution prevention and by many government agencies for identifying persistent, bioaccumulative, and toxic (PBT) chemicals (Jaworska et al. 2003).

Alhough the development paths of models may be different, many end up having long lives in the regulatory process. Table 3-1 shows the life history of the MOBILE model, which is used to estimate atmospheric emissions from vehicles. This table indicates the periodic revisions that necessarily accompany a model that has been in use for almost 30 years. In the case of MOBILE, such revisions are often major overhauls and updates of the model, resulting in emissions estimates being much different from these in previous versions (NRC 2000; Holmes and Russell 2001). Along with the UAM discussed above, the QUAL2 water quality model is an additional example of a regulatory model that has seen multiple versions and major scientific modifications and extensions in over 2 decades of existence (Barnwell et al. 2004).

OVERVIEW OF MODEL DEVELOPMENT

Jakeman et al. (2006) separates model development and evaluation into the 10 steps shown in Figure 3-1. The committee agrees with the concept shown in Figure 3-1 that model development is typically an iterative process, especially for long-lived models used over several decades. However, for the purposes of this chapter, the model development process is compressed into the six phases shown in Box 3-1. Documentation occurs at each step of the process, as do certain aspects of evaluation. Chapter 4 describes in detail the evaluation process that occurs throughout the model’s life cycle, compressing the model lifecycle further into 4 steps (problem identification, conceptual

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

TABLE 3-1 MOBILE Model RevisionsVersion

Version

Release Date

Model Revisions

MOBILE1

1978

  • Included modeling of exhaust emissions rates as functions of vehicle age and mileage (zero-mile levelsand deterioration rates).

MOBILE2

1981

  • Updated with substantial data (available for the first time) on emission-controlled vehicles (catalyticconverters, model years 1975 and later) at higher ages and mileages.

  • Provided additional model user control of input options.

MOBILE3

1984

  • Updated with substantial new in-use data.

  • Elimination of California vehicle emissions rates (continued to model low- and high-altitudeemissions).

  • Added tampering (rates and associated emissions impacts) and antitampering program benefits.

  • In-use emissions-factor estimates for nonexhaust emissions adjusted for real-world fuel volatility as measured by Reid vapor pressure (RVP).

MOBILE4

1989

  • Updated with new in-use data.

  • Added running losses as distinct emissions source from gasoline-powered vehicles.

  • Modeled fuel volatility (RVP) effects on exhaust emissions rates.

  • Continued expansion of user-controlled options for input data.

MOBILE4.1

1991

  • Updated with new in-use data.

  • Added numerous features allowing user control of more parameters affecting in-use emissions levels, including more inspection and maintenance (I/M) program designs.

  • Included effects of various new emissions standards and related regulatory changes (for example, test procedures).

  • Included impact of oxygenated fuels (for example, gasohol) on CO emissions.

MOBILE5

1993

  • Updated with new in-use data, including basing new basic emissions-rate equations on much larger database derived from state-implemented IM240 test programs.

  • Included effects of new evaporative emissions test procedure (impact on in-use nonexhaust emissions levels).

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

MOBILE5

1993

  • Included effects of reformulated gasoline (RFG).

  • Included effects of new NOx standard of 4.0 g/bhp-hr for heavy-duty engines.

  • Included impact of oxygenated fuels on VOC emissions.

  • Included Tier 1 emissions standards under 1990 Clean Air Act Amendments.

  • Added July 1 evaluation option.

  • Included impact of low-emission vehicle (LEV) programs patterned after California regulations.

  • Revised speed corrections used to model emissions factors over range of traffic speeds.

MOBILE5a

1993

  • Corrected a number of minor errors in MOBILE5.

MOBILE5b

1996

  • Included final on-board vapor-recovery regulations.

  • Included final reformulated gasoline regulations.

  • Added more user options for I/M programs.

MOBILE6

2002

  • Added the effects of Tier 2 and new heavy-duty engine and diesel fuel rules.

  • Updated with new and improved data in many areas, including in-use deterioration of 1981 and newer vehicles, light-duty speed effects, gasoline sulfur effects, and evaporative emissions.

  • Revised I/M benefits algorithm; removed calculation of purge test benefit.

  • Revised algorithms for air conditioning and high acceleration driving.

  • Expanded number of vehicle subclasses from 8 to 28.

  • Added hourly calculation of emissions and emission estimates by roadway type.

  • Separated start and running exhaust emissions.

MOBILE6.2

2004

  • Added ability to model emission factors for particulate matter and six air toxics; added ability tomodel additional air toxics with user-supplied emission factors.

  • Updated carbon monoxide emission factors.

Source: EPA 1999c, 2006j.

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

FIGURE 3-1 Iterative steps in model development proposed by Jakeman et al. (2006). Source: Jakeman et al. 2006. Reprinted with permission; copyright 2006, Environmental Modelling & Software.

model development, computational model development, model use) to make the evaluation process more tractable to the reader.

A general issue concerns the uses for which a model is being constructed. This report includes models that are used for two main purposes: those used before regulations are developed to strategically plan and assess priorities and design, evaluate, and propose regulatory approaches (hereafter referred to as pre-regulatory planning models) and those used to implement regulatory programs, including programs that have been delegated to states and local governments (hereafter referred to as post-regulatory implementation and compliance models). These are two quite separate uses, although some models can be used for both purposes as discussed in Chapter 2. However, there might be some important differences. Pre-regulatory planning models require a more general

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

BOX 3-1

Basic Steps in Modeling Development Process

Model Development Step

Modeling Issues

Definition of Model Purpose

Goal

Decisions to be supported

Predictions to be made

Specification of Modeling Context

Scale (spatial and temporal)

Application domain

User community

Required inputs

Desired output

Evaluation criteria

Conceptual Model Formulation

Assumptions (dynamic, static, stochastic, deterministic)

State variables represented

Level of process detail necessary

Scientific foundations

Computational Model Development

Algorithms

Mathematical/computational methods

Inputs

Hardware platforms and software infrastructure

User interface

Calibration/parameter determination

Documentation

Model Testing and Revision

Theoretical corroboration

Model components verification

Corroboration (independent data)

Sensitivity analysis

Uncertainty analysis

Robustness determination

Comparison to evaluation criteria set during formulation

Model Use

Analysis of Scenarios

Predictions evaluation

Regulations assessment

Policy analysis and evaluation

framework, allowing alternative policy initiatives to be analyzed, perhaps by varying basic model assumptions. This analysis may be done at the national scale and by EPA. Post-regulatory implementation and compli-

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

ance models will typically be more closely tied to site-specific observational data, producing a plan for implementing a regulation or assessment of compliance for a given location or substance. Besides being used by EPA, modeling in the post-regulatory process may also be done by state and local governments and their consultants.

INTERDEPENDENCE OF MODELS AND DATA FROM MEASUREMENTS

Developing and evaluating models typically requires dependence on measurements. In some cases, there are plenty of measurement data for developing model parameters, boundary conditions, and other inputs. Often, however, data are missing, which is an inherent factor in the need for models. Optimally, measurements and models develop iteratively, each informing the other. Box 3-2 describes some examples where measurement data have influenced model development. Although there are trade-offs about whether it is preferable to invest in more data or in better models, the committee does not conclude that the problem of resource allocation for data versus models can be viewed simply as an optimization problem. The difficulty in attempting to formulate such a problem is how to define the optimization criteria and objective function—how does one define and represent the benefit from additional data one does not have relative to investing in additional modeling one does not have. Further, data are typically collected to fulfill multiple objectives, including determining compliance with environmental regulations, further complicating attempts to formulate the data versus model issue into an optimization context.

MODEL DEVELOPMENT PHASES

Definition of Model Purpose

The first step involves defining the major purpose or purposes for which the model is developed. As discussed in Chapter 4, this occurs at the problem identification stage when decision makers, model developers, and other analysts must consider regulatory needs and whether

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

BOX 3-2

Interdependence of Models and Data from Measurements

Models are developed and evaluated using a wide range of data, theories, and assumptions and are revised in the process. The wisdom of iteration of measurements and modeling is illustrated by three examples.

Persistent organic pollutants (POPs) in the Arctic. POPs are chemical substances that persist in the environment, bioaccumulate through the food web, and pose a risk of causing adverse effects to human health and the environment. The first evidence for long-range transport of these substances came about when measurements in animals and the environment of the Arctic revealed the presence of POPs that were never produced there. The lack of reliable emissions data led to a number of modeling efforts used to explore hypotheses regarding the atmospheric transport of and deposition of POPs in the Arctic. For example, Wania and Mackay (1995, 1999) introduced multimedia global distribution models for persistent organic chemicals with a focus on transport and deposition to the Arctic. Then Scheringer (1996, 1997) developed evaluative models to assess global persistence and spatial range as end points in screening level assessments. These models and their results provided key insight both to international agencies, such as the United Nations, and to innovative scientists working independently to measure how POP concentrations vary with latitude. These new measurements provided important feedback that made it possible to develop the next generation of models by merging results from both the first generation of models and the new measurements.

Pharmacokinetic modeling. Andersen et al. (2005) describe examples of how integrated measurements and modeling have advanced risk assessment modeling by providing more insight on how intake of chemicals by humans relates to tissue dose and metabolism. Early pharmacokinetic models of the time course of absorption, distribution, metabolism, and excretion of chemicals relied on concepts buttressed by rudimentary information. By the 1950s, data on tissue volume, blood flow, and metabolic pathways were emerging, resulting in early physiologically based pharmacokinetic (PBPK) models. These models, in turn, led to the identification of key input variables (for example, blood flow through various tissues and metabolic parameters), the measurement of which would advance the models. The first use of a PBPK model in a formal risk assessment was for dichloromethane in 1987. Advances were made in assessment methods (for example, EPA’s reference concentration method) as well as in PBPK models of specific chemicals (acrylic acid, vinyl chloride, and dioxin). This iterative process continues today to inform risk assessments that can be used in regulation. It also provides a platform for more novel computational and biological systems approaches of the future (Anderson et al. 2005).

Comprehensive Everglades Restoration Plan. The planned restoration of the Florida Everglades is the largest ecosystem restoration effort ever undertaken in terms of it geographical extent and number of individual components. The NRC Committee on Restoration of the Greater Everglades Ecosystem, which was charged with providing scientific advice on this effort, describes the role that modeling and measurements should play in implementing an adaptive approach to restoration (NRC 2003). Under the committee’s vision, monitoring of

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

hydrological and ecological performance measures should be integrated with mechanistic modeling and experimentation to better understand how the Everglades functions and how the system will respond to management practices and external stresses. Because the individual components of the restoration plan will be staggered in time, the early components can be used as experiments to provide scientific feedback to guide and refine implementation of later components of the plan.

modeling could contribute to the regulatory process. If there is sufficient need for computational modeling, modelers must work with decision makers to define the goal of the model, the decisions it supports, and the groups that might use the model.

Addressing these questions is important for setting the direction of the model. As described in Chapter 2, legislative, regulatory, or policy mandates will often drive model development and implementation. For example, a legislative or policy mandate may require that EPA protect the most exposed individual, the most vulnerable individual, or reasonably highly exposed individual and that the agency consider long-term average exposure, the highest one-day exposure, the most exposed subpopulation, or the location of highest concentration. Indeed, legislative mandates may sometimes force development of new models or require major modifications to existing ones. This initial stage sets the direction for the conceptual model and the computational model development. The sidebar from Alice in Wonderland illustrates this message. If you do not know where you want to go, it may appear to others that the direction you take is not particularly important. The key goal of this initial phase is to identify whether modeling would be an effective tool for the problem at hand.

Potential uses of an environmental model include the following (Jakeman et al. 2006):

  • Long-term prediction (both extrapolating from the past and answering “what if” questions).

  • Short-term forecasting.

  • Interpolation (estimating variables that have not or cannot be measured directly).

  • Concise summarizing of data.

  • Data assessment (coverage, limitations, inconsistencies, and gaps).

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
  • Control system design (monitoring, diagnosis, decision making, and action taking).

Regulatory models are also used to do the following:

  • Help determine compliance with a particular regulation.

  • Evaluate a variety of alternative regulations.

  • Provide a general framework to assess compliance with multiple regulations.

  • Summarize available knowledge needed for regulatory decisions.

Insight from Alice’s Adventures in Wonderland by Lewis Carroll

Alice speaking to the Cheshire cat: “Would you tell me, please, which way I ought to go from here?”

Cheshire cat: “That depends a good deal on where you want to get to.”

Alice: I don’t much care where.”

Cheshire cat: “Then it doesn’t matter which way you go.”

Even if defining the model purpose appears to be a straightforward and easy step, it is often difficult to be clear about the purposes of an environmental model and its application domain. For scientists, the major objective is often to describe the processes dominating the behavior of the system, and for a decision maker, the objective might be to provide clear assessments of policy options. These motives are not mutually exclusive, but neither do they overlap completely. In addition, policy makers may want results for policy variables not directly represented in the model, or they may want results at scales not easily represented by a model. In any case, it is important to establish clearly the purpose and priorities of the specific model.

Specification of Modeling Context

After determining the purposes of the model, the modeler must develop specifications for the model context. This task involves addressing such questions as

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
  • At what temporal and spatial scales is the model to be applied? This question involves the grain (resolution in time and space) and the extent (spatial and temporal domain) at which the model is to be focused.

  • Who will be the major model users and what constraints does that imply for model application once developed? What is the level of expertise of the proposed users?

  • What type of input data must the model users provide? How can these data be obtained (from other models and measurements)?

  • What sources of data are available to support model evaluation?

  • What are the basic outputs needed and must they be constrained by a deterministic approach or is a probabilistic approach allowable? What additional outputs, although not strictly required, might be useful to enhance model transparency (for example, ability to explain it to various stakeholders and users) and flexibility (for example, capacity for the model to be modified and applied to situations for which it was not constructed)?

  • What level of reliability is required?

  • What evaluation criteria should be applied to determine the applicability of the model or of particular model components?

For example, when developing a cancer health assessment of a chemical, considerations include whether to use a linear or a nonlinear model. If the latter is chosen, the model specifications will need to be based on interpretation of the mode of action. A second example is the assessment of human exposure to mobile-source emissions of particulate matter. Here the model developers must work with others to determine whether the objective is to estimate cumulative exposure, time history of exposure, peak exposure, or another measure of exposure. Model developers and others must consider whether they need to consider particle mass, particle number, and particle volume as a metric of exposure. They must also consider the spatial and temporal resolution in the data and parameters that probably will be available for the model. Finally, if the goal is to create linkages to broader health assessments of particulate matter, they must make decisions on whether to consider mobile-source contributions only or mobile sources as a component of all sources of airborne particulate matter.

The mismatch between data needed by the model and data available to the model often results in failure of the model exercise, even if the

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

model itself may be an accurate representation of the science governing the behavior of the specific system. For example, air quality models require two major types of input: weather fields and emissions inventories. This input can be large (gigabytes of information) and impossible to obtain exclusively from measurements, so meteorological and emissions models estimate the input data and prepare the corresponding input files. Ideally, the input to the meteorological and emissions models would be based on actual measurements (for example, wind speed and direction in specific locations, vertical profiles of atmospheric properties, vehicle activity patterns, and emissions factors). Often, however, these models must use default inputs (for example, those based on emission factors from other parts of the country or even the world without taking into account the local conditions). Improving model inputs with measurements can be costly, and especially for emissions, measurement costs may overwhelm the actual modeling cost. Dealing with the issue of how to obtain the required inputs before developing the computational model and before building bridges with the measurement communities can make a substantial difference in the success of the modeling effort.

All the above questions apply to both pre- and post-regulatory models. Some specific questions probably will arise for each model, including the following:


For pre-regulatory planning models:

  • What range of plans and scenarios must be considered?

  • What array of impacts is to be included in the assessments of alternatives?

For post-regulatory planning and compliance models:

  • Is the required decision a “bright line” compliant/noncompliant one or is a broader view (for example risk of noncompliance) allowable?

  • What constraints are there on computational complexity? Will users insist on rapid assessments from the model (for example, does this need to be available in field situations) that preclude more advanced computational equipment?

Conceptual Model Formulation

A conceptual model formulates the basic model organization,

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

sometimes expressed graphically without the details of individual model components or assumptions. As discussed in Chapter 1, such an abstract representation provides the general structure of a system and the relationships within the system that are known or hypothesized to be important. Figure 3-2 provides an example of a conceptual model for assessing eutrophication. This conceptual model can be viewed as a map summarizing the structure of a model, the inputs, the state variables and outputs, and possibly the domain of applicability. Indeed, one of the critical roles of the conceptual model is to provide a visual description to decision makers, stakeholders, and interested parties of the model, including the fundamental relationships within the model and how inputs lead to outputs. Determination of an appropriate conceptual model relies first upon the problem formulation decisions discussed above as well as the decisions on the following:

  • What basic scientific principles are involved in the model (for example, areas of physics, chemistry, and biology that need to be considered on the basis of the objectives)? Is there agreement about these principles or does their inclusion potentially result in controversy (in which case, allowing for alternative assumptions might be necessary)?

  • Is an appropriate model formulation already extant?

  • What level of aggregation is appropriate to the model objectives? This question applies to the scales for the model (for example, spatial and temporal averaging may be needed) and the structure of model components (for example, including demographic structure or putting all ages into a single class).

  • What are the variables for the model (for example, what will it explicitly track in characterizing the system) and how are they related (often characterized by a box and arrow diagram, flow chart, influence diagram, or similar graphic)?

  • What are the means by which the variables will be expressed? Are they deterministic (discrete, continuous, nominal), stochastic (discrete, continuous, nominal), and spatially or temporally dependent or static?

  • What level of mechanistic detail is needed (for example, processes operating at what level should be included: cell, tissue, individual organism, population, and so forth)? Is a purely empirical approach (for example, a data-driven model, including many statistical ones) appropriate? Is a mixture of these necessary?

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

FIGURE 3-2 Conceptual model for assessing eutrophication in the European seas linking nutrient enrichments and its direct and indirect effects in the ecosystem. Source: EC 2004. Reprinted with permission; copyright 2004, the European Commission’s Joint Research Centre.

  • What are the model inputs and the scales at which the inputs will be provided?

There are distinct trade-offs in model development that should be addressed at the time of the conceptual model formulation. No one model can do everything. Development of a more comprehensive model will not necessarily resolve or even reduce all uncertainties in understanding and in predicting how a system will react. It is at this stage of model development that constraints, assumptions, and acceptability criteria should be established. Given financial or effort limitations, it is appropriate to set “stopping” criteria for when to decide that a model is sufficiently useful to be applied, even while acknowledging its limitations.

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

All the above trade-offs apply to both pre- and post-regulatory models. Some specific questions probably will arise for each model, including the following:


For pre-regulatory planning models:

  • How are alternative plans formulated or specified? Do they arise from modifications of a single plan (say, by varying constraints on structures allowable to be built) or are they chosen from a broad array of options?

  • What metrics are applied to compare and contrast alternative plans?

For post-regulatory implementation and compliance models:

  • What criteria determine compliance versus noncompliance and how do they relate to the model state variables?

  • What level of detail is needed in applications involving regulatory implementation?

Computational Model Development

This stage requires formulating the model explicitly by translating the model assumptions from the previous step into a mathematical formulation, by determining the detailed structure of the model, and by encoding the resulting model. This stage requires decisions about the following:

  • Model equations that determine the relationships between variables (rules, statements, equations, statistics) and account for the mathematical structure of the model (for example, static, dynamic, discrete, continuous).

  • Parameter estimations (from either data or underlying scientific assumptions) to determine input model parameters or distribution of such parameters in the case of a stochastic formulation.

  • Appropriate software design and engineering tools to encode and/or solve the model, appropriate computational algorithms, and appropriate model interface to ensure applicability by the user community.

  • Methods for analysis of model results, including graphic outputs and the capability to conduct sensitivity and uncertainty analysis.

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
  • Flexibility to modify model structure and inputs in the future as new data arise, alternative objectives are specified, or different regulations are assessed.

  • Documentation to allow for transparency of the model based on the needs of the user community and the potential for future modification. Such documentation maintains the history of major revisions of the model.

One critical issue is whether to revise an existing model or to develop a new one. “Model recycling” can save a huge development effort by applying a tested model to a purpose that is different from its original one. Furthermore, modelers often face the difficult decision between the development of one model that describes everything (the “swiss army knife” of models) and can be used for a variety of purposes and the development of multiple smaller models that have a common core but are developed separately for different purposes.

For example, the main purpose of the CMAQ model is to simulate concentrations of fine particulate matter and ozone in the lower atmosphere and to assist the analysis of the corresponding regulations. As a complex model, it describes the concentrations of more than 100 air pollutants in space and time. It has become a family of models (for example, CMAQ-MADRID and CMAQ-Hg) addressing a range of different air quality problems, including visibility reduction and acid deposition. Given that the different versions of CMAQ take advantage of the core of the model (atmospheric transport, gas-phase chemistry, and so forth) without violating any of the major assumptions of CMAQ, the strategy is a good one. For example, CMAQ has been extended (after some nontrivial modifications in its code) to address mercury (CMAQ-Hg). Although there are many gaps in our scientific understanding of the corresponding problem, CMAQ is an appropriate platform for such an extension. On the other hand, the model would require major redevelopment to address potential regulations of ultrafine particles (diameter less than 100 nanometers [nm]) due to numerical issues with its description of the particle-size distribution. Thus, there are limitations to the degree that CMAQ can be adapted.

Another example is the MOBILE model (Table 3-1), which has evolved from a tool for estimating regional motor-vehicle emissions inventories to a model used for estimating emissions on individual highway segments where instantaneous operating conditions of individual vehicles may be critical but that are not represented in the

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

model. As concluded by the NRC (2000), the farther MOBILE’s applications deviate from its original purpose of estimating aggregate regional emissions, the more difficult it becomes to verify the accuracy of its predictions. Because of the difficulty in developing a single motor-vehicle emissions model appropriate for all applications, the NRC (2000) recommended that EPA develop a toolkit of models based on a consistent data set and model interface. Such a toolkit would include an aggregated regional emissions component, a smaller scale model for simulating emissions along major highway corridors, and a microscale instantaneous emissions-modeling component for more transient and localized traffic conditions. For a toolkit approach, the type of motor-vehicle emissions model applied could better meet the characteristics of the problem while being consistent from one problem scale to another. Tierney (2004) described how EPA’s new mobile-source emissions model, known as the MOVES model, will address this and other issues raised by the NRC.

Additional considerations probably will arise for pre- and post-regulatory models.


For pre-regulatory planning models:

  • Provide methods to compare and contrast the implications of alternative plans, perhaps requiring the capability for exploratory analysis by the users of model outputs.

  • Provide methods either to vary the constraints on plans-scenarios or to vary the metrics to evaluate each plan.

  • Provide automated optimization methods to specify the highest ranked plan from a given set based on chosen criteria.

For post-regulatory implementation and compliance models:

  • Provide methods to modify inputs to determine how readily a noncompliant case might become a compliant one and vice versa.

  • Provide methods to ascertain the impacts of additional data on model results and assist users in determining the most effective methods to obtain such data (for example, methods to choose optimal locations for new data collection).

  • Understand whether models used for implementing regulation would be used widely by state and local governments and consulting firms that help those entities to develop implementation plans.

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

Modular Approaches for Environmental Model Development

The code of environmental models often can be written in a modular form. A module is an independent piece of code that forms a part of one or more models. Often, each module describes one process. For example, CMAQ includes modules for the description of horizontal and vertical advection, horizontal and vertical dispersion, gas-phase chemistry, aqueous-phase chemistry, aerosol thermodynamics and dynamics, plume chemistry effects, dry and wet deposition, and process analysis. This modular approach facilitates testing of the model (one can test the individual pieces separately) and reuse of the relevant modules in separate modeling projects. The parts of the model can be replaced with others without changing the overall structure of the model. There are also choices of modules for the same task. CMAQ allows its user to choose among three gas-phase chemistry mechanisms, depending on the specifics of the problem being modeled. The modular approach to CMAQ allows the level of complexity in the application to be aligned with the needs of the regulatory decisions. For example, the use of the mercury chemistry simulation capability of CMAQ is not necessary for ozone or particulate matter applications.

A major advantage of the modular model development approach is the ability to easily add or remove parts of the model, thus creating models of different complexity. For example, the full range of available modules (describing all potentially relevant processes) can be used, and then after quantifying the importance of each one of them for the specific application, the model can be simplified and used by removing the parts that have little or no effect on the results. For example, one can remove the cloud chemistry module from an application focusing on ozone episodes. The rest of the analysis can be done with the simplified model. Therefore, the modular approach and the resulting models of different degrees of complexity allow the user to satisfy the scientific requirements about quantifying the influence of the different processes and to avoid unnecessary complexity in the model used for the regulation. This approach allows modelers to defend their choices of excluding parts of the system from the analysis by allowing modelers to demonstrate the impacts of including or excluding various processes from the model. This approach also allows models to be updated more easily.

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

RECOMMENDATIONS

The committee offers several recommendations based on the discussion in this chapter. They deal with the interdependence of models and measurements, the model extrapolation, and the need for model parsimony.

The Interdependence of Models and Measurements

The interdependence of models and measurements is complex and iterative for several reasons. Measurements help to provide the conceptual basis of a model and inform model development, including parameter estimation. Measurements are also a critical tool for corroborating model results. Once developed, models can drive priorities for measurements that ultimately get used in modifying existing models or in developing new ones.

Measurement and model activities are often conducted in isolation. For example, modelers often add details to models without sufficient measurements to justify or confirm the importance of these changes. Likewise, field and laboratory scientists might expand their compilation of samples without understanding the utility of such information for modeling. Although environmental data systems serve a range of purposes, including compliance assessment, monitoring of trends in indicators, and basic research performance, the importance of models in the regulatory process requires measurements and models to be better integrated. Adaptive strategies that rely on iterations of measurements and modeling, such as those discussed in the 2003 NRC report titled Adaptive Monitoring and Assessment for the Comprehensive Everglades Restoration Plan, provide examples of how improved coordination might be achieved.

Recommendations

Using adaptive strategies to coordinate data collection and modeling should be a priority of decision makers and those responsible for regulatory model development and application. The interdependence of measurements and modeling needs to be fully considered as early as the conceptual model development phase. Developing adaptive strategies

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×

will benefit from the contributions of modelers, measurement experts, decision makers, and resource managers.

Model Parsimony

Models are always incomplete, and efforts to make them more complete can be problematic. As features and capabilities are added to a model, the cumulative effect on model performance needs to be evaluated carefully. Increasing the complexity of models without adequate consideration can introduce more model parameters with uncertain values, and decrease the potential for a model to be transparent and accessible to users and reviewers. It is often preferable to omit capabilities that do not improve model performance substantially. Even more problematic are models that accrue substantial uncertainties because they contain more parameters than can be estimated or calibrated with available observations.

Recommendations

Models used in the regulatory process should be no more complicated than is necessary to inform regulatory decisions. In the process of evaluating whether a model is suitable for its given application, there should be a critical evaluation of whether the model has been made unreasonably complicated. This evaluation should include how model developers and those that select a model for a particular application have addressed the trade-offs between the need for a given model application to be an accurate representation of the system of interest and the need for it to be reproducible, transparent, and useful for the regulatory decision at hand.

Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 83
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 84
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 85
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 86
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 87
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 88
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 89
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 90
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 91
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 92
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 93
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 94
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 95
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 96
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 97
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 98
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 99
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 100
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 101
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 102
Suggested Citation:"3 Model Development." National Research Council. 2007. Models in Environmental Regulatory Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/11972.
×
Page 103
Next: 4 Model Evaluation »
Models in Environmental Regulatory Decision Making Get This Book
×
Buy Paperback | $65.00 Buy Ebook | $54.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Many regulations issued by the U.S. Environmental Protection Agency (EPA) are based on the results of computer models. Models help EPA explain environmental phenomena in settings where direct observations are limited or unavailable, and anticipate the effects of agency policies on the environment, human health and the economy. Given the critical role played by models, the EPA asked the National Research Council to assess scientific issues related to the agency's selection and use of models in its decisions. The book recommends a series of guidelines and principles for improving agency models and decision-making processes. The centerpiece of the book's recommended vision is a life-cycle approach to model evaluation which includes peer review, corroboration of results, and other activities. This will enhance the agency's ability to respond to requirements from a 2001 law on information quality and improve policy development and implementation.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!