2
NEMS REQUIREMENTS

This chapter describes the critera that a National Energy Modeling System (NEMS) should meet to be of greatest value in energy policy analysis, strategic planning and decision making. Based on these requirements, an architecture for the NEMS is recommended in Chapter 3.

The committee's view of NEMS requirements was shaped by five major issues that are addressed in this chapter: (1) the role of policy models and their strengths and weaknesses; (2) the mission and functions of the U.S. Department of Energy (DOE) and the Energy Information Administration (EIA); (3) the current modeling capabilities of DOE/EIA; (4) the National Energy Strategy (NES) experience as it bears on NEMS requirements; and (5) future general directions in energy modeling. These considerations provide the primary basis for the NEMS requirements summarized at the end of this chapter.

THE ROLE OF MODELS IN POLICY ANALYSIS AND PLANNING.

Energy Policy-Related Models

Energy policy models can provide important information for national decision making (Ayres, 1978; AES, 1990; Hogan and Weyant, 1983; House and McLead, 1977; Koreisha and Stobaugh, 1979; Manne et al., 1985; Sweeney and Weyant, 1979). In fact, as stated by the committee in its first advisory report: ''…the committee believes that models can play a crucial role in enabling informed judgments and decisions to be made in matters of national energy policy. Thus, the committee considers it vital that DOE continue to develop and sustain capabilities for analyzing national energy issues using resources from



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 15
The National Energy Modeling System 2 NEMS REQUIREMENTS This chapter describes the critera that a National Energy Modeling System (NEMS) should meet to be of greatest value in energy policy analysis, strategic planning and decision making. Based on these requirements, an architecture for the NEMS is recommended in Chapter 3. The committee's view of NEMS requirements was shaped by five major issues that are addressed in this chapter: (1) the role of policy models and their strengths and weaknesses; (2) the mission and functions of the U.S. Department of Energy (DOE) and the Energy Information Administration (EIA); (3) the current modeling capabilities of DOE/EIA; (4) the National Energy Strategy (NES) experience as it bears on NEMS requirements; and (5) future general directions in energy modeling. These considerations provide the primary basis for the NEMS requirements summarized at the end of this chapter. THE ROLE OF MODELS IN POLICY ANALYSIS AND PLANNING. Energy Policy-Related Models Energy policy models can provide important information for national decision making (Ayres, 1978; AES, 1990; Hogan and Weyant, 1983; House and McLead, 1977; Koreisha and Stobaugh, 1979; Manne et al., 1985; Sweeney and Weyant, 1979). In fact, as stated by the committee in its first advisory report: ''…the committee believes that models can play a crucial role in enabling informed judgments and decisions to be made in matters of national energy policy. Thus, the committee considers it vital that DOE continue to develop and sustain capabilities for analyzing national energy issues using resources from

OCR for page 15
The National Energy Modeling System within the Department and from appropriate organizations in both the public and private sectors. President Bush articulated three primary areas of interest for energy policy in his introduction to the 1991 National Energy Strategy (NES): economic prosperity, environmental quality, and national security. The NES represents a new national effort, he said, to achieve “balance among our increasing need for energy at reasonable prices, our commitment to a safer, healthier environment, our determination to maintain an economy second to none, and our goal to reduce dependency by ourselves and our friends and allies on potentially unreliable energy supplies” (DOE, 1991a). These aims are all widely accepted. Yet despite such general agreement, federal energy policy making has been marked by deep disagreements among different interests and little movement forward on significant issues. The disagreements have concerned the means of achieving desired ends, the precise nature of the goals, and the forms of acceptable tradeoffs among sometimes conflicting objectives. High quality energy modeling can help bound these debates and move policy forward in two ways. First, modeling can provide insights to decision makers about the likely results of different policies. Second, it can help focus debates on scientific rather than ideological questions. Thus, energy modeling can help policy makers achieve greater consensus on proposed solutions and increase the likelihood of their adoption. Such model-assisted decision making has been used effectively in state and regional contexts over the last ten years. One important kind of energy decision is whether to invest in new supply-side energy resources, such as oil wells, coal mines, and power plants, or in end-use efficiency measures, such as better insulation for refrigerators. Models can provide quantitative measures to help guide the understanding of the economic, environmental, and security implications of alternative energy investments. They also can help to estimate the results of different types of public policies (such as tax incentives to producers, transporters, and consumers, or payments to utilities for improving end-use efficiency). Some of the policy options that DOE and others considered in the 1991 NES activity are covered later in this chapter to further illustrate this point. In considering the types of energy issues that models can help address, the committee distinguished three time horizons relevant to policy making: the short-term (up to about 2 years), the medium-term (up to about 25 years), and the long-term (beyond 25 years). For each time horizon a different modeling approach is needed. Policy actions directed at short-term issues are likely to be more reactive than proactive. Short-term models therefore must address such problems as disequilibrium phenomena and the macroeconomic effects of energy disruptions (e.g., embargoes of energy supplies or sudden price hikes). As explained below, most energy policy issues of national concern involve the medium term (up to about 25 years), such as the evaluation of alternative energy investments whose effects will not materialize in the short term. The emphasis in these cases is on market

OCR for page 15
The National Energy Modeling System clearing and relatively stable trends. In considering priorities for NEMS development, subsequent discussion identifies the mid-term modeling capability to be of highest priority. Long-term energy models projecting beyond 25 years are needed primarily for research and development (R&D) planning and for certain types of assessments, such as the environmental impacts of greenhouse gas emissions. The committee believes that the emphasis of long-term modeling should be relatively simple models and scenarios that incorporate fundamental technical and economic principles. While the distinction between mid-term and long-term approaches is not clear-cut, a key characteristic of long-term models is their ability to accommodate fundamental changes in current trends that are generally not anticipated by mid-term models. For some types of policy analysis, the use of both short-term and mid-term models (or both mid-term and long-term models) may offer the best approach. For example, an analysis of policy options to deal with future oil market disturbances might draw heavily on short-term models (e.g., those related to the draw-down of strategic petroleum reserves and emergency fuel switching capabilities) in conjunction with mid-term models that estimate the types and levels of future fuel use (including the effects of policy initiatives such as gasoline taxes or energy efficiency incentives). In general, therefore, all three modeling approaches are ultimately required to address the full range of energy policy issues. General Approaches To Modeling Two general approaches can be used to construct a model: the comprehensive strategy and the iterative problem-directed strategy. The following comparison is based closely on Initiation of Integration (EPRI, 1977). The iterative, problem-directed strategy emphasizes in all its phases the interaction between the client and the analysts. The analysis activities assemble information and transfer it to the client in a form that is tailored to the client's problems. The client provides feedback to the analysts on important problems and information needs that can be used to guide the direction of future analysis. The iterative problem-directed strategy is conducted in three phases. The first phase involves description of the decision alternatives available to the client and identification of information crucial to making the optimal decision. The second phase is analytical. The analyst assembles from the available models, data, and information the combination that is best suited to the client's needs. Where uncertainty is present and is crucial to the comparison of alternatives, a probabilistic formulation of the analysis is structured. The resulting analytical structure is then used, in close collaboration with the client, to generate quantitative projections of key variables bearing on the client's problem and quantitative comparisons of the decision alternatives from the individual perspectives of the client and other concerned parties. The third and final phase of this strategy is the transfer of the analytical output to the client and feedback from the client as to the needs for additional analysis. If the first and second phases have been done in close communication with the client, the third phase is straightforward.

OCR for page 15
The National Energy Modeling System The efficiency and quality of the iterative problem-directed strategy is greatly enhanced if the development of analytical tools is done within a common framework of accounting conventions and variable definitions. The comprehensive modeling strategy centers on the development of a comprehensive model system covering a wide range of issues of potential concern to clients and providing forecast information of common concern to many clients. The principal product of this strategy is thus the models and particularly the forecast information they generate. The comprehensive modeling strategy begins with the formulation of an overall analytical framework within which specific modules representing energy supply, demand and other information can be integrated. When the development of these submodels and the overall integrating model are complete, then the submodels are integrated to produce forecasts and analyses of interest to clients. In the comprehensive modeling strategy, the nature of the model is primarily determined by the research process, thus requiring less involvement of the client. This however makes the transfer of the information to the client more difficult. Given the need to build a national energy modeling system, it seems inevitable that significant elements of the comprehensive modeling strategy will have to be employed. Nonetheless the committee is favorably disposed towards certain elements of the iterative problem-directed strategy, for instance the close involvement of analysts with clients, and the focus on decision problems of importance to those clients. It follows that the NEMS model builders should be continuously well-informed about the types of policy initiatives and R&D strategies that may be informed by the use of the model and this awareness should shape design and implementation decisions about the NEMS. It also follows that the NEMS should be built as a modular system with several versions of the same module differing in size and complexity. In this way, if detailed analyses are required of only one energy sector, then that particular module can be very detailed while most of the other modules can be relatively simple. In this way, the NEMS as employed in analysis can be kept relatively small and transparent. The Benefits of Models The primary advantage of models is that they enable the user to better understand phenomena of interest, based on available information. A model provides a framework in which to analyze different combinations and various kinds of information (e.g., on the magnitude of variables and on cause-and-effect relationships). Such a framework provides insights that are not obtained by analyzing less organized, separate pieces of information. Counterintuitive but valid results from modeling sometimes offer a deeper understanding of the phenomena studied. Models also force the analyst to be explicit about the assumptions of the analysis. Such assumptions, like those about parameter values or structural relationships, can then be debated or verified by independent observers. Similarly, results of a given model can be replicated from a common set of input values, and differences in results can, in principle, be traced to differences in assumptions. Often, the effort to specify needed model

OCR for page 15
The National Energy Modeling System assumptions reveals gaps in essential information (whether in data or in concepts) that the analysis requires. The well-defined link between assumptions and results means that models provide a convenient way to examine changes in outcomes as inputs change. This is a valuable attribute in policy analysis: sensitivity analysis can reveal the nature of tradeoffs involved in policy interventions and can help to identify an appropriate level of policy intervention. Models can also help verify results obtained from other forms of analysis. Where the conclusions of different analyses are inconsistent, models can sometimes reveal the reasons for the inconsistency, such as differences in implied assumptions. The Limitations of Models While analytical models have come to play an increasingly critical role in policy analysis and planning, the inappropriate use of models, or the failure to understand their limitations, also must be of major concern in developing a NEMS. Models are simplified descriptions of reality. The simplified descriptions represent those aspects of the real phenomena that the modeler believes to be the most important for the issues under consideration. Any simplification may prove to have omitted significant considerations, which can produce invalid results. Models impose a fixed structure on an analysis, including the structure that describes the process of change (which is usually based on historical experience). Input assumptions also are often based on past experience. However, future reality may differ in critical ways from the history embodied in the model. In particular, models that fail to incorporate the recognition that the policies they analyze can themselves cause future structural changes may be especially misleading. Because models produce numerical results, users often tend to attach greater value to such results than are warranted. Often the numbers take on a life of their own, as if they represented reality rather than a highly simplified characterization of reality. Thus, models may lead to a narrowing of the analyst's perspective, rather than to the discerning of new insights. The aura of reality that attaches to model results often leads users to expect too much, and model builders to promise too much, from this kind of analysis. The desire for realism leads model builders to push toward increasing levels of complexity, placing at risk the simplicity that many times adds most value to a model. Excessively complex models become “black boxes” whose underlying assumptions may no longer be apparent and whose results have implications no longer easy to discern either by the model builders or model users. It is important to remember that model outputs are not “facts” and models are not reality. In general, several elements affect models' accuracy: (1) the overall structure of the model; (2) the omission of relevant factors that may significantly affect the modeled

OCR for page 15
The National Energy Modeling System outcome; (3) the lack of dynamics in the model structure (i.e., assumptions, functional relationships and parameter estimates that are invariant in the model but which actually change in the real world); and (4) the use of data and information of inherently poor quality. Another serious limitation of most models is that they fail to consider uncertainties explicitly. This issue is especially important for NEMS and is discussed more fully below. THE MISSION AND FUNCTIONS OF DOE AND EIA DOE was established in 1977 to manage U.S. energy programs more effectively than its predecessor organizations, and to coordinate national energy policy. The responsibility of the Secretary of Energy is to bring knowledge of energy issues to bear on the formulation of national policy. An important requirement for DOE--and thus for NEMS--is to provide the information and analytical support that the Secretary, a principal client for a NEMS, requires. Within DOE, the EIA was established and charged with broad responsibility for providing energy-related data and analysis. The EIA was set up to be an independent and objective source of energy-related information, including forecasts of energy trends. By law, the EIA Administrator has independent authority to collect information, conduct analyses, and publish reports (P.L. 95-91). As currently envisioned, the NEMS will be developed and maintained by EIA, both for its own use and to provide services to DOE and others. Thus, EIA would also be a client of the NEMS. Appendix C provides more detail on the missions and roles of DOE and EIA, including roles as strategic planner, information provider, and energy R&D manager. Each of these roles is described briefly below. All of these functions should influence the design and use of the NEMS. Strategic Analysis Strategic planning involves the examination of likely future U.S. energy trends anticipating emerging policy issues; developing options for policy interventions to address these issues; evaluating and comparing the efficacy of such options; and describing and justifying the policy initiatives proposed. Model-supported analysis can be valuable in several of these activities. The energy outlook and emerging energy problems can be clarified by model-generated projections, such as those for alternative future scenarios. Models can be used to explore the future implications of current or altered trends and of the perceived problems. In strategic analysis, the effects of policy options are simulated and compared to the outcomes of continued current policy. In some instances, the outcomes of alternative policy options can be estimated quantitatively and the options thus made more amenable to political debate.

OCR for page 15
The National Energy Modeling System The NEMS should become the model framework for strategic planning within DOE. It should be capable of generating alternative future scenarios and testing the effects of policy initiatives. For the most part, such strategic analysis would concern the mid-range horizon of 2 to 25 years. Data Collection and Information Dissemination The DOE, particularly the EIA, is charged with collecting and reporting information about the national energy system, including production, transport, consumption, and pricing series of major energy forms, and descriptive statistics on the energy production, transportation, and consumption infrastructure. These data supply the building blocks in constructing energy models. The models, in turn, can be used to help assess deficiencies in the existing data. For example, end-use demand models may require data for specific processes or end-use subsectors that are not currently available. In light of DOE's mission, moreover, energy information has always included the extension of historical data series through energy projections, and models are an important tool in such forecasting. Special studies of anomalies in energy-related data and relationships suggested by data sets (e.g., concerning the delivery capacity of the natural gas supply system, a subject of current interest) also are supported by models. The NEMS will depend greatly on DOE data, as well as data from other sources, and might be used by EIA as the model framework to generate projections and conduct special studies. To help satisfy the full information mission of EIA, however, the NEMS will need to be augmented by at least a short-term projection model for annual forecasts. R&D Program Planning As administrator of a major energy R&D program, DOE must make annual appraisals of its program, evaluate competing R&D projects, and allocate scarce manpower and funds according to some determination of R&D priorities. The traditional approach to allocating R&D resources is to perform a cost-benefit analysis, in which the potential accomplishments of competing projects, adjusted according to their probabilities of success, are compared to their costs. One approach to assessing the potential accomplishment of a new technology is to evaluate its impact on a reference scenario of the future. Because of the attention to detail required, it is likely that the benefits of specific R&D proposals will have to be evaluated outside of the general NEMS framework (e.g., see the case study of magnetic levitation transportation in Appendix D). Nevertheless, some assumptions used in these analyses can be derived from the general scenarios generated by the NEMS. Prices for competitive fuels, requirements for energy services, and economic and financial variables are examples. Insofar as a generally applicable reference scenario for the national energy outlook (generated by NEMS) is employed for input to the detailed R&D evaluations, consistency and comparability among such analyses will be enhanced. This use of NEMS could also

OCR for page 15
The National Energy Modeling System generate feedback: the more detailed evaluations of the R&D programs may reveal shortcomings in the general model. In summary, the responsibilities of the DOE and EIA clearly encompass modeling for the three time horizons earlier noted, i.e., short-term (up to roughly 2 years); mid-term (up to about 25 years); and long-term (beyond 25 years). The consistency that needs to be established between short-and mid-term modeling is also recognized as is the link between mid-and long-term modeling capabilities. More specific recommendations about modeling priorities and requirements for the different time horizons are offered later in the report. Current Modeling Capabilities Within DOE and EIA. From its inception, DOE/EIA has made use of a wide variety of models (EIA, 1990b). They range from comprehensive macro-models of the U.S. economy to very particular models concerning electric utilities' investment decisions for power plants. Over the years the EIA and the DOE have contributed to energy modeling through early and continued development of large scale energy models and through active participation and funding of the Stanford Energy Modeling Forum (EMF) and the MIT Model Assessment Laboratory. For example, the Project Independence System (PIES) was initially developed in 1974 by the Federal Energy Administration (FEA), and later called the Mid-term Energy Forecasting System (MEFS). A number of other modeling systems were developed in the late 1970s and early 1980s by DOE and EIA. The DOE program offices and national laboratories also developed models useful for their particular technology interests. However, during the 1980s modeling activities declined as the pace and priorities of programs at DOE were redefined by the Administration of President Reagan. The EIA, of course, continued to meet its statutory obligations and published its Annual Energy Outlook with the use of its forecasting models. These past energy modeling and data collection efforts addressed to varying degrees the energy supply, demand and conversion sectors and associated economic measures of the U.S. economy; but continuity of efforts at DOE was lost during most of the 1980s until attention to the first National Energy Strategy analysis exercise stimulated a renewed interest at the Department in policy models. A description of some of the current models appears in Appendix E. The committee doesn't view this collection of models as representing an adequate energy policy modeling system with regard to the requirements set forth in this report. The models have been used primarily to forecast future conditions of energy supply and demand based on alternative assumptions about such external parameters as world oil prices and economic growth rates, and to explore the impacts of such policy initiatives as oil import tariffs and efficiency standards, with reference to a baseline or projection for the future. Several EIA models have been used to obtain external parameter and data estimates, but they are not part of an integrated modeling system. Typically, EIA forecasts are updated annually and serve as a baseline for special forecasting and projects requested in the following year. One trend in such EIA modeling has been increased modularity and decomposition of the modeling systems. A related trend has been a greater diversity of modeling methods.

OCR for page 15
The National Energy Modeling System These trends have developed as the coverage of systems and the complexity of energy issues have increased (Conti and Shaw, 1988). EIA's quarterly Short-Term Energy Outlook presents two-year forecasts of energy supply and demand, produced using the Short-Term Integrated Forecasting System (STIFS). This modeling system has been used by EIA since 1979 for short-term forecasting and related analysis. Occasionally a model is developed for a one-time project. The set of models are continually modified as energy markets and the issues addressed change. The current EIA mid-term modeling system is the Intermediate Future Forecasting System (IFFS), developed in 1982 (Conti and Shaw, 1988). IFFS partitions the energy system into fuel supply, conversion, and end-use sectors and then solves for supply-demand equilibrium by successively and repeatedly invoking the modules that represent these sectors. The model provides forecasts year-by-year and has a forecast horizon of 2010. Fundamental assumptions for this modeling system are world crude oil price and a baseline macroeconomic forecast. In parallel with EIA activities, DOE's Office of Policy, Planning and Analysis supports an independent integrating model called Fossil2 (AES, 1990; DOE, 1991c). This is the integrating model employed for the recent NES analysis. Fossil2 is a mid-term systems dynamics model that has evolved over a decade. In the recent NES exercise, it was also used to obtain long-term projections out to the year 2030. In summary, the models available within DOE, especially the EIA models used to make mid-range projections and the Fossil2 model used in policy analysis, already provide significant modeling capability. In the recent analytical effort leading to the NES, however, a number of policy issues proved beyond these models' scope. Thus, the committee finds that the set of DOE and EIA models used in the NES exercise does not constitute an adequate NEMS meeting the requirements for policy analysis described in this report. (This is further elaborated in Appendix B.) The suite of existing models within EIA, however, appears to provide a reasonable starting point for the proposed design of NEMS described in Chapter 3. However, in developing the NEMS, the committee believes the existing models should be modified to address particular high-priority policy issues. Furthermore, Chapter 3 describes the capabilities of some of the models and how they need to be modified. NEMS IN THE BROAD CONTEXT OF POLICY ANALYSIS The NEMS to be developed by DOE/EIA must operate in the broader context of an analytical system that for convenience we call the National Energy Analysis System. As illustrated in Figure 2-1, comprehensive models are only one of several tools important for national energy analysis and decision making. Analytic results will derive not only from the models incorporated in NEMS, but also from various data sets, other external models and a wide variety of independent judgments and assumptions. The dashed line in Figure 2-1 indicates that the EIA is involved in all aspects of the analysis process, but its main focus is on models and data sets.

OCR for page 15
The National Energy Modeling System Figure 2-1 Scheme of the National Energy Analysis System and the EIA's scope within it. The development of an appropriate set of models for the NEMS requires a good understanding of the broader context within which these models will be used. Figure 2-2 shows very schematically how the NEMS will interface with parts of DOE, as well as with other public and private organizations in the United States and abroad. NEMS must establish links both within and outside of DOE to ensure that it remains responsive and able to fulfill its mission. The questions posed by policy makers, and the kinds of information needed for energy policy decisions, moreover, must drive the design of the NEMS. Formal as well as informal processes must be established to ensure that communication between NEMS modelers and their clients is good. The development of the NEMS and associated data bases will be dynamic, evolving in response to changing policy issues and needs. Without effective communication, modelers are unlikely to anticipate and develop the relevant tools and information in a timely way. Keeping NEMS Outward Looking Significant data resources and modeling capabilities exist in the private and the nonprofit sectors, and in other state, regional, and federal agencies. To avoid duplication of effort,

OCR for page 15
The National Energy Modeling System Figure 2-2 Scheme of the interface between the NEMS and the National Energy Analysis System. and to take advantage of the specialized expertise and capabilities of other organizations, the NEMS design should allow outside models to interface with those developed by DOE/EIA. While EIA's use of external models will depend on particular circumstances, it is important to recognize that models outside DOE/EIA constitute a valuable resource that should be considered in developing the NEMS. Like any modeling system, NEMS will be dependent on its input data and underlying assumptions. NEMS needs valid, current data on resources, technologies, and consumer demands, and sound assumptions about the projection of current trends. By maintaining an outward-looking philosophy, NEMS can draw on the detailed work of outside study groups and detailed sector models, and can help identify and support improvements in key areas. EIA should identify annually those energy supply, demand, and technology areas that require a new look, as, for example, the impacts of the emerging demand-side management movement in the electric power industry (OTA, 1985). DOE should initiate and participate in broadly based efforts to compile available information on such issues and to increase knowledge about them in the modeling community. The Energy Modeling Forum (EMF) at Stanford University provides an example of how this might be done (Sweeney and Weyant, 1979).

OCR for page 15
The National Energy Modeling System Quantification of Uncertainty The importance of representing uncertainty in energy policy models is especially emphasized by the committee. This section discusses the difficulties such uncertainty poses for energy modeling, and approaches that might be incorporated in the NEMS. Because policy analysis models are necessarily built with both incomplete theoretical foundations and incomplete empirical data, both the modelers and users of results should be concerned about the uncertainties inherent in the results of energy policy modeling. Such uncertainties may arise from a number of sources including data errors, exogenous forecast errors, future technology characteristics, specification of dependencies, estimation error, and modeling behavior. A more detailed treatment of these and related issues may be found in Morgan and Henrion (1990). Most contemporary models of the energy system are deterministic. They require specific input variables, and in theory, a level of detail that almost always greatly exceeds the precision with which real-world variables are measured. For example, the use of energy by the entire U.S. household population is described in DOE models based on the survey reports of only a few thousand respondents with virtually no physical measurements to confirm the data. As noted earlier, model structures or algorithms represent relationships that are simplifications of historically observed or assumed relationships among the variables. For example, energy-related consumer decisions typically are represented as economic choices among available options exclusive of other needs or desires that may be competing for the same investment. The increasing range of possible outcomes that proceeds from alternative decisions as the future unfolds is usually ignored. For example, many energy models assume a fixed-price track for the world oil market, that continues to guide future model decisions, even after intermediate results of the model run, such as those relating to oil demand, have become inconsistent with that track. These and other simplifications introduce large uncertainties in the results presented to decision makers as a basis for action. Uncertainty is inherent in the nature of models and cannot be eliminated. Nor should it be ignored. It is important to deal explicitly with uncertainty for a number of reasons. First, one desires actions that are robust across a variety of possible situations. An explicit treatment of forecast variability allows an evaluation not only of the success of a policy in some “expected” or most likely future, but also its success in, or sensitivity to, a range of alternative futures. With the recognition of uncertainty comes a better appreciation of risk. If risks can be characterized and quantified, an appropriate risk management strategy can be undertaken. For example, by quantifying the likely potential variability in electricity demand forecasts, the Northwest Power Planning Council (NPPC) has been able to consider the robustness

OCR for page 15
The National Energy Modeling System of their decisions across possible levels of regional economic activity. Because future loads are uncertain, the NPPC developed a range of estimates plus analytical models to evaluate the impacts of uncertainties on the distribution of future power systems costs. Both expected costs and uncertainty in costs were estimated for alternative resources strategies using Monte Carlo methods to combine hundreds of future scenarios into probabilistic statements about key parameters of system performance. As a result of such analyses, in one case NPPC chose to reduce risk in the face of uncertain future demand levels by investing in higher-cost resources in the near future (Northwest Power Planning Council, 1991). Hirst and Schweitzer (1990) give other examples of electric utilities' use of strategic models to accommodate for uncertainty and risk in their long-range planning. New applications of uncertainty analysis to advanced energy technologies and R&D planning also are emerging that may be of significance for mid-term and long-term energy modeling. Diewekar and Rubin (1991), for example, have developed probabilistic models that allow DOE to estimate the performance and cost uncertainties for advanced power generation systems. Illustrative results, based in part on the technical judgments of DOE experts, show a much broader range of cost and performance uncertainty than reflected in current deterministic estimates of the type now used in energy policy models (Frey and Rubin, 1991). To the extent that policy analysis results depend strongly on assumptions about future energy technologies (as often they do), failure to adequately incorporate uncertainties could lead to very misleading results. For long-term analysis consideration of uncertainties is especially critical. In their analysis of global warming, for example, Manne and Richels (1990) determine an optimal energy path for the United States for the next 20 years given three possible scenarios for U.S. energy development starting in 2010. The scenario choice in 2010 is determined by a judgment about the severity of future global warming. Again, the explicit treatment of uncertainty engenders considerations about the robustness of potential policy options across a range of future situations. In all cases, attempts to explicitly quantify current ignorance may also help direct the collection of additional data, or the revision of model structures, to reduce the most critical forms of uncertainty for future analyses. The NEMS can be designed to accommodate various methods for explicitly treating uncertainty in energy models, including sensitivity analysis, closed-form statistical approaches, Monte Carlo methods, and alternate model formulations. A modular structure for NEMS can facilitate uncertainty analysis, first, by allowing many of the full modules to be replaced by corresponding reduced-form modules, thus considerably reducing the run time, and secondly, by allowing the use of alternative module structures for a particular segment of the energy system (for further discussion of uncertainty see Cohen, 1986; Hausman, 1981; Heyde and Cohen, 1985; Hodges, 1987; Leamer, 1983; Malliaris and Brock, 1982; and Sims, 1982, 1984).

OCR for page 15
The National Energy Modeling System Long-Term Forecasting While the focus of the committee's attention has been modeling for the medium-term, there is also a clear need to consider longer time periods. A mid-term projection horizon of roughly two to 25 years contemplates less than a complete change in the current generation of adult decision makers, which suggests some continuity of behavior in household customs and business practices. Similarly, much of the energy infrastructure of 25 years from now already exists or will result from currently observed trends. Important new technologies that are not evident in some form today are unlikely to penetrate the marketplace significantly within 25 years; and structural changes in the makeup of the economy are less likely to diverge from current trends within that time. None of these assertions can be made with the same degree of confidence for a longer-term projection horizon. As a result, the simple extension of a mid-term energy model for an additional decade or two might suggest a degree of certainty out of proportion to the confidence that can generally be placed in the method. In fact, the “inertia” in the model's relationships, rather than adding information, may give rise to misleading results. For long-term projections, the past loses relevance as a guide. Since long-range projections are needed for some types of policy analysis, an entirely different method may be required. The notion of predictability, central to the conventional meaning of projections and forecasts, should be de-emphasized, while that of developmental constructs or working hypotheses should be stressed. Long-term modeling methods should be based on a critical selection of those relationships and trends that are most likely to persist (e.g., certain demographic trends). Judgments about the long-term direction of important generic factors, such as the efficiency of energy conversion, should be made based on fundamental physical or economic principles, rather than by extrapolating current experience. Factors that are fixed in the short-or medium-term must become variables in the long term, including social and political factors that shape the economy today. In developing the long-term capabilities of the NEMS, therefore, it would be well to begin with the broadest notions about alternative futures. For instance, the rapid development and deployment of information and communications technology over the next decade or two may make it possible for employees in many service industries to perform most of their work at home, thus eliminating a large fraction of automobile usage. The South Coast Air Quality Management District's 20-year master plan for the Los Angeles Basin specifies air quality standards that cannot be met by conventional internal combustion engines or their foreseeable successors. If followed, this 20-year plan might lead to a future dominated by electric or alternative fuel vehicles, with profound implications for gasoline usage, and economic and environmental outcomes. Because over the long term there may be significant changes in structural relations (i.e., in those relationships often treated as invariant to medium forecasting), long-term models must be able to anticipate structural changes. In these cases, the parameters and

OCR for page 15
The National Energy Modeling System coefficients that define the various relationships should be obtained from basic technological information, exogenous models, or subject matter knowledge. The attributes of simplicity and transparency that are desirable characteristics for all models are especially important for long-term models. In addition, a third characteristic applies exclusively to long-term models: Judgment-Based. Because fundamental structural changes are likely over the long term, greater emphasis needs to be placed on judgments and reasoning about basic assumptions rather than on extrapolation of current trends or the use of mathematical relationships applicable for mid-term modeling. The main ingredients of long-term modeling are core assumptions and judgments, representing the modeler's outlook on the context in which projected trends will develop. If the core assumptions fail to capture the reality of the future context, the modeling used will make little difference to the quality or utility of the results. The results from long-term models should not be interpreted as predictions of future events but rather as tools to provide insights or indications about where the basic assumptions might lead. A common practice in long-term modeling is to conduct scenario analyses to assess the impacts and implications of changes in regulations, laws, or the assumptions themselves. Useful information can be derived from such scenarios. For example, important issues that recur in many scenarios can be identified. Extreme situations and the most threatening or beneficial consequences of potential trends can be explored. Gross requirements for energy resources and cumulative consumption quantities can be estimated and compared to resource availability. Pollutant loadings can be calculated and compared to the environment's assimilative capacity. To some extent, the longer-term impacts of proposed policy initiatives can be described. The scenarios, however, should not be treated as quantitatively as mid-range projections. It is advisable that some scenarios also be free of the imposition of “plausibility” or “sanity” checks. In thinking about longer-term models, it is useful to remember that conditions that today seem implausible could be normal in the distant future. Different degrees of modeling detail are necessary in long-term models compared to the mid-term model; however, these differences depend on the peculiarities of the sector being modelled. Considerable thought and analysis is necessary before the modeler can choose rationally the types of methods used for long-term forecasting. Modelers disagree on the extent of disaggregation that is useful with such long-term models. Those modelers emphasizing the extreme uncertainty of such models suggest simple, highly aggregated models. Others feel that disaggregated models, especially on the demand side, can be quite useful and illuminating. Most modelers agree, though, that these models will be different than those use for the mid term. The committee heard a number of presentations on efforts outside of EIA that address long-term energy issues (see Appendix E). For example, Manne and Richels (1990) and Edmonds (1990) are modeling on a time scale of 100 years in studying the global carbon cycle and its connection to the energy system. Others discuss option theory and hedging

OCR for page 15
The National Energy Modeling System as strategies to guard against an uncertain future. And, of course, there is much work directed toward modeling long-term climate and environmental change. DOE and EIA have a rich assortment of activities to look out to in embarking on the development of a long-range modeling capability. While the committee believes that the highest priority must be given to the development of a NEMS mid-term modeling, EIA also should devote some resources now to studying activities on long-range modeling outside DOE/EIA. Based on this review and the needs of the NES process, DOE/EIA should devote additional resources to the development or acquisition of appropriate long-range modeling capability, consistent with the discussion above. NEMS REQUIREMENTS Based on the previous considerations, the committee recommends that the requirements for a viable and useful NEMS include the following: Output measures of concern to decision makers: NEMS output should including key economic, environmental, and national security measures relevant to policy decisions (as outlined below), plus income and regional distribution characterizations as appropriate. Economic measures: Key economic outcomes include measures of changes in consumers' and producers' surpluses from a change in the supply curve of energy as a means of determining the societal cost savings associated with such a change (Willig, 1976), gross national product (GNP), and federal budget effects. Since the value of energy services is an important issue in contemporary energy policy discussions, an attempt should be made to report this variable as well. Environmental measures: Ideally, the environmental characteristics modeled should include both flows and stocks of major identified pollutants and, to the extent possible, measures of their economic or physical effects. Measures of environmental effects are very imprecise at this time. The committee believes that initially NEMS should estimate the total emissions of primary pollutants and related measures of environmental impact (see Table 2-2). Later, secondary or indirect impacts should also be quantified (e.g., land use needs for biomass). Thus, NEMS development should be capable of supporting models of environmental effects developed by other groups or agencies, such as the Environmental Protection Agency. Ultimately it may be desirable to incorporate simple models of environmental effects directly into the NEMS. Energy Security measures: Energy security is a derived characteristic. NEMS should attempt to measure the probabilistic effects on the economy and the environment of various potential disruptions of the energy system. The security concern that has received the greatest attention is that associated with disruption of the oil market. One proxy variable that was proposed in the NES to measure the vulnerability of the U.S. economy to oil disruption is the aggregate cost of oil relative to GNP. The larger the ratio of oil cost to GNP, the more vulnerable is the economy to oil price

OCR for page 15
The National Energy Modeling System TABLE 2-2 Environmental Issues Relevant to NEMS General Concern Problem Area Primary Measures Air Pollution Global warming CO2, CH4   Acid rain SO2, NOx   Photochemical smong VOC, NOx   Air toxies Metals, organics Water Pollution Oil spills Crude, products Waste Disposal Nuclear wastes High-level waste   Non-nuclear wasters Ash, sludge, other Natural Resources Land use Acreage   Water use Consumption Health and Safety Nuclear accidents (a)   Other energy-related accidents (a)   Electromagnetic fields (60Hz) (a) (a) To be defined in later studies. or supply disruptions. Deeper understanding of the effects of oil disruptions is required, and could be gained from more focused studies relating to the role of oil in the U.S. economy and the structure of the world oil trade. Once these studies are completed, a better proxy measure of oil vulnerability probably can be designed and incorporated in the NEMS. Regional and international measures: In general, output from NEMS will be required for the United States as a whole. Many issues, however, will have regional or international implications. Environmental impacts and transfers of wealth resulting from U.S. policy initiatives are only two examples. Most energy production and consumption patterns in the United States vary significantly by region. From the outset the NEMS will need some regional disaggregation of results (e.g., for some economic and environmental impacts). The specific needs will be defined by the problem-oriented focus recommended below. In addition, NEMS will have to be sensitive to international issues that influence U.S. energy modeling (e.g., world oil price) and that are affected by U.S. energy policy (e.g., global warming). Income distribution measures: In certain cases, energy policy initiatives may have different effects on classes of people with different incomes. While the first implementation of NEMS presumably will not disaggregate results by income

OCR for page 15
The National Energy Modeling System distribution, it will be desirable that it do so in future analyses for some policy initiatives.2 The committee also believes it would be appropriate to pursue research on approaches for summarizing the values or tradeoffs for different groups likely to be affected by energy policy decisions. Use of the iterative problem-directed approach. Given the diversity of energy policy issues of concern to DOE/EIA the development of the NEMS will have to incorporate some elements of the comprehensive modeling approach. Nonetheless, it is the opinion of the committee that NEMS model construction should be driven to the extent possible by an iterative problem-directed approach. Thus, the model builders should be continually aware of the types of policy initiatives that are to be analyzed, and this awareness should determine the priority design and implementation decisions for the NEMS. Use of a modular structure. The design of NEMS should be modular to facilitate its use, and, to enable model builders to work on particular aspects of the model independently. This specialization of labor requires that modules be designed with specific input and output requirements so that all modules subsequently can be linked together to derive an equilibrium solution. It will also be necessary to maintain overall flexibility and to design efficient algorithms to link the modules. Integration of engineering and economic approaches. NEMS models should strive to project actual producer and consumer behavior as well as explore technically efficient outcomes. Integrating engineering and economic decision making algorithms would permit the NEMS to better project actual consumer and producer responses to changing events, to existing regulations, and to proposed policies. Efforts should also be made to incorporate alternative models of expectation formation. Many energy decisions by households and businesses depend on expectations about future energy prices. To the extent possible, the NEMS should be designed to eventually incorporate alternative models of price expectations, namely, myopic, extrapolative, and rational. Incorporation of uncertainty analysis. There are several methods for accommodating uncertainty in energy models, including scenario analysis, parameter variations, alternative model structures, closed-form statistical approaches, and Monte Carlo methods. The committee recommends that NEMS be designed to accommodate the explicit treatment and estimation of uncertainty using a variety of methods. In developing and using the NEMS, uncertainty should be explicitly addressed in two ways. Initially, the existence of uncertainty in analytical results should be brought to the attention of decision makers in every analytical report. Uncertainty also should be made 2    In Chapter 3, the architecture proposed for NEMS indicates that certain analyses, such as calculating the economic distributional impacts, would be analyzed with models separate from NEMS (see Figures 3-1 and 3-4).

OCR for page 15
The National Energy Modeling System explicit by describing the limitations of data and algorithms in the most accurate way possible. Focus initial development on the mid-term. To address the full range of energy policy issues confronting DOE, model results generally are desired for three time horizons: (1) the short-term, roughly 2 years; (2) the medium-term up to about 25 years; and (3) the long-term, beyond 25 years. Although the NEMS is intended to cover all time periods, the committee agrees that its initial focus should be on the medium term. However, because the impacts of many policy initiatives, and many of the R&D questions of concern to DOE, will involve evaluations beyond a 25-year time horizon, DOE should ultimately develop adequate long-term modeling capability. The problems of long-term modeling are unique. While there is a body of experience available, original research will likely be required to develop a meaningful modeling capability within the NEMS to evaluate the long-term consequences of policy options. The treatment of uncertainty and of alternative outcomes become very significant in such evaluations. It is recommended that the DOE/EIA create a group to develop a long-term modeling capability. In addition to relating this capability to the medium-term modeling system in a consistent fashion, this group should incorporate the expertise of external experts and groups who already focus on the very long term in energy modeling. Short-term modeling already is used by EIA in preparing its short-term energy projections. Because the focus of short-term models is on economic cycles, base-year anomalies, supply disruptions, and monthly or quarterly outputs, the methods employed are inconsistent with longer-term model structures. Development of short-term capability for the NEMS framework will thus need to focus on compatibility and consistency with the mid-range framework to the greatest extent possible. At the outset, however, the short-term capability will likely continue to be separate from the mid-range modeling system. Use of widely available computers. To maximize the likelihood that NEMS will be usable by the many different parties at interest in the national energy debate, the NEMS should be constructed to run on one or more widely available hardware configurations (e.g., a “personal” computer or a more powerful workstation unless this would significantly restrict the NEMS content). In that case development for workstations would be regrettable but preferable to mainframe computers. The portability of the NEMS code across different computer operating systems also will require attention in this effort. Outreach to the broader energy analysis community. The NEMS should be designed so that it invites interfacing with existing or future models outside of the NEMS. A modular design should facilitate such outreach. Interaction with outside peer groups. Critical reviews from outside peers, participation in workshops, oversight committees and subjecting the NEMS development work to open interaction will all lead to higher standards and greater credibility of the NEMS.

OCR for page 15
The National Energy Modeling System Transparency of results. The transparency of the NEMS model should enable users other than DOE/EIA modelers to use parts or all of the NEMS independently. Fast turnaround. A version of the NEMS should be designed to operate in reduced form to facilitate repetitive runs or simplified analytical efforts. Rapid response should be facilitated by reduced-form modeling and abbreviated administrative procedures (in contrast to the detailed procedures currently in place for EIA models). Quality control. In all aspects of NEMS development, procedures for maintaining high levels of quality control will be required. These procedures will include verification and documentation of coding, data and model validity, as appropriate. Adequate flexibility, however, must be preserved to ensure that NEMS development and use does not become stifled by overly elaborate procedures and specifications. The suggested structure and architecture for the NEMS that can meet these requirements are outlined in the following chapter. FINDINGS AND RECOMMENDATIONS The primary committee findings that emerge from this chapter are the following: The set of EIA models reviewed by the committee at the beginning of this study constitutes a reasonable starting point for developing a National Energy Modeling System. However, considerable development will be needed to attain a modeling system satisfying the requirements outlined in this report. The NEMS program, once established, should complement, interact with and draw upon analyses from a variety of other public and private groups that contribute to policy analysis. Successful development of a NEMS will require the Secretary of Energy and the EIA Administrator to establish and foster an organizational environment that is outward-looking and ensures greater intellectual and institutional commitment to its development and maintenance. Although the NEMS needs to satisfy a number of requirements, the committee especially recommends the following: The NEMS should be designed to estimate the economic, environmental, and security implications of alternative energy policies. The NEMS architecture should be modular and should provide for quick turnaround applications. The NEMS should be designed to allow analysts to incorporate uncertainty explicitly.

OCR for page 15
The National Energy Modeling System The primary focus of NEMS development should be on capabilities that address the mid-term time horizon of up to about 25 years. A problem-focused approach should guide the development of all NEMS capabilities, to the extent possible. Furthermore, with regard to long-term models, the committee recommends: The EIA should create a group to develop a long-term modeling and analysis capability.

OCR for page 15
The National Energy Modeling System This page in the original is blank.