4
Use of Models

MODELING SCENARIOS

Modeling of baselines and future project actions is a standard practice of evaluating impacts. Both biological opinions relied on the use of modeling scenarios (known as Studies) provided by the Operations Criteria and Plan (OCAP) biological assessment (BA) (http://www.usbr.gov/mp/cvo/ocap_page.html), although the extent to which such results were used in each biological opinion and in the formulation of RPAs varied significantly. The “proposed action” with reference to ESA is the continued operation of the CVP and SWP with additional operational and structural changes (USBR, 2008, Table 2-1) to the system. The U.S. Bureau of Reclamation (USBR) and the California Department of Water Resources (DWR) provided the results of the modeling conducted for simulating baseline conditions, future system components, operational strategies, and the water supply demands. In addition to simulating the water-supply deliveries of the project, the modeling also attempted to mimic the project operations associated with the regulatory environments described in operating criteria described in D-1485, D-1641,CVPIA Section 3406 (b)(2) and the Environmental Water Account (EWA) (USBR, 2008). A major difference in the current and future scenarios is the extent to which EWA is used. The purpose of EWA was to enable diversion of water by the SWP and CVP from the delta to be reduced at times to benefit fish species while minimizing uncompensated loss of water to SWP and CVP contractors (USBR, 2008, Chapter 2). The EWA is intended to replace the water loss due to pumping curtailments by purchasing surface water and groundwater from willing sellers and through increasing the flexibility of operations. The simulations include both a “full EWA” characterizing the full use of EWA assets as well as a “limited EWA” focusing only on a limited number of assets. The EWA is currently under review to determine its future (FWS, 2008, p. 34) and the RPA actions were not based on it.

Another factor that changed from current to future conditions is the way water demand by CVP/SWP users is simulated. Demands have been pre-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 28
4 Use of Models MODELING SCENARIOS Modeling of baselines and future project actions is a standard practice of evaluating impacts. Both biological opinions relied on the use of modeling sce- narios (known as Studies) provided by the Operations Criteria and Plan (OCAP) biological assessment (BA) (http://www.usbr.gov/mp/cvo/ocap_page.html), al- though the extent to which such results were used in each biological opinion and in the formulation of RPAs varied significantly. The “proposed action” with reference to ESA is the continued operation of the CVP and SWP with addi- tional operational and structural changes (USBR, 2008, Table 2-1) to the system. The U.S. Bureau of Reclamation (USBR) and the California Department of Wa- ter Resources (DWR) provided the results of the modeling conducted for simu- lating baseline conditions, future system components, operational strategies, and the water supply demands. In addition to simulating the water-supply deliveries of the project, the modeling also attempted to mimic the project operations asso- ciated with the regulatory environments described in operating criteria described in D-1485, D-1641,CVPIA Section 3406 (b)(2) and the Environmental Water Account (EWA) (USBR, 2008). A major difference in the current and future scenarios is the extent to which EWA is used. The purpose of EWA was to en- able diversion of water by the SWP and CVP from the delta to be reduced at times to benefit fish species while minimizing uncompensated loss of water to SWP and CVP contractors (USBR, 2008, Chapter 2). The EWA is intended to replace the water loss due to pumping curtailments by purchasing surface water and groundwater from willing sellers and through increasing the flexibility of operations. The simulations include both a “full EWA” characterizing the full use of EWA assets as well as a “limited EWA” focusing only on a limited num- ber of assets. The EWA is currently under review to determine its future (FWS, 2008, p. 34) and the RPA actions were not based on it. Another factor that changed from current to future conditions is the way water demand by CVP/SWP users is simulated. Demands have been pre- 28

OCR for page 28
Use of Models 29 processed using either contractual amounts and/or level of development (exist- ing versus future). Some demands were assumed to be fixed at contractual amounts whereas in other cases they varied according to the hydrologic condi- tions. This topic will be considered in the committee’s second report. While several study scenarios were developed for the OCAP biological as- sessment (USBR, 2008), the use of modeling results in the biological opinions was largely limited to a smaller set of scenarios (Table 4-1). Study 7.0 describes the existing condition (circa 2005), whereas Study 7.1 presents the existing condition demands with near future facilities as well as the projected modification to EWA. Study 8 describes the future condition corre- sponding to the year 2030 (USBR, 2008, pp. 9-33, 9-53, 9-54). Study series 9 constitutes a future condition representing modified hydrology (warm and warmer, dry and wet) along with a projected sea level rise of one foot. CENTRAL ISSUES CONCERNING MODEL USE IN THE BIOLOGICAL OPINIONS The USFWS and NMFS supplemented the modeling results provided by USBR and DWR with their own modeling efforts and available science on the implications of management actions on species. The primary suite of models provided to FWS and NMFS include (USBR, 2008, Chapter 9): (a) Operations and hydrodynamic models: CalSim-II, CalLite, the Delta Simulation Model II (DSM2), including particle-tracking models (PTMs, which also are considered as surrogates for biological models) TABLE 4-1 Key scenarios used for biological opinions of FWS and NMFS Study Level of Environmental Future Climate and Development Water Account Project Sea Level Facilities1 (Year) (EWA) Rise 7.0 2005 Full EWA No No 7.1 2005 Limited EWA Yes No 8.0 2030 Limited EWA Yes No 9.0-9.5 2030 Same as in Yes Yes 2 Study 8.0 1 Future project features include South Delta Improvement Program (Stage 1), Freeport Regional Water Project, California Aqueduct and Delta-Mendota Canal intertie 2 According to the OCAP BA (USBR, 2008), Study suite 9 is identical to Study 8.0 except for climate change and sea-level rise

OCR for page 28
30 Threatened and Endangered Fishes in California’s Bay-Delta (b) Temperature models: Reclamation Temperature, SRWQM, and Feather River Mode (c) Biological models: Reclamation Mortality, and SALMOD The modeling framework used by the agencies is diagrammed in Figure 4-1. The USFWS, in its biological opinion, used available results from a combi- nation of tools and data sources, including CalSim-II, DSM2-PTM, DAYFLOW historical flows, and statistical models based on observational data and particle- tracking simulations (FWS, 2008, p. 204). NMFS analyses included results from coupled CalSim-II simulations with various water-quality and biological models for a few of the life stages (NMFS, 2009, p. 64). The CalSim-II model, the primary tool used to evaluate the water-resources implication of the proposed actions, was developed by the DWR and the USBR to simulate water storage and supply, streamflows, and delta export capability for the Central Valley Project (CVP) and the State Water Project (SWP). Cal- Sim-II simulates water deliveries and the regulatory environment associated with the water-resources system north of the delta and south of the delta using a FIGURE 4-1 Modeling framework used in NMFS and USFWS biological opinions and RPAs.

OCR for page 28
Use of Models 31 single time step (one month) optimization procedure based on a linear pro- gramming algorithm. CalSim-II represents the best available planning model for the CVP-SWP system, according to a CALFED Science Program peer review by Close et al. (2003) (USBR, 2008, p. 9-4). However, many users have suggested that its primary limitation is its monthly time step, and the model should be used primarily for comparative analysis between scenarios and discouraged its use for absolute predictions (Ferreira et al., 2005; USBR, 2008, Chapter 9). In response to the peer review by Close et al. (2003), DWR and USBR provided a list of development priorities (Table 2, DWR/USBR, 2004), including the use of a daily time step, but it is not clear how many of such planned improvements have been incorporated into the version of CalSim-II used in the biological opinions. Several other tools and models were central in effects analysis and develop- ing RPAs, including hydrodynamic and water-quality (DSM2, USBR’s tempera- ture, SRWQM), habitat (SALMOD), and statistical and particle-tracking models (salvage, DSM2-PTM). Some of these models have already been evaluated in the literature for their individual strengths and limitations, though some (SALMOD and USBR’s mortality models) have not yet been formally peer re- viewed. We first review some of the challenges of applying these individual models in the determination of RPAs, and then focus on examining the model- ing process, including how the models contributed to the development of RPAs, and where the uncertainties and vulnerabilities in that process lie. Model Scale and Management Implications Very generally, the tiered modeling approach (Figure 4-1) applied the re- sults of CalSim-II as input to various hydrodynamic and ecological models to predict impacts of project operations and, to a very limited extent, to explore RPAs. At one level, model simulations were also used or performed to investi- gate the feasibility of some proposed actions. For example, CalSim-II was used at the planning level to investigate whether the USBR could meet the 1.9 MAF (at the end of September) required by actions I.2.3 and I.2.4 (maintaining cold water supplies necessary for egg incubation for the following summer’s cohort of winter-run), and to recommend storage conservation in severe and extended droughts (NMFS, 2009, p. 596). Similarly, examination of CalSim results and hydrologic records demonstrated to the agencies that the first year of a drought sequence is particularly critical to storage and operations in the following drought year (NMFS, 2009, p. 596). The benefits of using models at this plan- ning level, especially given the importance of water-year types, is clear, and there is little controversy about this application of the models.

OCR for page 28
32 Threatened and Endangered Fishes in California’s Bay-Delta At another level, model scenarios were examined to investigate the relation- ships between operations and impacts on various life stages of the fish across the water-year types and operations scenarios. For example, NMFS used DWR’s Delta Survival Model (Greene, 2008) to estimate mortality of smolts associated with three CalSim-II Study scenarios (7.0, 7.1, 8.0). The USFWS used statisti- cal models of salvage and total entrainment (Grimaldo et al., 2009; Kimmerer, 2008) to investigate the effects of proposed operations by comparing actual and predicted salvage and entrainment losses under modeled OMR flows (FWS, 2008, p. 211). While some challenges exist in linking models in this tiered approach (see next section), concerns and controversies appear to be largely directed at the various forms of statistical relationships of salvage versus OMR flows, extrapo- lation of these relationships that describe impacts on single life stages to assess the population impacts on species, and the use of biological models without full consideration of their underlying uncertainties. In particular, this nested se- quence of statistical models does not allow for uncertainties at one step to influ- ence predictions at the next step. As a result, some of the RPA actions, espe- cially those involving X2 and OMR flow triggers, are based on less reliable sci- entific and modeling foundations than others. In these cases, the incomplete data and resolution of the models do not closely match the resolution of the ac- tions. Adequacy of Current Models Life-cycle Models Both agencies have been criticized for the lack of adequate life-cycle mod- els to address population level responses (e.g., Deriso, 2009; Hilborn, 2009; Manly, 2009). Nonlinear and compensatory relationships between different life- history stages are common in many fish species. Moreover, many life-history traits exhibit significant patterns of autocorrelation, such that changes in one life-history trait induce or cause related changes in others. These patterns can most effectively be understood through integrated analyses conducted in a mod- eling framework that represents the complete life cycle. However, complete life-cycle models were not used in either biological opinion to evaluate the ef- fects of changes in operations. The agencies acknowledge that further model development is required, including the “cooperative development of a salmonid life-cycle model acceptable to NMFS, Reclamation (USBR), CDFG, and DWR” (NMSF biological opinion, p. 584). While one life-cycle model (Interactive Ob-

OCR for page 28
Use of Models 33 ject-Oriented Salmon simulation) was available for winter-run salmon from the OCAP BA (USBR, 2008), this model was rejected based on model resolution and data limitation issues (NMFS, 2009, p. 65). Similarly, a better life-cycle model for delta smelt is critically needed (PBS&J, 2008). Such life-cycle mod- els for delta smelt are currently under development. The committee recom- mends that development of such models be given a high priority within the agencies. The committee also encourages the agencies to develop several differ- ent modeling approaches to enable the results of models with different structure and assumptions to be compared. When multiple models agree, the confidence in their predictions is increased. Particle-Tracking Models (PTMs) Particle-tracking models (PTMs) are models that treat eggs and larval fishes as if they were particles and simulate their movements based on hydraulic mod- els of flows. Criticisms have applied to the use of PTMs, which rely on some key assumptions (e.g., neutral buoyancy, no active swimming) that have been challenged at least for some life stages (Kimmerer and Nobriga, 2008) on the basis that fish live and move in three dimensions. Other limitations of the use of PTMs in this case include the reliance on the one-dimensional DSM2, use of random-walks to simulate lateral movements, and the lack of simulation of fish behavior. In view of these limitations, PTMs as used in this case may not be suitable for predicting the movement of fish of some life stages (juvenile and adults) where behavior becomes relevant to the question of potential entrainment (Kimmerer and Nobriga, 2008). The NMFS acknowledges these limitations, noting that “The acoustic tagging studies also indicate that fish behavior is com- plex, with fish exhibiting behavior that is not captured by the ‘tidal surfing’ model utilized as one of the options in the PTM simulations. Fish made their way downstream in a way that was more complicated than simply riding the tide, and no discernable phase of the tide had greater net downstream movement than another” (NMFS, 2009, p. 651). However, while fish seldom behave like passive particles, results based on passive particles can provide insights. For example, the NMFS used a combina- tion of models to simulate mortality rates of salmonids for three CalSim-II sce- narios. The results were used to compare the inter- and intra-annual impacts of the three scenarios (NMFS, 2009, p. 381). Further, the agencies advocate im- proving the model through further study, such as Action iV.2.2, which includes an acoustic tag experiment in part to evaluate action benefits and in part to im- prove PTM results (USBR, 2008, p. 645). Thus, while there is uncertainty re-

OCR for page 28
34 Threatened and Endangered Fishes in California’s Bay-Delta garding the accuracy of the mortality losses, the use of the models in a compara- tive way is probably acceptable. However, it should be made clear how the model is used, and the explicit consideration of the PTM assumptions and uncer- tainties should be more clearly documented in the biological opinions. Although there has not been an assessment of the degree to which these limitations affect the conclusions, PTM results were used for RPA development. Although the DSM2 has been calibrated adequately for OMR flows, there is no clear evidence concerning the accuracy of the PTM’s ability to simulate smelt entrainment in relation to how the models are used for jeopardy determination and RPA development. This is particularly important because a number of ac- tions driven by the RPAs recommend trigger values for OMR to curtail exports. As discussed in a later section, the science surrounding these OMR triggers is less clear than for many other aspects of the RPAs, and this trigger may result in significant water requirements. The committee’s recommendations for improv- ing the modeling and associated science are intended to improve the best science available to the agencies. The committee will address such improvements in greater detail in its second report. Other Biological Models The NMFS used other biological models to simulate the effects of opera- tions on various life stages of salmon. These models involve several key as- sumptions and data limitations that influence the reliability of their results. For example, SALMOD, developed by the USGS, was used by the NMFS to investigate the population level responses of the freshwater life stages to habi- tat changes caused by project operations (NMFS, 2009, p. 269). A variety of weekly averaged inputs are required, including streamflow, water temperature, and number and distribution of adult spawners (USBR, 2008, p. 9-25). This model provides some valuable insight, but requires greater consideration of the model assumptions (e.g., linear stream, habitat as primary limiting factor, inde- pendence of food resources on flow and temperature, density independence for some life stages) and uncertainties. Otherwise, the use of this model is limited to comparative, rather than absolute, analysis of RPA actions. Further, it would be important to investigate the sensitivity of the model to initial conditions and input data, particularly those prone to measurement error (e.g., number and dis- tribution of spawners) to provide some indication of the reliability of model out- puts. While SALMOD has not been thoroughly peer-reviewed, criticisms of similar modeling approaches (e.g., NRC, 2008) have highlighted some key is- sues with habitat-suitability models (e.g., the need for greater clarity concerning

OCR for page 28
Use of Models 35 the assumption that habitat is a limiting factor and the need for a thorough as- sessment of the representativeness of the areas sampled) and have provided ex- tensive discussions of the use of models in an adaptive-management approach, which is relevant to this committee’s recommendations. Finally, the NMFS acknowledges that SALMOD is most appropriately applied to large populations that are not sensitive to individual variability and environmental stochasticity (NMFS, 2009, p. 270), which means that the predictions for the relatively small population in the delta river system are subject to considerable uncertainty. The uncertainties again highlight the need for an adaptive management approach. The NMFS also used results from the USBR’s salmon mortality model (Hydrologic Consultants, Inc., 1996) to examine daily salmon spawning losses for early life stages (pre-spawned eggs, fertilized eggs, and pre-emergent fry) due to exposure of high temperatures. Temperature-exposure mortality criteria for the three life stages are combined with modeled temperature predictions and spawning distribution data to compute percents of salmon spawning losses. Because simulations of river temperatures are run on a daily or shorter time step, downscaling of monthly CalSim-II data is required (USBR, 2008, Attachment H-1). Moreover, the monthly temperature models do not adequately capture the range of daily temperature variability (USBR, 2008, pp. 9-109). In addition, several assumptions (e.g., density independence) and important data limitations (USBR, 2008, pp. L-6, L-7) challenge the reliability of this model. Finally, while this model has been applied in other systems, it is not thoroughly peer reviewed and no analysis of sensitivity or uncertainty has been performed. Ad- dressing these model shortcomings would help increase confidence in the analy- ses. Developing, Evaluating, and Applying Best Available Models As the agencies work within the constraints of best available science, some recognition of the adequacy and reliability of the models should be reflected in the management decisions by making them adaptive. The following five fac- tors, in particular, need better documentation. 1. Incompatible temporal resolution and implications for management decisions. The individual models used in this tiered analysis approach have a broad range of temporal resolutions (Figure 4-1). Care must be exercised in such

OCR for page 28
36 Threatened and Endangered Fishes in California’s Bay-Delta situations so that the linkages of models with different temporal and spatial reso- lutions do not result in propagation of large errors that may influence decisions derived from the modeling results. For example, CalSim-II uses a monthly time step whereas the DSM2 uses a 15-minute time step. Although the tidal boundary condition in DSM2 is pre-processed at 15-minutes, average monthly flow, simu- lated by CalSim-II, is provided as the upstream flow boundary condition at many delta inflow points. The linkage of CalSim-II and DSM2 attempts to smooth out the step change in monthly simulated flows (USBR, 2008, pp. 9-14, 9-15), but this is not necessarily adequate to simulate the fluctuations of flows within the month. The use of the monthly time step certainly could have a sig- nificant influence on such performance measures as OMR flows, particularly when such flows are recommended in RPAs for triggering export curtailments. USFWS and NMFS should provide a comparison of daily versus monthly aver- age simulations of DSM2 for a historical period to ascertain the reliability of using monthly CalSim output as input to DSM2. The incompatibility of temporal resolutions is particularly important given that flows in the delta are strongly influenced by tides. The flows at such loca- tions as Old River and Middle River are characterized by two flood-ebb cycles per day, with positive and negative values of much larger magnitude than the average net flow at these locations (Gartrell, 2010). In view of the fact that OMR flows have sub-hourly hydrodynamic components, averaging over a longer period such as 5 to 14 days to define the thresholds in the implementation of the RPAs could produce unnecessary changes in water exports. The use of monthly average flows produced by CalSim-II could further add to the concerns regarding the recommended thresholds of OMR flows. In view of these model- ing uncertainties, further clarification as to how the modeled OMR flows were used for jeopardy determination and hence for the development and implementa- tion of RPAs is needed. 2. Inconsistent use of baselines. Both biological opinions use historical data along with modeling results of the CALSIM-II scenarios. Study 7.0, which represents the existing condition, is expected to be closest to historical conditions. However, important differences between the two (historical and existing conditions) could exist due to differ- ences in demands and more importantly due to deviations in operations. Be- cause of the simplifying assumptions used in CalSim-II historical simulations, the FWS BO opted to use actual historical data to develop their baseline (FWS, 2008, p. 206) and continued to compare historical data with the modeling results

OCR for page 28
Use of Models 37 of the numerous scenarios described above (see, for example, Figures E-3 through E-19). The results suggest that often, actual data are very different in magnitude in comparison to Study 7.0 and furthermore, most scenarios (Studies 7, 7.1, 8, and study series 9) are clumped together with relatively small differences between them in relation to the magnitude of differences with the historical data. In view of these differences, the validation of Study 7.0 and consequently others, be- comes even more important for the purpose of RPA development. The use of historical data to make inferences is very typical and appropriate in the biological opinions. However, since the evaluation of project actions and the development of RPAs are based on the evaluation of modeling scenarios, which appear to greatly differ from historical data, a comparison of the two sets of data (historical and simulated) may incur errors in interpretation. The com- mittee recommends that the biological opinions provide a better justification for the reasonableness of the baseline scenario, Study 7.0, as well as the comparison of scenario results with historical data. 3. Challenges in calibrating and validating any of the models to historical observations and operations. It is a standard practice to ensure the appropriate use of models through the processes of calibration and testing (ASTM, 2004; NRC 2008). Validation of CalSim-II is described in Appendix U of the OCAP BA (USBR, 2008), which provides a comparison of Study 7.0 (existing condition) with the recent histori- cal data. A review of those results shows that there are significant deviations of the historical data from the simulated storages and exports that may be of the same magnitude as the differences between the scenarios being evaluated. Thus, while the tool itself performs well, some questions remain regarding the gross nature of generalized rules used in CalSim-II to operate CVP and SWP systems, relative to actual variability of dynamic operations (USBR, 2008, pages 9-4). In their peer review of the CalSim-II model, Close et al. (2003) suggested that “Given present and anticipated uses of CalSim-II, the model should be cali- brated, tested, and documented for “absolute” or non-comparative uses.” It is not clear if the agencies that developed the model have responded to this sugges- tion in a comprehensive manner. As emphasized above, a clear presentation of the realism of Study 7.0 with respect to recent operations or observations would help avoid the criticism as to the results of Study 7.0 as well as other derivatives of it (Studies 7.1, 8.0 and series 9).

OCR for page 28
38 Threatened and Endangered Fishes in California’s Bay-Delta The OCAP BA (USBR, 2008) provides sufficient information on the cali- bration and testing of temperature models, and the time steps vary among mod- els, although all used the monthly output of CalSim-II in predictions. Thus, they appear to be adequate for predicting temperature variation and making compari- sons at the monthly time scale. Information on the calibration of DSM2 and PTM is provided in part by DWR, which has been posted online (http://modeling.water.ca.gov/delta/studies/validation2000/) results of the cali- bration of this 1-D, hydrodynamic model of the delta. Based on the information provided, it appears to adequately mimic the historical data at a daily time-scale. However, the DSM2 simulations should demonstrate that the range of negative OMR flows used for calibration covers the high negative flows simulated by CalSim-II for future scenarios. There has been an attempt to test PTM (Wilbur, 2001), but clearly this tool needs further improvements. Wilbur (2001) reports that the existing velocity profiles used in PTM consistently over-predict the field observations (i.e., the predicted velocities exceed the observed velocities). In addition, with the potential for changes in the historical patterns of cli- mate and hydrology, calibrating models with historical data alone may be less meaningful for projection of future operations. Thus, in addition to providing support for model improvement and adaptive management, a more robust moni- toring program will also support calibration and testing of models with more relevant representation of the current and future system. For example, drought- induced low flows of the past several years provide opportunities to calibrate and test models under infrequent but foreseeable conditions. Realistic modeling of the system that incorporates what actually happens in an operational setting with climate outlook will be important in the future. The biological models such as USBR’s mortality model and SALMOD are essentially uncalibrated for the system, and further concerns about these models were addressed in previous sections. 4. Challenges of the Tiered Modeling Approach. Temperature, OMR flows, and X2 performance measures are particularly challenged by the tiered modeling approach, with limitations related to data availability and inconsistency in model resolution (spatial and temporal) and complexity (USBR, 2008, pp. 9-31). However, the use of models may still be beneficial in planning and triggering adaptive management needs. For example, for NMFS implementation of Action II.2 (Lower American River Temperature Management), forecasts will be used to simulate operations and compliance with thermal criteria for specific life stages in months when salmon would be present

OCR for page 28
Use of Models 39 (NMFS, 2009, p. 614). However, if the USBR determines that it cannot meet the temperature requirement, and can demonstrate this through modeling of alloca- tions and delivery schedules, consultation with the NMFS will occur. In this example, modeling results are used to evaluate the feasibility of meeting criteria, rather than trying to derive direct loss estimates. The RPA then leads to a proc- ess for adaptive management of the temperature operations based on updates to the hydrologic information. Thus, despite the particularly challenging example of managing temperature, the use of models appears to have allowed for flexibil- ity. However, no qualitative or quantitative analysis of the magnitude of errors across these model linkages and the resulting uncertainties are presented. While not required for the justification of RPAs, failing to consider error propagation across the models makes it difficult to evaluate the reliability of meeting the RPAs and their ability to provide the intended benefits. 5. Lack of an integrative analysis of RPAs Numerous RPA actions proposed in both biological opinions cover new projects as well as operational changes. However, the information provided to the committee did not include a comprehensive analysis of all RPA actions, ei- ther individually or, more important, jointly, with respect to their ability to re- duce the risks to the fish or to estimate system-wide water requirements. Clearly, the agencies lacked properly linked operations/hydrodynamic/biological models at the appropriate scales for RPA development. The agencies should be complimented for using historical data as well as best available science when modeling was not adequate. However, the proposed RPAs could incur signifi- cant water supply costs, and there should be an attempt to provide an integrative analysis of the RPAs with quantitative tools. The committee also acknowledges the challenges associated with estimating water requirements for some RPAs, particularly those based on adaptive management strategies, but explicit and transparent consideration of water requirements and biological benefits of spe- cific actions and of subsets of actions would provide the basis for a smoother implementation of the RPAs. The committee recommends that the agencies consider investigating the use of CalSim-II and other quantitative tools (e.g., PTM, life-cycle models) to simu- late appropriate RPA actions of both biological opinions. These linked models would allow an integrated evaluation of the biological benefits and water re- quirements of individual actions and suites of actions, and the identification of potential species conflicts among the RPAs. Although not required by the ESA,

OCR for page 28
40 Threatened and Endangered Fishes in California’s Bay-Delta such an integrative analysis would be helpful to all concerned to evaluate the degree to which the RPAs are likely to produce biological benefits and to quan- tify the water requirements to those who might be affected by the future actions of the two biological opinions. In addition to further model development, efforts to improve documentation of model use would be beneficial. Documentation should include a record of the decisions, assumptions, and limitations of the models (e.g., NRC, 2008). Thus, we find that, while used appropriately in this analysis, the PTM and biological models for both salmon and smelt should be further developed, evalu- ated, and documented. The models show promise for being quantitative tools that would allow for examination of alternative ideas about key relationships underlying the RPAs. In addition, complete life-cycle models capable of being linked to these other models should be developed. Although developing, testing, and evaluating such models would require a significant investment, the commit- tee judges that the investment would be worthwhile in the long term. CONCLUSION Modeling is useful for understanding the system as well as predicting future performance. As long as modelers understand and accurately convey the uncer- tainties of models, they can provide valuable information for making decisions. The committee reviewed the models the agencies used to determine to what de- gree they used the models in developing the RPAs. The biological opinions have used results of a variety of operations, hydrodynamic, and biological models currently available to them for RPA development. However, the agencies have not developed a comprehensive modeling strategy that includes the development of new models (e.g., life-cycle and movement models that link behavior and hydrology); such models may have provided important additional information for the development of RPAs. Nonetheless, the agencies should be compli- mented for combining the available modeling results with historical observa- tions and peer-reviewed literature. The committee also compliments the agen- cies for the extensive discussion and presentation of the rationale for the particu- lar types of actions proposed in the RPAs. The committee concluded that as far as they went, despite flaws, the indi- vidual models were scientifically justified, but that they needed improvements and that they did not go far enough toward an integrated analysis of the RPAs. The committee has raised several important issues related to the modeling proc- ess used, including the model scale and management information; the adequacy of models, particularly the particle-tracking model and the lack of life-cycle

OCR for page 28
Use of Models 41 models; incompatibilities in both temporal and spatial scales among the models and between model output and the scale of the RPA actions; the use of base- lines; inadequate calibration and testing of modeling tools (in some cases); and inadequate model documentation. A more-thorough, integrative evaluation of RPA actions with respect to their likelihood of reducing adverse effects on the listed fishes and their likely economic consequences, coupled with clear docu- mentation would improve the credibility and perhaps the acceptance of the RPAs. Thus the committee concluded that improving the models by making them more realistic and by better matching the scale of their outputs to the scale of the actions, and by extending the modeling to be more comprehensive and to include features such as fish life cycles would improve the agencies’ abilities to assess risks to the fishes, to fine-tune various actions, and to predict the effects of the actions. Three-dimensional models are more expensive and time- consuming than simpler models, but they can contribute valuable understanding if used appropriately (e.g., Gross et al., 1999; Gross et al., 2009). In addition, the committee concludes that opportunities exist for developing a framework to improve the credibility, accountability, and utility of models used in implementing the RPAs. The framework will be particularly important for some of the more-complex actions, such as those involving Shasta and San Joaquin storage and flows, which rely heavily on model predictions. The com- mittee plans to address such issues, including the framework mentioned above, in more detail in its second report.