|
Items in cart [1] |
Questions? Call 888-624-8373 |
PAPERBACK PDF BOOK Free Resources |
||||||||||||||||
|
||||||||||||||||
|
|
|||||||||||||||
| ||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 114
4
Risk Methodology
The U.S. Army Corps of Engineers (USAGE) has adopted new risk and
uncertainty analysis procedures for project evaluation that explicitly include un-
certainties in the hydrology, hydraulics, and economics of a planning study
(USAGE, EC 1105-2-205, 1994) (hereafter referred to as EC 1105-2-2051. This
procedure represents an extension of the traditional paradigm for flood control
project planning and community flood protection evaluation. USACE observed
that the new risk and uncertainty methodology is similar to present practice but
differs in that uncertainty is explicitly quantified and integrated into the analysis
(USAGE, EC 1 105-2-205, 1994).
The 1994 Alternatives Report (USAGE, Sacramento District, 1994a) indi-
cated that USAGE's analysis now considers "varying degrees of uncertainty in
the causes of flooding, such as inflow to Folsom Reservoir, regulated outflow-
frequency relationships for Folsom Dam, river stages, and levee stability." The
methodology computes the risk of flooding due to combinations of hydrologic
events, hydrologic parameter uncertainty, uncertainty in stage-discharge rela-
tions, and levee performance.
This change in methodology is important to the American River Watershed
Investigation (ARWI) because the ongoing evaluation of flood control alterna-
tives for the basin by the Sacramento District is one of the first applications of the
approach, and almost certainly the most complex application yet attempted by
USACE. The risk and uncertainty methodology specifically addresses many
assumptions in the 1991 ARWI that were subject to controversy, and which the
committee was charged to review. Whether the controversy will be resolved
remains to be seen.
114
OCR for page 115
RISK METHODOLOGY
115
In particular, assumptions about levee freeboard for American River basin
levees are replaced by a distribution on the stage at which levees fail. Likewise,
hydrologic uncertainty that was described by an expected probability adjustment,
and assumptions about delays between the beginning of the flood and increased
releases, are now described by explicit probability distributions. Issues that were
in contention have not disappeared; what some viewed as conservative values
have been replaced by probability distributions, which may also be contested.
For decades, civil engineers have realized that it is not practical to protect
communities in the floodplain from all conceivable floods (Foster, 1924; Riggs,
1966~. Such protection measures would be prohibitively expensive, even if they
were practicable. Communities and individuals who choose to locate in flood-
prone areas will generally be exposed to some risk of flooding. However, it is
often economically advantageous to provide protection from flood events that
have a 1 in 50, 1 in 100, or a 1 in 500 chance of occurring in any year, depending
on the value of the property at risk, the chance of loss of life, and the costs of
flood risk reduction opportunities. Derivation of probability distributions to
describe the possible magnitude of flood flows has been a practice in civil engi-
neering since the early part of the century. They provide a description of hydro-
logic risk. When a particular flood flow with a 1 in T chance of being exceeded
in any year serves as a design flood for a project, USACE has said that the project
provides a T-year "level of protection."
The new USACE risk and uncertainty methodology explicitly introduces
into the planning process consideration of hydrologic, hydraulic, and economic
uncertainty. Before, the USACE planning procedure selected a level of protec-
tion corresponding to perhaps the 1 in 250 chance event (often called the 250-
year flood), and then determined the corresponding design flood flow. Use of an
expected probability correction did incorporate hydrologic uncertainty into flood
risk estimates (Beard, 1960, 1978; Stedinger, 1983a). Alternative hydraulic flood
control structures including levees, flood storage capacity in dams, and channel
improvements, in addition to flood-proofing efforts, were selected to control a
flood of that magnitude.
In the evaluation of flood control projects, there are a number of uncertain-
ties that make it difficult to determine whether a specified flood can be passed
safely. For example, flood control dams might have surcharge capacity that was
not included in the flood routing calculations. Levees are a more common con-
cern. Levee failure depends on factors such as the structural integrity of the levee
embankment, possible scour and undercutting, variation in the state of levee
repair, and other factors in addition to high water levels. Hydraulic predictions of
the flood stage associated with different flow rates may also be in error. Levee
failure stage predictions and stage-discharge relationships are affected by survey-
ing inaccuracies in the measurements of channel geometry and riverbed eleva-
tions, errors in estimation of flow resistance, simplifications in hydraulic routing
OCR for page 116
6
FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN
calculations, waves and wave effects, and possible settling of levees that affect
crest elevation.
Risk-based analysis of hydrologic and hydraulic engineering problems has
been and is an active area of research (Davis et al., 1972; Tung and Mays, 1981;
Ha~mes and Stakhiv, 1986, 1989, 1990; Duckstein et al., 1987; USACE, 1992a,b;
Haimes et al., 1993; Taylor et al., 1993~. In most risk analysis applications, the
risks of concern arise from the distributions of annual flood peaks, rainfall depths,
and other hydrologic variables (Mays and Tung, 1992~. In a few cases, project
performance is described probabilistically (Duckstein and Bernier, 1986; Chow
et al., 1988, section 13.4; Mays and Tung, 1992~. Uncertainty in structure perfor-
mance was important in several studies addressing dam rehabilitation and dam
safety (McCann et al., 1985; Goicoechea et al., 1987; Von Thun, 1987; Stedinger
et al., 1989; Bowles, 1990; see also NRC, 19851.
There are relatively few applications where risk analyses have considered
the natural variability in hydrologic and hydraulic variables as well as the uncer-
tainty in the parameters of fitted flood-flow frequency curves and calculated
stage-discharge relationships, and in economic quantities; these analyses might
best be described as risk and uncertainty analyses to make the distinction clear.
The Bayesian~ framework that is appropriate for hydrologic uncertainty has been
employed in proposals to include hydrologic parameter uncertainty in planning
studies (Benjamin and Cornell, 1970; Duckstein et al., 1975; Vicens et al., 1975;
Wood, 1978; Stedinger, 1983a). The USACE use of expected probability adjust-
ments is one way to include parameter uncertainty in flood control project evalu-
ation.
RISK AND UNCERTAINTY: A PRIMER
Uncertainties, Safety Factors, and the Meaning of Level of Protection
USACE traditionally has included safety factors in its design of facilities and
the specification of operating policies to address important hydraulic uncertain-
ties in flood control planning calculations. Surcharge storage in reservoirs might
be one safety factor. For levees, engineers have required that the design flood
The statistical literature includes several methods for dealing with parameter estimation, statisti-
cal inference, and decisionmaking. Bayesian statistical methods treat unknown statistical parameters
(the population mean, population variance, or a probability or quartile) as random variables whose
probability distributions reflect the degree to which the value of a parameter can be resolved from
available sample information as well as prior beliefs and other sources of information. With the
traditional statistical procedures employing standard confidence interval estimators and classical
hypothesis tests, such parameters are treated as if they have fixed (but unknown) values, and prob-
ability distributions describing the sample-to-sample variability of sample statistics and parameter
estimators are the focus of the analyses. The topic is addressed in more detail in the text.
OCR for page 117
RISK METHODOLOGY
117
pass through the levee system with some specified freeboard. Such a safety
factor enables the engineer and the planning agency to be confident that in an
actual flood event approximating the design storm, there will be sufficient chan-
nel capacity to pass that flood flow without the levees failing from overtopping or
excessive stages. In planning studies, encroachment within levee freeboard might
be treated as sufficient to cause levee failure, even though in an actual flood
failure might not necessarily occur at that stage. From an economic perspective,
one can ask how much freeboard is justified economically to increase project
reliability (Davis, 1991 J.
The practice of including freeboard in design suggests that at the design
flood associated with a target probability, called the "level of protection," there
will often be some residual safety factor before actual flooding would occur. If
there is, then the true chance of levee failure resulting in major flooding is less
than the specified target probability. The question arises as to what was meant by
the traditional "level of protection." Should it have been viewed as (1) an esti-
mate of the chance of flooding due to levee overtopping or breaching, or was it
simply (2) the exceedance probability of the design flood that a reservoir and
levee system was designed to pass with some safety margin?
Generally, evacuation plans would begin before a levee breached or was
overtopped. Thus the "level of protection" could be viewed as the probability
that the design event would be exceeded and thus that emergency measures
would be required, even though widespread flooding might not occur.
What seems clear is that there is confusion on this issue. Although calcu-
lated levels of protection might appear to address (2) above, their use to estimate
expected damages suggests that they are often used as an answer to (1~. This has
led to the conventional wisdom that USACE projects provide more protection
than acknowledged because safety factors built into levee design and reservoir
operating policies appear to add an additional increment of safety. If this conven-
tional wisdom is true, then by lowering the apparent benefit-cost ratios this prac-
tice may have worked against some proposals to provide needed flood protection.
For example, if levees can almost always pass flood flows that encroach within
the specified design freeboard, they actually provide protection from larger floods
than has been assumed in many analyses.
However, the inclusion of safety factors in reservoir-levee system design to
compensate for hydraulic uncertainty may not be sufficient to actually decrease
the risk of levee system failure or levee overtopping. If levee settlement in one
location ensures that a levee system failure will occur before the design flood
event is reached, excess channel capacity or extra freeboard at other locations
will not improve system reliability. In a levee system, failure occurs at the
weakest point. However, if in a flood event a reservoir operator can vary releases
in response to actual developments within the channel-levee system, it is possible
that variation in reservoir operations taking advantage of excess surcharge stor
OCR for page 118
8
FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN
age could avoid levee system failures due to other deficiencies within some range
of hydrologic loading.
Planners and engineers also realize that the condition of levees and some
equipment degrades with time. Safety factors are a reasonable way for designers
of flood control works to ensure that over time a system can continue to pass the
design flood without levee overtopping or breaching. However, it may not be
immediately clear how safety factors included in different components of a reser-
voir-channel-levee system interact to affect overall system reliability.
Definitions for Risk and Uncertainty
USACE will be wrestling for some time with the implementation of its new
risk and uncertainty methodology. Of concern will be both a consistent scientific
methodology, and a vocabulary and style for the presentation of the results to
technical audiences and the public. The choice of words is very important be-
cause they help us distinguish one concept or idea from another. In this regard,
the terms "risk" and "uncertainty" can cause problems because different authors
have ascribed to them significantly different ideas. Risk has been used to convey
each of the following meanings (USAGE, 1992a, pp. 10-1 1~:
1. The idea of hazard, when something is described as being "at risk."
2. The expected losses or risk related to a venture.
3. The probability of some outcome, such as the risk that a levee will be
overtopped.
All three definitions attribute to risk a probabilistic character related to the
possibility of an adverse and unwanted event in a particular system. Risk may be
due solely to physical phenomenon or to the interaction between man-made
systems and natural events.
The tea uncertainty has been given a broad and sometimes conflicting
range of meanings. There is a literature wherein the term uncertainty is used to
describe events for which objective probabilities are not available (USAGE,
1992a). On the other hand, it could simply to be used to describe situations that
are not certain; USACE (1992a) stated that "uncertainty means simply the lack
of certainty. It is the reality of inadequate information. When information is
imprecise or absent, that is uncertainty."
The USAGE's guidelines provide the following operational definitions of
risk and uncertainty (USAGE, 1992a, p. 123:
Risk: The potential for realization of unwanted, adverse consequences; estima
tion of risk is usually based on the expected result of the conditional probability
of the occurrence of event multiplied by the consequence of the event, given
that it has occurred.
OCR for page 119
RISK METHODOLOGY
Uncertainty: Uncertain situations are those in which the probability of potential
outcomes and their results cannot be described by objectively known probabili-
ty distributions, or the outcomes themselves, or the results of those outcomes
are indeterminate.
Those guidelines indicate that actual uncertain planning situations are lo-
cated on a continuum between situations of known risk (where the probability
distributions of interest are well specified) and situations characterized by uncer-
tainty (where those distributions are hardly specified at all; USACE, 1992a'.
119
A Distinction Between Risk and Uncertainty
Although the cited distinctions between risk and uncertainty are some times
useful, they are not the distinctions that are needed for our discussion of the
USACE methodology for risk and uncertainty analysis. Of particular concern
here with regard to the USACE risk and uncertainty methodology are:
· models of natural and operational variability and randomness, including
probability distributions describing flood flows, event-to-event variability in
stage-discharge relationships and reservoir operations, and variability in flood
damages due to factors not captured by flood stage, and
.
uncertainty representing limited understanding of system processes and
the lack of accuracy with which the parameters in models describing natural
processes can be specified, including the parameters of a probability distribution,
the cross-sections used to derive a stage-discharge curve, and the value and the
count of the number of dwellings in a protected portion of the floodplain.
In some cases one may be uncertain as to which of several competing models
to employ, such as alternative probability distributions. Uncertainty refers to our
lack of understanding of characteristics of nature that we conceptualize as being
fixed at any given time. Ideally, the values of various model parameters could be
determined. However, due to data limitations there are generally residual errors
in our understanding of those characteristics of nature that cannot be eliminated
with reasonable levels of effort.
The first situation is referred to here as natural variability or randomness in
the indicated process. The second situation is referred to as "specification error,"
or simply uncertainty. This use of uncertainty to describe lack of knowledge is
not strictly consistent with the operational definition for the term suggested in
USACE (1992a), although it may be consistent with the way the term is used.
This use is consistent with the definitions adopted by other groups (IS O TAG 4,
1993; Taylor and Kuyatt, 1993; NRC, 1994~.
OCR for page 120
120
FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN
Sources of Uncertainty
Recently, in a report on risk assessment of hazardous air pollutants, the
National Research Council (NRC, 1994) recommended making a clear distinc-
tion between parameter uncertainty, which is associated with the parameters of a
particular model, and uncertainty as to the appropriate model, or model uncer-
tainty. The report noted that parameter uncertainty often is described by continu-
ous parameter ranges (NRC, 1994) that result in corresponding uncertainty inter-
vals associated with predictions; however the choice among competing health
risk models generally corresponds to distinct and mutually exclusive choices.
The authors observed that "indiscriminately" combining the two types of uncer-
tainty in health risk assessment could result in the calculation of average health
risks and uncertainty ranges that are inconsistent with any of the alternative
models. The report recommended that parameter uncertainty be evaluated sepa-
rately for each competing model. Hydrologists face similar issues when choos-
ing between alternative flood flow probability distributions or between methods
for calculation of stage-discharge relations.
Hydrologists often consider what can be classified as a third type of uncer-
tainty, which arises due to model imprecision, or model prediction error. Thus,
even with the best parameters, operational hydrologic models may fail to pre-
cisely predict flood stages at some locations in a system; such model prediction
errors are another source of uncertainty in the analysis of flood projects. The
error here is not due to natural variability, which might be best described explic-
itly, or to a failure to have the best set of model parameters, which is described by
model parameter uncertainty, but is instead due to lack of model accuracy and
thus is a source of uncertainty associated with model predictions. Such predic-
tion errors can be thought of as a type of model uncertainty, because if one had a
more accurate model, such errors might be eliminated. However, better models
in most cases would have greater data requirements, requiring a finer spatial
description of channel cross-sections and roughness coefficients with fewer
lumped representations of watershed and channel characteristics. In fact, most
operational models deliberately employ simplifications and lumped representa-
tions of natural processes to restrict the parameter space to a manageable dimen-
sion so that available data are sufficient for model calibration. Thus uncertainty
due to model prediction error often reflects both data/parameter limitations and
model uncertainty. In this discussion, model prediction error is included with
other parameter and model uncertainties.
A FRAMEWORK FOR RISK AND UNCERTAINTY ANALYSES
A framework is needed to understand the structure of risk and uncertainty
analysis efforts for flood protection project evaluation and to understand the
relative roles of the natural variability of flood volumes, reservoir operations,
OCR for page 121
RISK METHODOLOGY
/ Q
_ / In
'~ ~ ~
3 /
i a
~\ Reservoir Outflow
121
-
S(O) :> Failure
/PL(S)
D(S)
us
no
a
River Stage
Possible values Possible reservoir Outflow peak 0 Probability levee Stage alone used to
of annual flood operation determines determines breaches is estimate damages D.
flows Q . outflow peak O for downstream stage S determined by Dotted line is without
any inflow peak Q. at points of interest. stage S. levee overtopping
or breach.
FIGURE 4.1 Deterministic and stochastic processes contributing to flood risk. Perfor-
mance of a flood control system involving both reservoirs and downstream levees can
depend on deterministic and stochastic components. Possible values of the inflow peak
for any year are described by a tree with branches, as are reservoir operations during that
event, because both are described as random processes. The transformation of the out-
flow peals O to downstream stage 5 is described by a deterministic relationship, though
there is uncertainty associated with parameters of that relationship. Likewise, damages
are described as a deterministic function of river stage for the levee breach/overtopping
case, and the case without levee failure or over topping. Levee failure is probabilistic and
occurs with a probability pit which depends on the stage S.
hydraulic system performance, stage-discharge errors, and uncertainty in hydro-
logic, hydraulic, and economic parameters. Figure 4.1 provides a conceptual
model for describing hydrologic risk, variation in reservoir operations, use of a
river stage-reservoir outflow relationship, levee reliability, and finally estimates
of the economic damages that would result should a levee fail. Several of these
relationships are stochastic, while others are described by deterministic relation-
ships.
The committee developed the event tree in Figure 4.1 to describe how the
volume distribution of the largest flood volume in a year is transformed first into
a river stage distribution and eventually into a damage distribution. This event
tree can be used to evaluate the probability that flood protection works are over-
whelmed and flooding occurs at some damage site, called the annual failure
probability (AFP). Likewise, it can serve as the basis for calculating the expected
annual damages (EAD), which would be the foundation of the economic evalua-
tion of project performance.
In Figure 4.1, a process is modeled either as being deterministic or as having
some random component reflecting natural process variability. To understand
the impact of specification errors or uncertainty in parameters of the selected
discharge-frequency model or in economic parameters, it is useful to note that for
each step in Figure 4.1 there is a set of parameters that define the relationship or
model employed at that step. For example, in the first step the flood flow
frequency relation requires specification of the parameters of that distribution;
OCR for page 122
22
FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN
these are often taken to be the mean, variance, and skewness coefficient of the
logarithms of the flows. Likewise, the variability in reservoir operation at the
second step will be described by some selected probability distribution, which
will also have parameters. In this presentation, uncertainty analysis focuses on
the parameters of the selected models; those models are assumed to accurately
reflect the probability distributions of processes that are variable (such as the
largest inflow in a year and the actual timing of reservoir releases in a future flood
event) and of deterministic processes such as the stage associated with different
channel flow, if only the correct or best values of the models' parameters could
be determined.
When the problem is structured as it is in Figure 4.1, one can identify the
parameters of each of the models that determine the numbers that enter into
calculation of risk and expected damages. One might then ask, how well or how
precisely are those model parameters defined? Or, how uncertain are values of
the project performance criteria AFP and EAD owing to the uncertainty or speci-
fication errors in various parameters?
There are several sources of variability in the economic damages that will be
experienced in any year. Extreme variability results from the magnitude of the
floods that may occur. Less important but still significant variability is intro-
duced by flood hydrograph timing and shape, variations in reservoir operations,
possible levee failure stage, and differences in the actual damages that would
occur to a structure depending on the duration of flooding, wave attack, and
differences in warning times; the effects of these factors are not captured by the
specification of stage alone. Planners understand that this variability exists and
so base their plans on AFP and EAD, which reflect the decision to average over
the probability distributions describing annual maximum flood volumes and other
variable processes.
In structuring the problem, as has been done in Figure 4.1, engineers can also
clarify how the various processes are thought to work. For example, the stage-
discharge relation can be conceived of as being time-invariant or deterministic,
so that a specific stage always corresponds to the same discharge. Then the
relevant uncertainty would pertain to the precise functional relation between
discharge and stage. Alternatively, there are certain stream reaches where the
stage-discharge relation varies significantly over time because of channel changes,
sediment movement, or the stages of tributaries or other streams with which the
river of interest merges. Such stage-discharge relations hence might best be
described by some random process. While in this second case the stage-dis-
charge relation might best be described as a source of variation, there would still
be uncertainty as to the best values of the parameters that describe that process.
Economic damages depend on several factors, and some are deterministic
while others are random. In particular, actual flood damages vary depending on
flood duration, the presence of ice and sediment, wave action, and warning time.
Flood damage uncertainties related to the number, types, and value of structures
OCR for page 123
RISK METHODOLOGY
123
in flood-prone areas would not change much from year to year, unless a major
flood occurs. The source and nature of variability and uncertainty in levee
performance present similar issues. USACE needs a clear framework for its risk
and uncertainty calculations to be able to articulate and explain its treatment of
such issues. Even so, it will not always be clear what should be described as
variability and what to represent as uncertainty.
Including Uncertainty in the Analysis
Planners should know by how much the estimates of AFP and EAD might be
in error. For example, a flood-frequency curve is based on a limited flood record.
By how much might the parameters of the discharge frequency function be in
error, and how big a change in AFP and EAD would result? Likewise, in deter-
mining the stage-discharge relationship, a limited amount of effort goes into the
surveying and the description of river cross-sections, geometry, and roughness
coefficients: the hydraulic model has a limited amount of detail. What errors
might this introduce into the evaluation of AFP and EAD? Similarly, limited
effort is devoted to determining the value of property at risk in flood-prone areas.
Additional effort could refine the data base describing the property at risk. Given
a statistical description of the likely specification errors in economic and struc-
tural survey data, a planner could quantify the magnitude of the corresponding
errors in AFP and EAD.
These questions can be addressed by sensitivity analysis procedures. The
document Guidelines for Risk and Uncertainly Analysis in Water Resources Plan-
ning (USAGE, 1992a), developed by the USACE Institute for Water Resources,
defined sensitivity analysis as
the technique of varying assumptions to examine the effects of alternative as-
sumptions on plan formulation, evaluation and selection. This can include
variation of model parameters as variation of benefit, cost and safety parame-
ters. One of the important uses of sensitivity analysis is to investigate how
different values of certain critical assumptions and parameters could result in
changing the choice of the selected project and report recommendations. Sensi-
tivity analysis is the systematic evaluation of the impacts on project formulation
and justification resulting from changes in key assumptions underlying the anal-
ys~s.
Sensitivity analysis can be used to bracket forecasts, parameters, benefit and
cost estimates, and other factors for which a range of values can be expected to
occur.
Generally, each model or process parameter is varied, one at a time, and the
result observed (USAGE, 1992a'. However, there are often so many parameters
in the models employed to evaluate flood protection projects that it would be
difficult to integrate such one-at-a-time evaluations, or to decide how they should
be incorporated into decisions (Moser, 1994~.
OCR for page 124
24
FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN
Describing Uncertainty
Useful descriptions of uncertainty can be developed by describing the speci-
fication errors or uncertainty in various economic parameters by probability dis-
tributions. This must be done with care so that the resultant distributions truly
reflect the probabilities planners should ascribe to the various parameters given
the sample information at their disposal, general information they have about the
processes of interest, and what is reasonable for the location in question. Then,
using those probability distributions over the uncertain parameters, a statistical
description of the uncertainty in AFP, EAD, and other performance criteria can
be computed.
For the purpose of developing a more mathematically precise notation for
describing uncertainty, denote a possible set of model parameters for the event
tree in Figure 4.1 by co. If the event tree in Figure 4.1 is evaluated with param-
eters ce, let the resulting values of AFP and EAD be denoted AFP(co) and EAD(co).
One can then ask what statistics should be calculated for the purposes of planning
and project evaluation. A reasonable and simple procedure would be for planners
and engineers to select their best estimate of ce, denoted here as Obese, and employ
the value of AFP and EAD calculated with that best estimate:
AFP(cubes~) and EAD(Cobes~)
This is what is done in many studies. It is generally satisfactory when model
parameter uncertainty is small.
Alternatively, if a probabilistic description has been developed to describe
the likelihood of different values of is, a different method could be employed.
Just as EAD(co) is obtained by averaging over the probability distribution for
annual floods, one could average the values EAD(co) over the probability distri-
bution for co. The resulting descriptions of average flood risk and average eco-
nomic losses are the average annual failure probability (denoted Avg~AFP]), and
the average expected annual damages (denoted AvgtEAD]), where
Avg{AFP] = Lover `,, {AFP(~) }
Avg[EAD] = Eover i,, ~ EAD(co) ~
and where E denotes expectation over the indicated variable. The choice between
AFP(cebes~) and Avg~AFP] and between EAD(cobes~) and AvgtEAD] reflects a
philosophical choice in planning. The choice should also reflect how well plan-
ners believe the available distribution for ~ has been specified. Even if Obese is
simply the average value of ce, because of the nonlinear relationship between a
probability distribution's parameters and exceedance probabilities, there will gen-
erally be a difference between the two descriptions of AFP and EAD.
Whether one uses average values of AFP and EAD or uses values of AFP
OCR for page 153
RISK METHODOLOGY
100
_ -
ct
i_
so
_'
.5, 50
o
-
._
ED
Ct
._
-
o
153
1 1 1 1 -~
50 100 200 300 500
year year year year year
Return Period of Flood Peak
FIGURE 4.5 Illustration of the trade-off between the return period of a flood peak and
the reliability of the reservoir-levee system with possible flood flows associated with that
return period.
Application of Reliability Indices in the ARWI
While the committee does not disagree with the analysis in Table 4.4 or Plate
12 of the 1994 Alternatives Report, it cannot see clearly what the public or most
engineers would do with such information. There are several concerns:
1. It is not at all clear how one should conceptualize the 1 percent chance
event given that it is not converted into a single flow estimate. Instead it is used
to generate a set of flows reflecting the hydrologic uncertainty in the computed
discharge-frequency relationship. This makes it very hard to anchor the analysis
mentally or to know for certain to what it is applied. In the definition of reliabil-
ity for the American River study taken from Ford (personal communication,
September 19, 1994, quoted above), what is the particular "specified event" to
which the chance of failure in Figure 4.5 or in Plate 12 of the 1994 Alternative
Report refers? Use of critical historic flood events with known flood flow peaks
would help resolve this conceptual vagueness.
2. The Sacramento District needs to clarify its reasons for wanting to calcu-
late this reliability index shown in Table 4.4. If the overall probability of levee
OCR for page 154
154
FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN
failure, which describes the residual risk of flooding, is already known for levees
of different heights, what does this other reliability calculation add?
3. The analysis does not indicate how much of the reliability (or likelihood
of failure) is due to hydrologic uncertainty, how much to stage-discharge uncer-
tainty, and how much to variability in levee failure stage. It is not clear what this
reliability calculation reflects.
The term reliability gave the committee the sense that it was a measure of
how certain the Sacramento District is that the levee system would perform as
intended. It was suggested that by using the new "reliability" index the Sacra-
mento District is trying to tell the public that there is some uncertainty about how
particular aspects of the project will perform. Ford (personal communication,
September 19, 1994) wrote, "The term reliability, as used in the American River
study, describes the likelihood that a proposed plan will perform as intended,
given the occurrence of a specified event." Because the committee could find no
explicit statement by the Sacramento District of what was intended, it had diffi-
culty interpreting such statements. How should one define the intent of an exist-
ing system?
Because the numbers in Table 4.4, Plate 12, and Figure 4.5 also include the
large uncertainty related to converting a "median" exceedance probability for a
flood into the correct discharge, the committee found it very difficult to develop
a useful interpretation of these numbers. It would be even more difficult for the
public to interpret them. Proponents of careful risk communication warn of the
pitfalls related to public misinterpretation of descriptions of risk (Plough and
Krimsky, 1987; Slovic, 1987; NRC, 1989, 1994~.
If each column in Table 4.4 corresponded to a discharge peak of a particular
magnitude, corresponding to a historic flood or a selected design hydrograph,
then one could interpret the calculated reliabilities as describing the consequences
of stage-discharge estimation errors and levee performance uncertainty. For
example, one could compute the reliability of the levee-reservoir system for a
flood flow with a peak of 300,000 cfs or 500,000 cfs into Folsom Reservoir due
to uncertainties in reservoir operation, stage-discharge relationships, and levee
performance. One could also provide the estimated probabilities that these par-
ticular peak flows are exceeded. For levee systems, a similar calculation would
result by having the columns in Table 4.4 represent particular discharges (and
perhaps the estimated probability each would be exceeded).
4. Calculations of project reliability may involve some difficulties that are
not apparent. In reservoir-levee projects the characteristics of a critical event
may depend on the capacity of the reservoirts) considered for different alterna-
tives. For a levee-only system, it is the peak inflow that matters most. As one
adds more storage, flood inflow volume becomes increasingly important. Thus
one wants to select for the columns of a table such as Table 4.4, and for the graph
OCR for page 155
RISK METHODOLOGY
155
in Figure 4.5, events that for all alternatives are equally critical. It may be
important that the public and the engineers who are reviewing project proposals
understand how this is done. But this issue is completely hidden in a table such
as Table 4.4, in which the character of the actual hydrologic event corresponding
to each probability is obscure.
5. The explanation that is reproduced at the beginning of this section from
EC 1105-2-205 also gives the sense that from Table 4.4 and Figure 4.5 one can
determine the risk of flooding should a particular project be adopted. The re-
sidual risk of flooding is certainly a primary concern. Ford (personal communi-
cation, September 19, 1994) wrote,
The importance of reliability is, to some extent, a function of the consequences
of exceedance. If the consequences are great, then high reliability is necessary.
For example, if overtopping a levee would inundate a high-density residential
development to a depth of 25 feet without warning, high reliability is required.
This discussion of the importance of reliability ignores the risk associated
with the target flow. The reliability of the system as calculated by USACE is
only part of the residual risk. It is the overall risk of flooding that is key, not the
reliability of the system for particular events. That overall residual risk of flood-
ing, described by AFP or Avg[AFP], and the expected annual damages (EAD)
are certainly the two most important system performance criteria.
A significant problem with the presentation of system reliability in Table 4.4
and Figure 4.5 is that reliability appears to address the residual risk of flooding,
while it actually hides the true answer in a matrix of less meaningful numbers. To
compute the actual risk of flooding (as described by AvgLAFP]), one would need
to compute the average across all failure probabilities of the reliability of the
system. There are insufficient numbers in the table to do this computation, and
interested individuals should not have to do it themselves. Engineers and plan-
ners should perform these calculations and provide the results.
Table 4.4 and Figure 4.5 appear to reflect a desire to hold on to the old idea
of "level of protection," expressed by the hydrologic return period T or
exceedance probability for a design flood, while moving to a new risk analysis
methodology that includes the idea of uncertainty and variability in other pro-
cesses. Davis (1991) noted that traditionally projects were defined by the target
"level of protection." The problem with the presentation of system reliability
versus a target failure probability is that it fails to integrate those two sources of
risk.
6. In the American River study, reliability is also used to demonstrate that
the reliability of the levee network across the American-Sacramento River sys-
tem is not impaired by a project. This is a legitimate concern and one that a risk
analysis methodology should be able to address. The Sacramento District has
demonstrated how its reliability index can be calculated at different points in the
OCR for page 156
56
FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN
system for different probability levels to demonstrate that "reliability" is not
Impaired.
The committee wondered if this is the most effective definition of reliability
for that purpose. If reliability is used to ensure that for every flood level de-
scribed by a cumulative probability p, throughout the river system the probability
of flooding is not increased by a project, it is not necessary to include hydrologic
discharge-frequency uncertainty. It is much simpler to specify a range of Folsom
inflow hydrographs and evaluate the reliability for each. The "reliability of the
system for a given inflow" is both simpler and more meaningful than the "reli-
ability of the system for a given exceedance probability including our inability to
determine the flow actually associated with that exceedance probability."
In this regard, requiring reliability to remain the same for every hydrograph
is a more demanding requirement than requiring that it not decrease for every
median exceedance probability after averaging over hydrologic uncertainty in the
frequency curve. The first approach requires that reliability not decrease at every
flood flow; the second requires that reliability not decrease for averages over
flow ranges.
Keying on clearly specified flood hydrographs with their associated peak
and volume seems to meet the requirement of ensuring that reliability not de-
crease more rigorously than the approach the Sacramento District has adopted. It
would also be clearer and easier to understand. Moreover, it is also easier to
compute and relate to levee and channel system performance because those un-
certainties will not be swamped by the potentially much larger hydrologic uncer-
tainty.
Overall, the committee applauds the USACE decision to adopt a risk-based
planning methodology that better incorporates uncertainties in key variables.
However, the committee does not believe that the definition for system reliability
that was proposed in USACE guidelines and adopted by the Sacramento District
is particularly effective at addressing the relevant issues. In many cases, it
seemed unnecessary or misleading. Annual failure probability (AFP, Avg~AFP],
or both) is likely to be the most straightforward and easily understood measure of
residual flood risk. It could be supplemented by the vulnerability criteria dis-
cussed in the risk communication section of Chapter 6.
THE 1994 ALTERNATIVES REPORT
The committee reviewed the 1994 Alternatives Report (USAGE, Sacramento
District, 1994a) and found the document to be particularly confusing. The report
provided a summary of its evaluation of different projects consisting of alterna-
tive modifications of the system. Unfortunately, essential details of the analysis
were omitted, so the committee could not determine what was actually done from
reading the report. In particular, the committee could not determine the extent to
which some criticisms of the 1991 analyses had been addressed.
OCR for page 157
RISK METHODOLOGY
157
The report also failed to associate with the estimated net benefits any mea-
sure of overall uncertainty due to economic uncertainty, or hydrologic and hy-
draulic uncertainties, as recommended by EC 1105-2-205. These uncertainties
could be important given the modest benefit-cost ratios calculated for the alterna-
tives considered.
A very serious concern is how the report addressed issues of risk terminol-
ogy and its reporting of flood risk. USACE now has two significantly different
ways to calculate flood risk. On pages 8 and 9 of the 1994 Alternatives Report,
they were both called "level of protection." No distinction was made between
estimates of flood risk calculated with the traditional level of protection method-
ology and those calculated with the new risk and uncertainty methodology.
Throughout the report a host of different terms and phrases were used inter-
changeably to describe these ideas. A layperson would have great difficulty
sorting out the following jumble of terms: T-year level of protection, exceedance
interval (p. 8), return period (p. 8), recurrence frequency (p. 9), control for Tyears
(pp. 18 to 23), T-year flood (p. 9), T-year flood protection (p. 6), T-year protec-
tion (pp. 27, 29), T-year return frequency (p. 34), expected exceedance (pp. 37,
39), expected level of protection (p. 57), annual recurrence (Plate 5), and flood
event return period (Plate 12J. The report should use a few terms whose defini-
tions are both clear and consistent with commonly accepted interpretations.
The most common terms in the report are T-year level of protection, T-year
protection, control for T years, and T-year flood protection. The use of the term
"level of protection" to describe flood risk is inconsistent with the new USACE
guidelines for risk and uncertainty analyses (EC 1105-2-205' and confuses the
traditional and the new approaches to calculating flood risk. This terminology
supports the erroneous idea that one and only one T-year flood occurs every T
years. Actual statements in the report reinforce the error. On page 9, flood risk
was described as a flood once in 78 years or 103 years, while the executive
summary indicated that "levees could fail about once in every 78 years" and "the
level of protection (or likelihood that levees would not fail) would be increased to
about once in 100 years." These are exactly the analogies that should be avoided.
With the new risk and uncertainty methodology, estimates of flood risk are
no longer tied to a single T-year design flood, but can depend on different combi-
nations of flood flows, operating decisions, and levee performance. Instead of
stating that a project has a 200-year "level of protection" or protection for the
200-year flood, the Sacramento District should instead indicate that the annual
risk of flooding is 0.5 percent per year, or the annual risk of flooding is 1 in 200
(see Stedinger et al., 1993, p. 18.3~. It is also informative to convert such annual
risks into the risk of flooding over 25 to 50 year periods, reflecting the likely
length of a mortgage or the anticipated economic life of structures and dwellings.
When producing the 1994 Alternatives Report, the Sacramento District was
under a great deal of pressure to revise its analysis of flood control alternatives to
provide protection for people and property along the lower American River. Its
OCR for page 158
158
FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN
difficulties were increased by the need to use the new risk and uncertainty meth-
odology being developed within the Sacramento District for the planning of flood
protection projects. Inadequacies in the 1994 Alternatives Report reflect those
pressures and constraints. The committee hopes that subsequent documents will
more clearly describe how the analyses were conducted and will more clearly
explain the basis for the risk and uncertainty analyses.
THE PROMISE OF ECOLOGICAL RISK ASSESSMENT
USACE has made a commendable effort to apply recently developed risk
and uncertainty analysis to the engineering problems faced in minimizing the
damage from floods. The question then arises: Should not the relevant ecologi-
cal risk and uncertainty that may be the consequence of each of the proposed
actions also be subjected to risk analysis? Applying ecological risk assessment to
the major areas of uncertainty would be a daunting task. The following discus-
sion highlights some of the advantages and disadvantages of such an approach.
Development of the Paradigm
Formal assessment of risk in ecological science and management is a rela-
tively new development. Until very recently, EPA had not developed any guide-
lines for risk assessment (Suter, 1993~. Thus far, the principal application of risk
assessment to ecological problems has been in the context of considering impacts
of hazardous chemicals in the environment, evaluating the risk of extinction of
rare or endangered species, or providing management advice for commercial
fisheries. Conceptually, there seems to be no reason that the process could not be
applied to assessment of potentially adverse effects of water projects such as the
ones considered here in the American River. However, the extension to such an
analysis is controversial (Lackey, 1994) and probably will not be generally ac-
cepted in the scientific community at this time.
Ecological risk assessment has evolved slowly over the past two decades, but
has received impetus from the National Research Council (NRC) paradigm for
human health risk assessment: Risk Assessment in the Federal Government: Man-
aging the Process (NRC, 1983~. EPA has recently released a Framework for
Ecological Risk Assessment (EPA, 1992), along with a series of case studies
(EPA, 1993, 1994~. These publications do not present final policy and proce-
dures but are designed to stimulate discussion and development of a process that
will be in flux for some time. EPA is developing formal guidelines for conduct-
ing ecological risk assessments, which are expected to be released in late 1995 or
early 1996.
NRC has been in the forefront of such development, with reports on risk
communication (NRC, 1989) and on issues in risk assessment, including a sig
OCR for page 159
RISK METHODOLOGY
159
nificant discussion of ecological risk (NRC, 1993~. Currently an NRC committee
is conducting workshops designed to build consensus on the philosophy and
methods for ecological risk analysis. In addition, many academic and industrial
scientists are developing and evaluating the process (Suter, 19934.
The debate about the extension of risk analysis to ecological problems has
focused on several contentious points:
· The process is based on a human health paradigm; extension to ecological
effects, particularly at the ecosystem level, is highly problematic. There is insuf-
ficient understanding of ecosystem processes to predict outcomes with any cer-
tainty.
· Risk assessment has the potential to produce a sort of an ecological triage,
whereby particular processes and species thought to be important might receive
attention at the expense of some potentially serious problems.
· Risk analysis may lead to a consideration of alternatives that is too nar-
row, particularly if the focus is on the risk of a particular action versus that of no
action. The analysis must consider the full range of alternatives, and benefits as
well as risks of all the alternatives.
· The process can be tilted in favor of a particular action, given that uncer-
tainty is great and the desired level of risk defined; the analysis may simply
proceed until the desired endpoint is reached.
In spite of these serious concerns, ecological risk analysis has had some
success, leading to models that may provide templates for further development.
A recent NRC report (NRC, 1993), in a section titled "A Paradigm for Ecological
Risk Assessment," recognized significant problems in extension of the health risk
approach from NRC (1983~. Nonetheless, that committee concluded that inte-
grating ecological risk into the original framework is possible and that such an
approach is preferable to developing a completely new framework. Key scien-
tific issues limiting the application of ecological risk assessment include the
following:
· Extrapolation across scales of space, time, and ecological organization.
Estimating ecosystem-level response on the basis of laboratory or small-plot
experiments is a particular concern.
· Quantification of uncertainty, including measurement uncertainty, natural
variability in ecological systems, and inadequacy of models.
.
Validation of predictive tools. Substantial improvements are needed in
the models fundamental to effective risk assessment.
· Valuation of outcomes. Analysis of both costs and benefits is essential,
but generally accepted principles for valuation of ecosystems are not available.
OCR for page 160
60
FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN
Ecological Risk Assessment and the American River
This committee believes that the Sacramento District has done a reasonably
effective job of framing alternatives in the American River planning activities,
particularly in the 1994 Alternatives Report. The recent organization of the
Lower American River Task Force under the sponsorship of SAFCA has substan-
tially improved communication among the various stakeholders in the basin.
Hence there is the potential for appropriate use of ecological risk analysis. None-
theless, there is little likelihood that such an analysis would be accepted by the
scientific and lay community at this stage in the development of flood control
proposals for the American River.
One of the most contentious environmental issues faced by the committee is
the assessment of the potential effects in the canyons of the North and Middle
Forks above a proposed detention dam at Auburn. Great uncertainty surrounds
estimates of the probability of mass soil failure and mortality of vegetation fol-
lowing inundation. A case study example is available of risk analysis applied to
a similar situation, modeling future losses of bottomland forest wetlands in Loui-
siana in the face of increased flooding (EPA, 19931. However, this analysis was
based on a substantial body of research in that region and on the application of a
simulation model adapted for the specific area. No such base of knowledge is
available for the American River canyon. Scientific understanding that would
allow accurate modeling of the processes involved in hillslope failure and mortal-
ity of vegetation is simply not available at this time, and most likely will not be
available for years. Significant opportunities were missed when research failed
to take advantage of the presence of the cofferdam upstream of the Auburn dam
site, though the detention dam concept was not developed until after the dam
breached in 1986.
One field of resource management has a relatively long history of recogniz-
ing uncertainty and may have lessons to provide as ecological risk assessment
develops. Managers of marine and anadromous fisheries have long faced uncer-
tainty. Stimulus for the development of more robust approaches to prediction in
the face of incomplete knowledge has often come from the collapse of large
fisheries (Ludwig et al., 1993~. The model of adaptive management advocated by
Holling (1978) and Walters (1986) recognizes that uncertainty is a pervasive
element of most resource management scenarios. The committee strongly rec-
ommends that the water resource issues in the American River be managed in this
adaptive context. Some important characteristics of this approach include the
following:
· recognizing and communicating uncertainty,
· treating management as an experiment, and
· providing sufficient monitoring to allow managers to learn from the expe-
rience gained from observing system behavior.
OCR for page 161
RISK METHODOLOGY
161
The current direction in models being developed for management advice
emphasizes Bayesian analysis (Walters, 1986; Hilborn and Walters, 1992) and
statistical decision theory (Frederick and Peterman, 1995~. Another trend in
more traditional statistical analysis of natural resource issues has been a focus on
statistical power analysis, particularly in the analysis of downward trends in
resource abundance (Peterman, 1990~. The approach has promoted more explicit
consideration of where the burden of proof properly lies. Incorporation of these
concepts into ecological risk analysis should improve future decisions in a wide
array of resource conflicts.
CONCLUSION
From its review of the material provided describing the new USACE risk and
uncertainty analysis guidelines, and the 1994 Alternatives Report, the committee
reached the following conclusions.
· Improvements in Planning Methodology. The USACE risk and uncer-
tainty methodology is an innovative and timely development. The explicit recog-
nition of modeling uncertainty should result in a better understanding of the
accuracy of flood risk and damage reduction estimates. The committee applauds
the USACE efforts to develop a better flood protection planning methodology
incorporating both risk and uncertainty in hydrologic, hydraulic, and economic
parameters and processes. However, USACE and the Sacramento District need
to more carefully develop and articulate the structure of their risk and uncertainty
methodology, employing an effective vocabulary for distinguishing among risk,
variability, uncertainty, and system reliability for use with technical and public
audiences. USACE leadership is encouraged to convene an intra-agency work-
shop, including outside experts, to review the risk and uncertainty procedures,
with special attention to the committee's concerns, and to recommend specific
changes to the guidelines as necessary.
· Impact of Uncertainty on Performance Criteria. The proposed USACE
risk and uncertainty methodology, which directly includes hydrologic uncertain-
ties (and potentially other sources of uncertainty) in the calculation of average
flood risk and the average annual flood damages that might be averted by a
project, inflates those estimates. This upward bias is a concern if the methodol-
ogy is adopted nationwide because it could distort the economic evaluation of
projects. The committee did not have the resources to determine the actual
distortion for the American River study.
· Descriptions of Project Performance. To avoid the problem of bias
described in the recommendation above, and to simplify the analysis so that it can
be more easily understood and is less dependent on hidden assumptions, the
committee recommends that the primary descriptions of the expected annual
flood damages and of the probability of flooding be based on best estimates of the
OCR for page 162
162
FLOOD RISK MANAGEMENT AND THE AMERICAN RIVER BASIN
parameters of models defining the deterministic and significant random pro-
cesses contributing to flood risk and flood damage.
· Descriptions of Project Performance Uncertainty. Best estimates of
expected annual flood damages and the risk of flooding should be supplemented
by descriptions of their uncertainty due to hydrologic, hydraulic, and economic
uncertainties. Uncertainty can be described by a standard error or the distribu-
tion of the likely values of the quantity of concern. The impact of uncertainty
can be illustrated by computing the probability that the national economic devel-
opment objective is negative, or various quartiles of its distribution. The ap-
proach should be consistent with the requirement in USACE guidelines for risk
and uncertainty analyses (EC 1105-2-205' that the estimate of NED benefits be
reported both as a single expected value and on a probabilistic basis (value of the
benefit and its associated probability) for each planning alternative. It is the
committee's understanding that the American River study will not address eco
. . .
noetic uncertainties.
· Measures of System Performance Reliability. Estimates of expected
annual flood damages and economic benefits associated with different projects,
and the probability of flooding at different locations, are likely to be the primary
criteria describing flood risk and economic impacts. It will often be useful to
calculate other indices of system performance and the reliability of different
components of the river channel and levee system. The committee questions in
general the value of the system reliability index proposed by USACE documents
and employed by the Sacramento District in the American River study. It seems
to be an awkward combination of traditional and new concepts.
In the case of the American River study, a reliability index did have an
important role in demonstrating that different projects do not increase the risk of
flooding in any reach of the American-Sacramento River system. Still, it is not
clear that the adopted definition is the most effective or easily understood. How-
ever, the Sacramento District's use of reliability does not affect the validity or
accuracy of the study results and the calculations upon which they are based.
· Risk Analysis in USACE Alternatives Report. The committee re-
viewed the risk and uncertainty analysis in the 1994 Alternatives Report. The
report failed to associate with the estimated net benefits any measure of overall
uncertainty due to economic, hydrologic, and hydraulic uncertainties. The com-
mittee found the explanation and presentation of the results particularly confus-
ing. No distinction was made between estimates of flood risk calculated with the
traditional level-of-protection methodology and those calculated with the new
risk and uncertainty methodology. Both were called "level of protection" and
described by a variety of terms, which further contributed to the confusion.
The most common terms in the report are control for T years, T-year level of
protection, and T-year flood protection. The use of the term level of protection to
describe flood risk is inconsistent with the new USACE guidelines for risk and
OCR for page 163
RISK METHODOLOGY
163
uncertainty analyses (EC 1105-2-205) and confuses the traditional and the new
approaches to calculating flood risk.
This terminology and phases appearing in the report fosters the erroneous
idea that one and only one T-year flood occurs every T years. Moreover, with the
new USACE risk and uncertainty methodology that was employed, failure is no
longer related to a single T-year design flood being exceeded, but can depend on
different combinations of flood flows, operating decisions, and levee perfor-
mance. Instead of stating that a project has a 200-year level of protection, or
protection for the 200-year flood, studies should indicate that the risk of flood-
ing is 0.5 percent per year, or equivalently that the chance of flooding is 1 in 200
each year.
· The Promise of Ecological Risk Assessment. At this time the committee
does not believe that the process of ecological risk analysis is sufficiently
evolved, nor that there is sufficient knowledge of the ecological system, for this
new tool to be applied usefully to problems of flood control in the American
River basin. However, ecological risk assessment does provide a new approach
that emphasizes the importance of uncertainty in the analysis of the conse-
quences of various alternatives. The process will help select questions for inves-
tigation and will be increasingly important in broadening the scope of future
planning. USACE should follow this rapidly evolving approach and adopt it as
soon as it shows promise of improving the decisionmaking process.
Representative terms from entire chapter:
flood risk
