APPENDIX E
Uncertainty, Sensitivity, and Bayesian Methods

SOURCES OF UNCERTAINTY

Risk analysis involves modeling, explicitly or implicitly. As such, it is subject to error and uncertainty. Model error is the difference between reality and its representation in the form of a model. Symbolically, Δ = rm, where Δ is the error, r is the reality (actual behavior), and m is its representation (or prediction) by the model. Models are usually created for a purpose, and the level of accuracy or the correspondence between the model and reality should be reviewed in the context of that purpose. A model could be accurate with respect to one aspect of reality and in relation to a certain purpose but inadequate in a different context. Any risk model has a “form” (or structure) and a set of “parameters.” Therefore, the assessment of the magnitude of Δ is subject to two sources of uncertainty—one stemming from the structure and form of the model, generally referred to as model uncertainty, and another due to uncertainty in the model parameters, generally known as parameter uncertainty.

Uncertainties exist for many reasons, including randomness, incomplete knowledge with regard to phenomena, inaccuracies in determination of the values of quantities and parameters (e.g., a probability value), high sensitivities of system performance to specific conditions, and omission of important factors (e.g., a basic event) from an analysis. Such sources of uncertainty should be



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 194
APPENDIX E Uncertainty, Sensitivity, and Bayesian Methods SOURCES OF UNCERTAINTY Risk analysis involves modeling, explicitly or implicitly. As such, it is subject to error and uncertainty. Model error is the difference between reality and its representation in the form of a model. Sym- bolically, Δ = r − m, where Δ is the error, r is the reality (actual behavior), and m is its representation (or prediction) by the model. Models are usually created for a purpose, and the level of accu- racy or the correspondence between the model and reality should be reviewed in the context of that purpose. A model could be accurate with respect to one aspect of reality and in relation to a cer- tain purpose but inadequate in a different context. Any risk model has a “form” (or structure) and a set of “parameters.” Therefore, the assessment of the magnitude of Δ is subject to two sources of uncertainty—one stemming from the structure and form of the model, generally referred to as model uncertainty, and another due to uncertainty in the model parameters, generally known as parameter uncertainty. Uncertainties exist for many reasons, including randomness, incomplete knowledge with regard to phenomena, inaccuracies in determination of the values of quantities and parameters (e.g., a probability value), high sensitivities of system performance to specific conditions, and omission of important factors (e.g., a basic event) from an analysis. Such sources of uncertainty should be 194

OCR for page 194
Uncertainty, Sensitivity, and Bayesian Methods • 195 documented in the risk analysis, whether qualitative or quanti- tative. Given these multiple sources, discussions of uncertainty can easily become complex. All of these sources of uncertainty can be portrayed in an analysis by means of a probability density function (pdf). This is a normal- ized function that portrays the relative likelihood that an uncertain variable will be observed within a particular interval. It is helpful in a practical way to characterize several classes of uncertainty, because careful thinking about the character of sources of uncertainty leads to a better understanding of the problem and better representation of the associated pdfs. Three classes of uncertainty should be considered: • Deterministic case—there is no variability or there is no imperfect state of knowledge that leads to variability in the results; • Aleatory uncertainty—there is random variability in any of the factors that leads to variability in the results; and • Epistemic uncertainty—the state of knowledge about the effects of specific factors is less than perfect. To help understand these terms, a more operational point of view is that uncertainty is aleatory if • It is (or is modeled as) irreducible; • The uncertainty is observable (i.e., repeated trials yield different results); or • Repeated trials of an idealized thought experiment will lead to a distribution of outcomes for the variable, and thus this distribution is a measure of the aleatory uncertainties in the variable. The uncertainty is epistemic if • One is dealing with uncertainties in a deterministic variable whose true value is unknown; • Repeated trials of a thought experiment involving the variable will result in a single outcome, the true value of the variable; or • It is reducible (at least in principle). The approach for treating uncertainty implements the subjec- tive framework for treating probabilities in analysis described by Apostolakis (1990) (see the discussion of Bayesian methods below). This approach provides the benefit of a clearer (and potentially

OCR for page 194
196 • Risk of Vessel Accidents and Spills in the Aleutian Islands simplified) elicitation-based quantification process. This benefit arises from the subjective framework’s distinction between aleatory and epistemic uncertainties, which requires a careful examination of the factors contributing to uncertainty, resulting in a clearer definition of the issues being addressed during the elicitation. It provides a clear way to tell the truth about what is and is not known. For this reason and with clear documentation, the analysis or elicitation can be well defended. The implications of the different values the uncertain variable might take and their corresponding probabilities of being true can be propagated to the ultimate answer obtained concerning the risk level (see the discussion of uncertainty propagation below). The main way in which incompleteness-related uncertainty can be treated by using a pdf relates to the magnitude by which an analysis could be in error because of omission of an important structural element. The possibility of an omission is treated in terms of the magnitudes of the consequences within an analysis to which such an omission is considered likely to contribute. SENSITIVITY ANALYSES AND UNCERTAINTY PROPAGATION Once the magnitudes of uncertainties of the factors of a risk analy- sis have been determined, they can be used either to examine the sensitivities of the results to variations in the factors within the stated domain or to propagate the effects of the uncertainties to ultimate risk results. Both types of analysis are important in per- forming risk assessment, since uncertainties typically cannot be eliminated (although their magnitudes can be reduced) and should be addressed. In sensitivity analyses, the influences of different values of a fac- tor on ultimate answers are estimated. This is done to identify the factors of greatest sensitivity and to obtain a rank ordering of such sensitive factors. This knowledge can be valuable in iden- tifying the factors for which improvement in performance would be most valuable and those for which degradation of performance is most important to avoid. This knowledge is also valuable to

OCR for page 194
Uncertainty, Sensitivity, and Bayesian Methods • 197 the analyst in helping to resolve or understand the epistemic uncer- tainty involved. In fact, there is a trade-off between the cost of an analysis or experiment and the potential impacts of living with resolvable epistemic uncertainty. In propagation of uncertainties, the pdfs discussed above are used in selecting the individual values of the uncertain factors on the basis of the Monte Carlo technique in order to propagate to the ultimate answers the influences of the important uncer- tain factors, considered in combination. In one such sample, a value of each factor is selected randomly, reflecting the influ- ences of its pdf. Then, the set of such values obtained from all factors is used to obtain the values of the individual performance measures of the system. When this procedure is repeated N times, it will produce a set of N values of each performance measure. Then, the relative abundance of values of a particular measure found within a specified interval of measure values provides an approximate estimate of the magnitude of the pdf of that mea- sure corresponding to the interval specified. As the magnitude of N increases, the error of this estimate will decrease asymptotically toward zero. Thus, one can obtain an approximate estimate of the range of the measure consistent with a stated confidence value. Since this estimate reflects both subjective and objective factors, it should be viewed as a statement of the estimator’s belief about the figure of merit (e.g., the level of risk). BAYESIAN STATISTICAL ANALYSIS For risk assessment, the Bayesian approach offers important advan- tages: all kinds of evidence are used, uncertainty bands are narrower, and evaluating data with zero occurrences of events in N trials is straightforward. The foundation of Bayesian statistical inference is Bayes’ theorem. The basic idea is simple. From the calculus of probability, we know that the probability of the joint occurrence of two events A and E is the following: P ( A ∩ E ) = P ( A) i P ( E A) = P ( E ) i P ( A E )

OCR for page 194
198 • Risk of Vessel Accidents and Spills in the Aleutian Islands which is a simple, uncontroversial statement. The Rev. Thomas Bayes (1958) went a step further by rearranging the above as P ( A) i P ( E A) P ( A E) = P (E) and interpreting these terms as follows: P(A⏐E) = “posterior” probability of A, after collecting evidence E (say, the result of an experiment); P(A) = “prior” probability of A; and P(E⏐A) = “likelihood” of the evidence (if the evidence can take on several values E1, E2, . . . , the likelihood is the probability of getting evidence Ei, given the prior). This Bayesian switch (called inverse probability) allows the use of something that is known or can be calculated (the likelihood of the evidence) to determine the value of P(A) (given all the evidence at hand). It leads to a meaningful definition of subjective or state-of- knowledge probability. Details of the approach can be found in many standard sources (De Finetti 1975; Jefferies 1961). E can be historical data, results of experiments, or expert opinion, all of which may be subject to uncertainty and ambiguity. General- ized forms of Bayesian inference developed in recent years provide ways of using such information in developing epistemic uncertainty distributions for risk model parameters. REFERENCES Apostolakis, G. 1990. The Concept of Probability in Safety Assessments of Technological Systems. Science, Vol. 250, pp. 1359–1364. Bayes, T. 1958. An Essay Towards Solving a Problem in the Doctrine of Chances. Philosophical Transactions of the Royal Society, Vol. 53, pp. 370–418; Vol. 54, pp. 296–325. Reprinted in Biometrika, 1958, Vol. 45, pp. 293–315. De Finetti, B. 1975. Theory of Probability: A Critical Introductory Treatment, Volumes 1 and 2. John Wiley and Sons, New York. Jefferies, H. 1961. Theory of Probability, 3rd ed. Oxford University Press, New York.