Appendix F
Assessment of the Probabilistic Model for Estimating Metal Loading and Effectiveness of Remedial Action

Edmund A. C. Crouch, Ph.D.

Senior Scientist

Cambridge Environmental, Inc.

Cambridge, Massachusetts

The Probabilistic Analysis of Post-Remediation Metal Loading Technical Memorandum (Revision 1) (URS Greiner, Inc. and CH2M Hill 2001a) (PTM) describes its purpose very well:

The probabilistic analysis is a risk management tool that can help quantify the certainty, conditional on available information and its interpretation, that a proposed remedy would meet cleanup goals. (PTM p. 1-1)


The purpose of the probabilistic analysis is to help support informed risk management decision-making. It does so by helping to quantify the certainty that a remedial alternative or a proposed remedy could actually meet cleanup goals…. (PTM, p. 1-3)

It formulates an approach intended to meet these objectives. The formulation can be readily summarized,1 and this summary is presented first without any comment on its correctness or applicability. It is assumed that dissolved metal loading to the Coeur d’Alene River (e.g., in pounds/day, the

1  

Understanding this appendix will require access to and some familiarity with the PTM.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 459
Appendix F Assessment of the Probabilistic Model for Estimating Metal Loading and Effectiveness of Remedial Action Edmund A. C. Crouch, Ph.D. Senior Scientist Cambridge Environmental, Inc. Cambridge, Massachusetts The Probabilistic Analysis of Post-Remediation Metal Loading Techni- cal Memorandum (Revision 1) (URS Greiner, Inc. and CH2M Hill 2001a) (PTM) describes its purpose very well: The probabilistic analysis is a risk management tool that can help quan- tify the certainty, conditional on available information and its interpreta- tion, that a proposed remedy would meet cleanup goals. (PTM p. 1-1) The purpose of the probabilistic analysis is to help support informed risk management decision-making. It does so by helping to quantify the cer- tainty that a remedial alternative or a proposed remedy could actually meet cleanup goals. . . . (PTM, p. 1-3) It formulates an approach intended to meet these objectives. The for- mulation can be readily summarized,1 and this summary is presented first without any comment on its correctness or applicability. It is assumed that dissolved metal loading to the Coeur d’Alene River (e.g., in pounds/day, the 1Understanding this appendix will require access to and some familiarity with the PTM. 459

OCR for page 459
460 APPENDIX F unit used throughout the PTM) at some specific location on the river, can be calculated as follows: K  Nj    L = ∑ RLPj Zref  ∑ Vi j , (1.1) j =1   i =1   where2 L = (preremedial) metal loading in the Coeur d’Alene River at the specific location examined (pounds/day); Zref = “loading potential” per unit volume (pounds/day/cubic yard) for the reference source type for the location on the river under examination (averaged over all sources of that type affecting that location);3 RLPj = “relative loading potential” for the location on the river under examination for a contamination source of type j, averaged over the sources of that type that affect the river location under examination (for the refer- ence source type, the RLP is unity); Vij = volume (cubic yards) of a source of type j with index i that affects the river at the location examined, all such sources being indexed; K = number of different types of sources that affect the river at the location examined; and Nj = number of sources of type j that affect the river at the location examined. The contamination sources are generally volumes of contaminated soil, sediment, and rock, categorized by type. The source types used in the PTM (pp. 2-18 to 2-19), for the upper basin, conceptual site model (CSM) Units 1 and 2, are adits (these are treated specially, by using measured flows and concentrations and deriving an effective volume for them), tailings-impacted floodplain sediments, unimpounded tailings piles, impounded tailings piles at inactive facilities, impounded tailings piles at active facilities, waste rock piles in floodplains, waste rock piles in upland areas, and deeper impacted floodplain sediments (unremediated sources). The reference source type (with RLP = 1) is taken to be tailings-affected floodplain sediments. 2The notation of the PTM is adopted, except that all symbols are italicized to agree to the degree possible with standard notational conventions (which are not observed in the PTM). The only possible confusion is between the symbols L and L which have distinct meanings in the PTM; however I avoid this confusion by using a different symbol, W, for what the PTM calls L. 3Only ratios of quantities each of which multiplies what I here call Z ref are required in the PTM, so no such term is defined anywhere in the PTM. The exposition is made more concise and direct by introducing Zref explicitly.

OCR for page 459
APPENDIX F 461 For the lower basin, CSM Unit 3, the source types used are riverbed sediments, banks and levees, wetland sediments, lake sediments, other flood- plain sediments, Cataldo/Mission Flats dredge spoils, and a composite of all the source types (unremediated sources). At some time t after remediation (the time of which defines t = 0), the dissolved metal loading F(t) at the same specific location is written as F(t) = R(t)L, (1.2) where R(t) is a remediation factor at time t.4 R(t) is a moving 1-year time average (it is defined in the PTM, p. B-4, as representing “one-year averages over each water-year”). The immediate effect of remedial action on each contamination source is supposed to be a reduction in the relative loading potential of that source by a “remedial action effectiveness” factor Rij for the source type j and with index i, so that immediately after remediation the remediation factor is given by K  Nj    ∑ RLPj  ∑ Vij Rij  j =1   i =1  . R(0) ≡ R0 = (1.3) K  Nj    ∑ RLPj ∑Vij  j =1  i =1    For future times, R(t) is written as R(t) = R0 exp(−βt), (1.4) where the decay rate β is estimated as the ratio of the preremedial “total effective mass” of metal (TEM′) available for leaching and the average preremedial rate W at which metal is removed via the river, or as the ratio of the same quantities immediately postremediation5 4Transient effects due to the remediation efforts themselves (e.g., stirring up sediments during remedial actions) are explicitly ignored and implicitly assumed to have no lasting effect. 5The PTM uses the symbol L to represent what I here call W. The notation in the PTM becomes confused, particularly in section B.2.2.2 starting on p. B-16, in not distinguishing 1-year time averages from instantaneous values. In Equation 1.5 above, F(0) is used as in the PTM (p. B-18, equation 6), but what is meant is a time-averaged version of F, because F is defined as proportional to L, which is not time-averaged (PTM, p. B-4, equation 1, and Equation 1.2 above).

OCR for page 459
462 APPENDIX F β =W / TEM ′ = F(0) / TEM ′′, (1.5) where 1 τ W= Ldt , (1.6) τ ∫0 and τ is 1 year, while the total effective mass of metal (TEM′ preremediation and TEM″ postremediation) available for leaching are assumed to be com- putable as K Nj TEM ′ = γCs ∑ RLPj ∑ Vij j =1 i =1 Nj (1.7) K TEM ′′ = γCs ∑ RLPj ∑ RijVij j =1 i =1 where γ = volumetric unit weight of the reference source type, and Cs = volumetric average metal concentration of the reference source type. Finally, the “load ratio”, Lr(t), is defined by Lr(t) = F(t) / CL , (1.8) where CL, the “loading capacity,” is the product of ambient water-quality criterion (AWQC) and river flow rate Q: CL = AWQC ∗ Q. (1.9) The AWQC is a concentration of the dissolved metal in water and is defined by regulation at a value that is supposed to be protective of fresh- water life. For many metals (and zinc in particular), the AWQC increases with the hardness of the water, and the hardness of the water in the Coeur d’Alene River varies inversely with the flow rate. The AWQC represents the target for most ecological cleanup efforts, in particular for the cleanup of the Coeur d’Alene River, so that a load ratio of unity represents the ulti- mate cleanup target. The PTM evaluates estimates only for dissolved zinc, claiming that results for other dissolved metals except lead could be obtained approxi- mately from those of zinc by using suitable scaling factors (PTM, p. 1-8, section 1.4). The above summary makes no mention of uncertainties in measure- ment of the various quantities discussed (for many of the quantities), of

OCR for page 459
APPENDIX F 463 their variability due to their unpredictable fluctuations with time, or of the correlations between such uncertainties or variabilities. The PTM attempts to account probabilistically for the uncertainties and variabilities. It does this analytically by assuming wherever necessary that uncertainty and vari- ability distributions are lognormal and matching means and coefficients of variation (equivalently, standard deviations). That is, uncertain or variable quantities included in equations are assumed to have that uncertainty or variability represented by lognormal distributions, and the mean and coef- ficient of variation for the quantity on the left-hand side of the equation are obtained as the mean and coefficient of variation of the expression on the right-hand side of the equation (even if, strictly speaking, the combination of uncertainty distributions on the right-hand side of the equation does not result in a lognormal distribution). DEFICIENCIES OF THE PTM The PTM suffers from multiple invalidating deficiencies in its formula- tion and application. The formulation in the PTM goes into considerably more detail (PTM, appendix B) than indicated by the summary given above (which itself contains invalidating deficiencies); however, most of that de- tail is trivial, in the sense that it is just application to specific cases of the general methodology given in PTM appendix A for combining lognormal distributions. Addition of that detail is unnecessary and substantially re- duces the comprehensibility of the PTM. Moreover, there are several in- stances (described below) where that detail is incorrect either conceptually (through confusion of uncertainty and time variation) or because the equa- tions are incorrect (apparently because of typographical errors in most cases). I did not examine the implementation of the methodology described in the PTM (in the form of the PAT1 and PAT2 spreadsheets;6 PTM, p. 3-1) in sufficient detail to comment on that implementation, because of the deficiencies identified here. It is claimed that: “The analysis results are estimates: engineering ap- proximations based on interpretation and synthesis of information avail- able at this time” (PTM, p. 1-5). It is further claimed that: “The estimates are objective within common standards of engineering practice and applied science. They are scientifi- cally sound and technically defensible within the limits of available infor- mation and adequately support informed risk management decisions” (PTM, p. 1-5) (the same claims are made in PTM, p. C-1). 6The committee was provided with copies of these spreadsheets. It is unclear if they were part of any public record until that time. I believe they should have been.

OCR for page 459
464 APPENDIX F Unfortunately, however, simply stating such claims does not make them true; in this case, they are not true. The analysis presented in the PTM lacks any scientific basis. Four reasons for this conclusion are summarized here: the dependence of the entire analysis on an untested hypothesis; the incor- rect treatment of time variation; the use of undocumented, un-validated, and nonreproducible values for parameter values; and incorrect handling of certain probabilistic aspects of the analysis. THE BASIS OF THE ANALYSIS IS AN UNTESTED HYPOTHESIS The analysis in the PTM is based entirely on an untested hypothesis for which no theoretical or experimental evidence is presented. The PTM is explicit in admitting that its entire basis is a hypothesis; for example: The relative load reduction is hypothesized proportional on average to the volume remediated for a given source type and alternative-specific reme- dial action. This hypothesis generalizes the practical approximation that the load reduction from a given source and remedial action is proportional to the volume remediated. (PTM, p. 1-14, italics in original) It was hypothesized that post-remediation loading reductions for a given source type and remedial action (e.g., removal and placement of impacted sediments into a repository) were proportional, on average, to the volume remediated. (PTM, p. 2-29, italics in original) But there is no attempt to justify the use of this hypothesis in the context of remedial actions either by reference to any experimental data or by presentation of plausible theoretical ideas. The statement that the hy- pothesis “generalizes the practical approximation” begs the question, be- cause there is no demonstration of any such practical approximation in the PTM. An attempt is made (PTM section B.2.2, pp. B-20 to B-25) to justify the hypothesis as the “most credible,” but that attempt is irrelevant to the hypothesis stated; it addresses a different problem entirely—namely, the time rate of change of loading (which is addressed separately below). The failure to present any evidence for the hypothesis would not necessarily render the claims of the PTM incorrect if the hypothesis were in fact correct or a reasonable approximation. Some theoretical ideas suggest that it is not correct;7 but the lack of any leaching experiments on any of the materials in 7For example, the PTM at (p. B-11) points out that loading from each source will occur as the result of at least four mechanisms: erosion, infiltration of surface water, leaching caused by groundwater fluctuations, and leaching by groundwater flow. Under certain physical con- ditions these mechanisms could produce loads proportional to source area for the first three examples (erosion, infiltration, groundwater fluctuations) or source linear dimensions for the

OCR for page 459
APPENDIX F 465 the basin, the lack of concentration measurements in groundwater, and the very limited information on groundwater flow deny the information needed to evaluate the hypothesis or propose any more correct one on which to build a plausible analysis. THE EVALUATION OF TIME IS INCORRECT Even if the principal hypothesis used in the PTM was correct and the calculation of the immediate postremediation situation was adequately ap- proximated, the treatment of time variation following remediation is incor- rect. This treatment is essentially captured in the summary above by Equa- tions (1.4), (1.5), and (1.7). The PTM claims (PTM, section B.2.2.2, starting at p. B-16) that the decay rate β is the same for all times and all remedial scenarios. Unfortunately, the analysis leading the PTM to such conclusions is incorrect in two ways. (1) The “relative loading potential” (RLP) intro- duced by the PTM is defined to account for the rate of leaching of metal from source material; it does not in any way represent the total mass of metal ultimately available for leaching or erosion. Even the original defini- tion of β (Equation 1.5) thus does not define a decay rate for the available leachable metal. (2) The PTM analysis that purports to show that there exists a constant decay rate, β, is based on (at least) two incorrect assump- tions and is itself incorrect. The Time Scale for Loading or Concentrations Varies with Remedial Option The first of these problems is easy to detect. The PTM analysis purports to show that the exponential decay rate for annual average loading or concentration is the same for all remedial actions (including no action). Assuming for arguments sake that the loading and concentration do de- crease exponentially, it is obvious that the decay rate cannot be the same for different remedial scenarios. Only one remedial option (chemical fixation) has the potential to substantially change the total amount of metal that ultimately could leach or erode down the Coeur d’Alene River (all other options simply reduce the rate of leaching or erosion). Because the expo- last (groundwater flow). If all sources were the same depth, the first three might be considered proportional to source volume, but the fourth would not. However, under different physical conditions these mechanisms would produce loads that differed in their relationship to source volume. Even if the physical conditions were just right to produce loading proportional to source volume, it does not follow that loading reduction is proportional to the reduction in source volume due to remediation, because remedial action may alter the relevant physical conditions.

OCR for page 459
466 APPENDIX F nential decay rate is just the ratio of the rate of transport down the river to the total mass ultimately available for leaching, reducing the rate of trans- port (the aim of the remedial actions) necessarily will decrease the decay rate (unless the only remedy applied is chemical fixation). All the available metal ultimately will leach or erode into the river and be carried down- stream; if the rate of leaching and erosion is reduced, the time scale over which leaching or erosion occurs is correspondingly increased. To explain where the fallacy arises in the PTM analysis, recall that the “relative loading potential” (RLP) is introduced (PTM pp. 2-17 to 2-18) in an attempt to take account of the differences between various source mate- rials in the combination of metal concentration and mass, its relative mobil- ity, and its exposure to leaching or erosion. Conceptually, therefore, the RLP is not proportional to metal mass available for leaching or erosion (that is, conceptually at least, two source types with substantially different average metal masses per unit volume available for leaching or erosion may have identical RLP values, and two source types with substantially different RLP values may have identical average metal masses per unit volume avail- able for leaching or erosion; in practice, as discussed below, it is unclear how the RLP values were derived). For example, the RLP for waste rock piles in upland areas may be very low compared with tailings-affected sediments (the PTM, p. C-6, gives an estimate value of 0.001 to 0.005 for upland waste rock, compared with 1 for the reference source, tailings- affected sediments), but that tells us nothing about the relative mass per unit volume ultimately available for leaching in these two source types. The preremedial total effective mass (TEM′) introduced in the PTM (Equation 1.7 and PTM p. B-15, equation 8) is thus conceptually related to loading, and the same goes for the postremedial total effective mass (TEM″), so that in concept it may be adequate to write the loading as proportional to this total effective mass; that is (PTM, p. B-16, equation 1, but see footnote 5) W = βTEM ′ pre-remediation (1.10) F(t) = βTEM ′′ post-remediation However, even if the preremediation total effective mass (TEM′) were somehow to represent the total mass available for leaching or erosion (as could happen in principle if the metal in all sources were present at the same concentration, equally mobile, and equally exposed to leaching or erosion), the same would not be true of the postremedial total effective mass (TEM″), because this is conceptually obtained by incorporating the remedial action effectiveness factors Rij. These factors measure the extent to which remedial actions reduce the loading potential—that is, the rate of leaching or ero- sion; in principle, they have nothing to do with changing the mass that is

OCR for page 459
APPENDIX F 467 available for leaching or erosion. Only one of the potential remedial actions (chemical fixation) is likely to have any substantial effect on the total mass ultimately available for leaching or erosion. It is therefore incorrect to write (PTM, p. B-16, equation 2) dTEM ′′ = −βTEM ′′ (1.11) dt The right-hand side represents the loss of metal mass down the river, but the left-hand side bears no relation to the rate of change in total metal mass ultimately available for transport down the river. Equation 1.11 therefore is not a mass balance equation, and all the arguments about mass balance in the PTM (e.g., p. B-18) fail for the same reason—that TEM′ and TEM″, despite the name given to them, have nothing to do with the total metal mass available for leaching or erosion. As a consequence of this failure, the PTM fails to appreciate that the time scale for leaching and erosion will change under different remedial options, and the entire evaluation of the future course of concentrations and loadings is completely incorrect (and even for the unremediated case, the “decay rate” obtained is incorrect). The Timecourse of Loading or Concentration Is Not Exponential Another error in the PTM analysis occurs in the assumption that the time course of loading or concentration will be exponential either before or after remediation. There is a long argument given (PTM, pp. B-20 to B-25) that purports to demonstrate that relationships of the form F(t) = β nTEM ′′(t)n dTEM ′′ (1.12) = − F(t) = −β nTEM ′′(t)n dt (PTM, p. B-20, equations 12 and 13), with (implicitly) constant n, are sufficiently general to be all that must be examined, and that the value n = 1 is the “most credible” (PTM, p. B-24).8 However, this argument is based on multiple fallacies, among which are the following: 8I have combined equations 12 and 13 (PMT, p. B-20) because this is the only way in which TEM″(t) is anywhere defined for arbitrary time t; the definition of TEM″(0) (immediately postremediation) is given in Equation 1.7.

OCR for page 459
468 APPENDIX F A Belief in the Generality of Equations 1.12 It is stated that “By varying exponent n, the relationship F(t) = βnTEM″(t)n could allow loading to be any hypothetical yet plausible con- tinuous function of total effective mass” (PTM, p. B-20), and then, after allowing the coefficient βn to be essentially an arbitrary function of time,9 “it would tentatively appear that F(t) = β TEM″(t)n could approximate the n net effect of any plausible theory of geochemical dependence between metal mass and loading.” These statements are either trivial (and useless) or meaningless. At time t = 0, the first equation with n ≠ 1 is incorrect by definition of TEM″(0) (see Equation 1.7), because TEM″(0) was explicitly constructed (all its terms were defined) so that the loading (F) was propor- tional to it. To make any meaningful statements requires definitions that can support some meaningful interpretation, and no such definitions are provided in the PTM; in this sense, alternatively, the PTM argument is trivial (but useless) in that it can mean whatever anybody wishes. Even if the statements were not meaningless or trivial, they would not be correct as used in the arguments, where βn and n are treated as constants. There are many potential leaching behaviors that cannot be represented by such func- tional forms (e.g., a constant leaching rate for some period followed by a decline that can be modeled as an error function, as might occur for infiltra- tion of groundwater into a waste pile). The Fallacy of Equating TEM″ to the Mass of Metal Available for Leaching or Erosion This was already pointed out. The second of Equations 1.12 has no physical meaning; it is not the mass balance equation that the PTM as- sumes. Although the right-hand side could represent the rate of loss of mass (if the definition of TEM″ were to be suitably modified at arbitrary times to account for n ≠ 1), the left-hand side is not the rate of change of mass available for leaching or erosion. The Fallacy That “n = 1 Is the Only Non-zero Value of n That Yields Physical Reasonable Results That Are Independent of Arbitrary Changes in Loading History” (PTM, p. B-24) This obtuse phrase is used to represent the (false) conclusion of the PTM (obtained on p. B-24) that the solution of the second of Equations (which is PTM equation 13 on p. B-20) is multivalued (“for any . . . arbitrary time periods such that tp1 < tp2 < tp3 < . . .< tpX . . . load F(tpX) 9But subsequently (in the argument) it is treated as a constant.

OCR for page 459
APPENDIX F 469 depends on the arbitrary time periods tp1 through tpX”, PTM, p. B-24).10 The error in the PTM probably arises from the careless use of notation—the substitution βn = β/TEM″ on p. B-20 followed by βn = F0/TEM″n on p. B-21 apparently without the realization that this makes β a function of F0. As a result, equation [18] (PTM, p. B-21) would more clearly be written as follows: n /(1− n ) ( F(t) = F0 1 + β1/ n F0(n −1)/ n t (n − 1) n ) , (1.13) 3 in which form it is immediately apparent that no such problem arises as imagined in the PTM (p. B-24), and the solution F(t) exists and is single- valued for all finite positive real t and all n (including n = 1 and n = 0 as limiting cases). In reality, the time course of loading even from a single uniform homo- geneous source need not be exponential. For example, consider the aver- age11 load due to infiltration of rainwater through an initially uniform waste pile, in which there is sufficient time for the infiltrating water to reach equilibrium with the waste before exiting at the bottom of the pile. In this situation, there may be a long period when the average load is constant as the infiltrating water removes contaminant from the upper part of the waste pile, exiting the waste pile with a constant concentration equal to the equilibrium concentration. The location of the dividing line between leached and unleached waste will travel downward through the waste pile until it reaches the bottom, when there may be a relatively rapid drop in loading from that waste pile (that in some circumstances can be modeled by an error function). Many other situations can easily be envisioned, and the physical situation for erosion, infiltration, groundwater leaching, and other mechanisms may all be different. It is then obvious that the time course of loading (in particular to the Coeur d’Alene River) can be extremely complex, as it will be the sum of many components from different sources each (potentially) with a different time behavior. For example, in the unlikely event that all sources do exhibit exponential behavior (but with different decay constants), the overall load- ing to the river will be a weighted sum of many exponentials with those different decay constants. It is plausible that this weighted sum might be- have approximately as a power law with time (e.g., consider the case of 10Carrying the “argument” of the PTM to its logical conclusion, F(t) is indeterminate for any t > 0 unless n = 1, contrary to a general theorem on the existence and uniqueness of solutions of differential equations! 11The argument given here applies to a time average over periods of over 1 year. Actual loads of course fluctuate on a shorter time scale due to variation in rainfall, pressure, tem- perature, variation in covering vegetation, and other conditions.

OCR for page 459
474 APPENDIX F will also be lognormally distributed. An important implication of both pre- and post-remediation loading being lognormal is that the effects of remedial action should also be lognormal (because products and quotients of lognormal distributions are also lognormal. . . . (PTM, p. 2-10) However, the claim is a complete nonsequitur and simply incorrect. The lognormality of the distribution representing variability in time of concentrations, stream flows, loadings, or other physical quantities has nothing to do with the shape of the uncertainty distribution for the effects of remedial actions. All the probability distributions for remedial actions presented in the PTM are uncertainty distributions (the PTM is not explicit, but no other interpretation is plausible). If, by some chance, the variability in time is what is contemplated in the PTM for one or more of the distribu- tions given for the remedial actions, then there is no implication. In that case, the remedial actions are presumably consistent with the “underlying phenomena” (whatever those are supposed to be). Moreover, as stated elsewhere (PTM, p. A-14), In addition, although theoretically, the sum of independent lognormal dis- tributions is not lognormal, it can be demonstrated by simulation that the sum closely approximates a lognormal PDF. Therefore, the sum of indepen- dent lognormal distributions can also be approximated as lognormal. Thus, the analysis is based on approximations anyway; so one might as well admit from the start that it is approximate, the same approximations would apply to the remedial actions, and no such conclusion can be drawn about any distribution for remedial actions. Erroneously Implying a Correlation It is concluded that there is some correlation between L and R(t) (PTM pp. B-37 to B-38): Estimates of the correlation between L and R(t) (as measured by plnL,lnR) were based on professional judgment and interpretation of potential reme- dial action behavior. Although there is no practical way to quantitatively predict the correlation, it is expected that remedial action will generally be relatively more effective at reducing high loadings (which correlate with high flow conditions) than reducing low loadings (which correlate with low flow conditions) such that L and R(t) will be negatively correlated. The midrange value of plnL,lnR = –0.5 was considered reasonable. Apart from the total lack of basis for any particular numerical value, as explicitly admitted, the whole concept of this correlation is erroneous. L is

OCR for page 459
APPENDIX F 475 the loading, with a distribution arising from its variability in time, particu- larly its variability during the year. R(t) is explicitly defined to be a time average over a year (PTM, p. B-4); there can be no correlation on this basis alone.15 More to the point, the distribution associated with R(t) is an uncertainty distribution, with nothing whatever to do with variability in time, so the concept of correlation does not even apply. What has been done in the PTM is to cancel out (by applying a negative correlation) some of the uncertainty in R(t) with the time variability of L! The “correlation” that is described in the cited paragraph is more accurately a claim that there is a functional relationship between the parameters of the distribution rep- resenting the time variability of L and the actual value of R(t)—specifically, that the upper end of the distribution of L is modified by the value of R(t). A potential way of modeling such an effect would be to treat the standard deviation of the distribution of L as a function of R(t). In this case, how- ever, there is no basis provided that the claim is accurate and that “remedial action will generally be relatively more effective at reducing high loadings (which correlate with high flow conditions) than reducing low loadings (which correlate with low flow conditions).” Whether this claim is true depends on details about leaching and erosion from each source, details that are not documented or (apparently) even examined in the PTM in reaching its conclusion. An Attempt to Estimate the Wrong Correlation The load ratio is defined by Equation 1.8 above (PTM, p. B-53, equa- tion 1); that is, Lr(t) = F(t) / CL , (1.14) and the metal loading F(t) is given by Equation 1.2 above (PTM, p. B-4, equation 1); that is, F(t) = R(t)L, (1.15) so that Lr(t) = R(t)L / CL . (1.16) Both L, the preremedial loading, and CL, the loading capacity, vary with time throughout the year, whereas R(t) is defined to be a yearly average. 15I discount as too unlikely the possibility that the PTM was implying a correlation between the uncertainty distributions for the parameters of the (current) loading and the (future) remedial effectiveness.

OCR for page 459
476 APPENDIX F Associated with R(t) is an uncertainty distribution but no unpredictable time variability (R(t) varies with time, but smoothly and in a predictable fashion), whereas the distributions associated with both L and CL are due to their (unpredictable) time variation (strictly, there are also uncertainty distributions associated with the parameters of the distributions describing their time variation, because of finite numbers of measurements, but these are ignored here, just as they are ignored in the PTM). There is a very high correlation between measured values for L and CL (the correlation coeffi- cient between their logarithms is approximately 0.95)16 but none between R(t) and L or CL (as discussed for L; the same arguments apply to CL as to L). The PTM (p. B-53, equations 1, 2, and 3), however, obtains the uncer- tainty distribution for F(t) at a random time within about a year of t by combining the time-variability distribution for L with the uncertainty dis- tribution for R(t). It then attempts to argue about the correlation between the resulting uncertainty distribution and the time-variability distribution for CL based on the correlation between L and CL. It states (pp. B-53 to B- 54): The future correlation between lnF(t) and lnCL, measured by plnF,lnCL, is expected to be very high. This expectation is based on an almost perfect correlation (p = 1.0) between lnCL and lnQ and a virtually certain high future correlation between lnF(t) and lnQ, just as there has been histori- cally between discharge and loading (which, being a function of discharge, induces correlation). In addition, as further discussed in Section B.3.4.1, and independent statistical analysis of the zinc concentrations, water hard- nesses, and discharge data corresponding to that used in developing the TMDL loading capacities for SF271 (EPA 2000) showed a correlation coefficient of 0.95 between the natural logs of zinc loadings (computed as the product of concentration and discharge) and the equivalent loading capacities (computed as the product of the zinc AWQC(H) and discharge). Consistent with this information, a value of plnF,lnCL = 0.9 was used in the analysis. There is no basis for the selection of the particular value 0.9. It is not possible to state whether it is “consistent with this information” without further examination, but in general it is not consistent with that informa- tion. The effect of assuming a high correlation between ln(F(t)) and ln(CL) is to substantially cancel the uncertainty in R(t) with the time variability in CL; but this cancellation is purely fictitious. This error compounds the 16C is measured by measuring the hardness of the water and the flow rate simultaneously, L computing the AWQC from the hardness, and forming the product of AWQC and flow rate. L is measured by measuring the metal concentration and flow rate and forming the product.

OCR for page 459
APPENDIX F 477 previous erroneous cancellation of the uncertainty of R(t) by the time vari- ability of L discussed above. The effect of these two incorrect cancellations can be large. This may be illustrated by supposing that what is required is the uncertainty distribution for Lr(t) at a random time, so that it is legitimate to (correctly) combine the uncertainty of R(t) with the time variability of the ratio L/CL in Equation 1.16. For dissolved zinc at location SF271 on the Coeur d’Alene, the mea- sured standard deviation of (the time variability of) ln(L) is 0.525, that of ln(CL) is 0.643, and that of the logarithm of their ratio, ln(L/CL), is 0.22517 (obtained from the joint measurements of concentration, hardness, and flow rate; [EPA 2000, for hardness and flow measurements; URS Greiner Inc. and CH2M Hill Inc. 2001b, for dissolved zinc and flow measure- ments]).18 With these measured standard deviations for ln(L) and ln(CL), Table F-1 shows the correct calculation of the random-time uncertainty for ln(Lr(t)) compared with that obtained by including the two erroneous cor- relations introduced in the PTM for various values of the standard devia- tion of ln(R(t)). The error introduced is clearly substantial for any plausible estimates for uncertainty in ln(R(t)). INCORRECT OR MISLEADING STATEMENTS IN THE PTM The following is an incomplete sampling of various incorrect or mis- leading statements and equations in the PTM. Attempting to list all such erroneous statements and equations would be too time-consuming, so the failure to list any statement or equation in this list cannot be considered an endorsement of the correctness of any statement or equation not listed here. • The term “power series” is used incorrectly throughout Appendix A. Where “power series” is used, the correct term would be something like “power product.” The discussion is not of power series in one or more random variable, but the product of powers of random variables. • “Minimum statistical assumptions are required” (PTM, p. A-13). No basis is provided for this statement. One can assume anything, but that does not make it correct, or even consistent, or useful. 17The distributions for ln(L) and ln(C ) are not distinguishable from normal (p = 0.42, L 0.44, respectively, Shapiro-Wilk test). The distribution for L/CL is closer to normal than lognormal (p = 0.39, 0.09, respectively; Shapiro-Wilk test) using the available data. 18Measurements taken on the same day were assumed to be simultaneous, and multiple measurements on the same day were averaged for the analysis. Only the subset of data with simultaneous hardness, flow rate, and concentration data are included in the statistics given here.

OCR for page 459
478 APPENDIX F TABLE F-1 Effect of the Two Erroneous Correlation Calculations Introduced in the PTM Standard Deviation of ln(Lr(t)) Standard deviation of ln(R(t)) Correct PTM 0.0 0.225 0.285 0.3 0.375 0.306 0.6 0.641 0.280 0.9 0.928 0.347 1.2 1.221 0.541 1.5 1.517 0.791 • “A lognormal PDF is believed to be a maximum entropy PDF for the log of variables where only the expected value and coefficient of variation of the distribution is known or estimated. Maximum entropy estimates give the ‘least prejudiced, or least biased, assignment of probabilities’” (Harr 1987) (PTM, p. A-13, footnote 9, italics in original). No connection is proposed between “minimum statistical assumptions” and “maximum en- tropy.” Nor is any application to the problem at hand proposed; on what basis, for example, is it supposed that only the expected value and coeffi- cient of variation are known for the log of variables, and how does this connect, for example, with the evaluation of probability to exceed the AWQC? • “a correlation coefficient of –1.0 implies perfect inverse linear corre- lation (i.e., X1 and X2 are inversely proportional)” (PTM, p. A-8). This is incorrect; perhaps what was intended is that if the correlation coefficient between logarithms ln(X1) and ln(X2) is –1.0 then X1 and X2 are inversely related (but not necessarily in direct inverse proportions). • “Unbounded positive values are allowed (which is generally conser- vative because it tends to overestimate true values)” (PTM, p. A-13). It does not follow that lognormal distributions lead to “generally” conservative estimates, without specifying the universe of discourse. For example, if some variable is (erroneously) assigned a lognormal distribution, and that variable occurs in the denominator of an expression, the result may be an underestimate rather than an overestimate. On the other hand, the inverse of a lognormal distribution is also lognormal, so the preceding example also shows that (erroneously) assigning a lognormal distribution to an expression in the numerator can lead to underestimates—because a lognor- mal distribution also allows unboundedly small values. • “Any PDF can be conservatively approximated using a lognormal PDF that envelopes the PDF over the range of interest” (PTM, p. A-13). Again, this statement is meaningless without a definition of “conserva- tively,” “envelopes,” and “range of interest” at the least. Even with such

OCR for page 459
APPENDIX F 479 definitions, it is likely to be untrue in general. Indeed, it is quite likely that a converse theorem holds—for any lognormal approximation to a given PDF, there exist statistics of that PDF that are not conservatively estimated by the lognormal approximation. • “Variables CDS and CS are, respectively, the metal (zinc) concentra- tion of the deeper sediments and floodplain sediments having RLP = 1. These sediment concentrations will be positively correlated” (PTM, p. B-47). “Also, because of the way CDS and CS were estimated, they would be positively correlated” (PTM, p. C-9). It is quite plausible that the con- centrations of deeper sediments and floodplain sediments are correlated spatially—that is, the concentration would tend to be higher in the deeper sediments beneath floodplain sediments with higher concentrations. Such a spatial correlation is entirely irrelevant, however, for variables CDS and CS, which are defined to be “volumetric average concentrations in the deeper impacted sediments” and “volumetric average concentration in the im- pacted sediments having an RLP = 1” (PTM, p. B-14). Any spatial correla- tion is entirely removed by the averaging. What is required is any correla- tion between the uncertainty distributions for these volumetric averages. No such correlation is induced “because of the way CDS and CS were estimated.” The only documented “estimation methods” are given in sec- tion C.2.4, where uncertainty confidence intervals for the values of CDS and CS are supposedly (very loosely) based on observed data in the BHSS and a background estimate based on measurements outside the BHSS. Nothing in the measurements supposedly used or in the described derivation correlates these uncertainty distributions; the fact that the same value is used as the lower uncertainty confidence bound for one and the upper uncertainty confidence bound for the other is the only connection between them, and that has no such effect. The subsequent estimate of a value of 0.5 for the correlation coefficient of this hypothetical, nonexistent correlation is sim- ply incorrect. • “An estimate of CV[M] = 0.5 was used in the analysis” (PTM, p. B-48). There is no basis given for this estimate. Nor is it clear why it was introduced, except to arbitrarily increase the uncertainty estimate. • “Since the estimates for L and TEM′ are independent of each other, plnL,lnTEM, was set to zero in the analysis. This lack of correlation in the estimates should not be confused with the positive correlation that must exist in the true values of L and TEM′, and is otherwise inherent in the data used to make the estimates. To the extent there was (positive) correlation between the estimates of L and TEM′, it would decrease both E[β] and CV[β]” (PTM, p. B-48). This statement demonstrates complete confusion, apparently stemming from a misunderstanding of what “correlation” means or perhaps the confusion in this document between measurement uncer- tainties, variability in time, and functional relationships. There is obviously

OCR for page 459
480 APPENDIX F no correlation possible between true values of L and TEM′,19 which are single values. • “The BHSS data do not represent the true values of CDS and CS, which are uncertain” (PTM, p. C-10). True values cannot be uncertain, although they may be unknown, so that we are uncertain about what they are. • “For example, the correlation coefficient between the natural logs of Q and H is 0.96 for the SFCDR at SF271 . . . the correlation coefficient between the natural logs of AWQC and Q at SF271 is also 0.96 for the TMDL data set” (PTM, p. B-26). Both these correlation coefficients are –0.96, not +0.96. • Page 1-1, footnote 1, the conversion factor is actually 0.005394 to 4 significant figures, or 0.00539 to 3 significant figures. The value used should at least be the correct rounding of the exact value. • Page A-24, equation [2] is incorrect. The correct expression is  n ai ( ai −1)/ 2   i =1 a ( E(X) =  ∏ E[ Xi ] i 1 + CV [ Xi ]2 )  Ω,  (1.17) and there is no need to introduce the variables Xi′. Indeed, the entire expo- sition would be greatly clarified by working with statistics of the logarithms of the variables. For example, define Ti = ln(Xi), T = ln(X), and let Xi have mean mi and coefficient of variation ci , Ti have mean µi and standard deviation σi, and similarly for X and T (with no subscripts). Then we have m = exp(µ + σ 2 / 2) (1.18) c2 = exp(σ 2 ) − 1, and similarly for all subscripted variables. Then equations 1 through 3 of PTM (p. A-24) become the considerably simpler equations: n µ= ∑aµ i i i =1 (1.19) n 2 2 2 σ = ∑a σ i i + 2∑ ρij ai aj σ i σ j , i =1 i OCR for page 459
APPENDIX F 481 S S S Lj / Vj = ∑ L ∑V js js ≠S ∑ (L js / Vjs )−1 , (1.20) 2 s =1 s =1 s =1 and the inequality applies except in certain special cases (which do not apply in general in this application). • Page B-12 and C-5, the last two entries in equation 4 of p. B-12, and the same equations repeated in section C.2.3 for RLPj are incorrect if any attempt is made to interpret them according to standard conventions. The first and second entries of equation 4 of p. B-12 correspond to the defini- tions given. It is just possible to interpret the last two entries in equation 4 of p. B-12 and the same equations in section C.2.3 in the correct sense if the phrases “per unit volume of source type j and FP” and “per unit load of source type j and FP” are interpreted as applying separately to the numera- tors and denominators of the respective equations, contrary to any stan- dard convention; coming on these equations by themselves (without the correct definition) in section C.2.3 is disconcerting. • Page B-26, “The analysis used the same discharge and H(Q) and AWQC(Q) relationship used in EPA 2000 for the TMDL.” This statement is incorrect, and the approach taken in the PTM is inconsistent with the intent of performing an uncertainty analysis. First, the statement is incor- rect because the relationship assumed in EPA (2000) was linear between hardness itself and the logarithm of flow rate, whereas the relationship assumed in the PTM is linear between the logarithm of hardness and the logarithm of flow rate. Second, the approach taken in the PTM is inconsis- tent, because the PTM analyzed the loading capacities derived for regula- tory purposes in EPA (2000). However, those loading capacities already have built into them the results of an uncertainty analysis; the loading capacities are derived as 90th percentiles of an uncertainty distribution.20 That uncertainty analysis should be incorporated in the PTM as part of the overall uncertainty analysis—the PTM should evaluate the original data, not the summary statistics produced by EPA (2000). • Pages A-18 to A-19 and B-33 to B-34, the technique used to estimate parameters (mean and standard deviations of the logarithm) of lognormal distributions by regressing order statistics of the logarithms of measure- ments against the “plotting points” (p. A-18, equation 8, and p. B-34, equation 1 has nothing to recommend it. The “plotting points” used are 20EPA (2000) states (p. 22) that the values were lower bounds of a 90th percentile confi- dence interval, thus at the 95th percentile. This is incorrect, however. The values obtained by EPA (2000) are the 90th percentiles (lower bounds of an 80th percentile confidence interval).

OCR for page 459
482 APPENDIX F only approximations of the expected values of the normal order statistics, so the technique is approximate at best (better approximations of normal order statistics are available (Royston 1993, 1995). The values obtained for mean and standard deviation are almost certainly biased and have un- known statistical properties. On the other hand, simply computing the mean and (sample) standard deviation of the logarithms of measured values gives unbiased estimates with known (and optimal for certain purposes) statistical properties for these parameters. Unless the PTM justifies the methodology used (by demonstrating, for example, superiority in some sense of the estimates obtained), standard (and simpler) approaches should be used. • Page B-35, equation 6, the right-hand side erroneously uses CV[L] where what is required is CV[C]. The expression for Ω erroneously omits p. • Page B-35, equation 8, the right-hand side erroneously uses CV[L] where what is required is CV[C]. • Page B-35, footnote 17, the expressions could be somewhat simpli- fied if the trivial identity {exp(A)}1/ 2 = exp(A / 2). (1.21) were applied. Better yet would be adoption of the suggestion discussed in the comment on p. A-24. • Page B-36, first equation on page (carried over from equation 10 of p. B-35), the expression for Ω erroneously omits p. • Page B-37, equation 3, the expression for Ω erroneously omits p. • Page B-43, equations 6 and 7, in both these equations the denomina- tors have been written incorrectly, because 2   K Nj  K Nj 2  E  ∑ ∑ RLPj ∗ Vij   ≠ ∑ ∑ E(RLPj ∗ Vij) . (1.22)   j =1 i =1   j =1 i =1 The left side of Equation 1.22 is what is required inside the square root in the denominator of equations 6 and 7, but the right side is what is written. • Page B-57, “The analysis showed the following principal results: Both the AWQC(H) and the equivalent loading capacities were lognor- mally distributed with respective r2’s of 0.94 and 0.97.” The list continues with similar statements about ratios of zinc loadings to loading capacities, zinc concentrations, loadings, and hardness. However, the given informa- tion is not sufficient to support the conclusion of lognormality for these quantities—some values of r2 would be obtained whether or not any par-

OCR for page 459
APPENDIX F 483 ticular distribution was lognormal. It is quite feasible to test whether a set of samples is consistent with lognormality—for example, by using the Shapiro-Wilk test (Royston 1982, 1993, 1995). Applying this test suggests that it is somewhat unlikely that the measured zinc concentrations (p = 0.002), AWQC(H) (p = 0.014), or hardness (p = 0.018) are lognormal, although zinc loading measurements (p = 0.3) and loading capacity (p = 0.5) are consistent with lognormality.21 It is already pointed out in footnote 17 that the ratio of zinc load to the load capacity is more consistent with normality than lognormality. • Page B-52 (section B.3.3.3), “For these reasons and because of its general theoretical and practical basis, Eq 1 was considered a valid and reasonable approximation for estimating CV[R(t)] for the lower basin, with further savings of effort.” But there is no theoretical basis whatever for equation 1, because it is purely an empirical approximation found for the upper basin using the specific values for the upper basin.22 Therefore, there is no basis whatever for extending this empirical approximation to the lower basin (with different source types, different mixes of sources, and so forth)—the results obtained there could be substantially different. REFERENCES EPA (U.S. Environmental Protection Agency). 2000. Total Maximum Daily Load for Dis- solved Cadmium, Dissolved Lead, and Dissolved Zinc in Surface Waters of the Coeur d’Alene River Basin, Appendix I: Hardness Data. EPA 910R01006. U.S. Environmental Protection Agency, Region 10, Seattle, WA and Idaho department of Environmental Quality, Boise ID [online]. Available: http://yosemite.epa.gov/.../ac5dc0447a281f4e 882569ed0073521f/b57be085215b66658825693b0076f71c/$FILE/CdAtecha.pdf [ac- cessed July 27, 2005]. Kaplan, S. 1992. “Expert information” versus “expert opinions.” Another approach to the problem of eliciting/combining/using expert knowledge in PRA. Reliab. Eng. Syst. Safe. 35(1):61-72. Ridolfi (Ridolfi Engineers Inc.). 2000. Containment Cover Evaluation Using the HELP Model, Coeur d’Alene River Basin Feasibility Study. December 2000 (as cited in URS Greiner, Inc., and CH2M Hill 2001a). Royston, J.P. 1982. Algorithm AS 181: The W test for normality. Appl. Statist. 31(2):176- 180. Royston, P. 1993. A toolkit for testing for non-normality in complete and censored samples. Statistician 42(1):37-43. Royston, P. 1995. Remark AS R94. A remark on algorithm AS 181: The W-test for normal- ity. Appl. Statist. 44(4):547-551. 21These statistics are obtained using all the available data, in the same manner as appar- ently used in the PTM. Footnote 17 gives similar statistics for the subset of these data for which all of hardness, zinc concentration, and flow were simultaneously measured. 22It is likely to be substantially incorrect, as indicated by the other comments made here, but that is irrelevant to the current argument.

OCR for page 459
484 APPENDIX F URS Greiner, Inc., and CH2M Hill. 2001a. Probabilistic Analysis of Post-Remediation Metal Loading Technical Memorandum (Revision 1). URSG DCN 4162500.06778.05.a. Pre- pared for U.S. Environmental Protection Agency, Region 10, Seattle, WA, by URS Greiner, Inc., Seattle, WA, and CH2M Hill, Bellevue, WA. September 20, 2001. URS Greiner, Inc., and CH2M Hill, Inc. 2001b. Final (Revision 2) Remedial Investigation Report for the Coeur d’Alene Basin Remedial Investigation/Feasibility Study, Volume 9, Appendix C. URSG DCN 4162500.6659.05a. Prepared for U.S. Environmental Protec- tion Agency, Region 10, Seattle, WA, by URS Greiner, Inc., Seattle, WA, and CH2M Hill, Bellevue, WA. September 2001.