Appendix F
Assessment of the Probabilistic Model for Estimating Metal Loading and Effectiveness of Remedial Action

Edmund A. C. Crouch, Ph.D.

Senior Scientist

Cambridge Environmental, Inc.

Cambridge, Massachusetts

The Probabilistic Analysis of Post-Remediation Metal Loading Technical Memorandum (Revision 1) (URS Greiner, Inc. and CH2M Hill 2001a) (PTM) describes its purpose very well:

The probabilistic analysis is a risk management tool that can help quantify the certainty, conditional on available information and its interpretation, that a proposed remedy would meet cleanup goals. (PTM p. 1-1)


The purpose of the probabilistic analysis is to help support informed risk management decision-making. It does so by helping to quantify the certainty that a remedial alternative or a proposed remedy could actually meet cleanup goals…. (PTM, p. 1-3)

It formulates an approach intended to meet these objectives. The formulation can be readily summarized,1 and this summary is presented first without any comment on its correctness or applicability. It is assumed that dissolved metal loading to the Coeur d’Alene River (e.g., in pounds/day, the

1  

Understanding this appendix will require access to and some familiarity with the PTM.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin Appendix F Assessment of the Probabilistic Model for Estimating Metal Loading and Effectiveness of Remedial Action Edmund A. C. Crouch, Ph.D. Senior Scientist Cambridge Environmental, Inc. Cambridge, Massachusetts The Probabilistic Analysis of Post-Remediation Metal Loading Technical Memorandum (Revision 1) (URS Greiner, Inc. and CH2M Hill 2001a) (PTM) describes its purpose very well: The probabilistic analysis is a risk management tool that can help quantify the certainty, conditional on available information and its interpretation, that a proposed remedy would meet cleanup goals. (PTM p. 1-1) The purpose of the probabilistic analysis is to help support informed risk management decision-making. It does so by helping to quantify the certainty that a remedial alternative or a proposed remedy could actually meet cleanup goals…. (PTM, p. 1-3) It formulates an approach intended to meet these objectives. The formulation can be readily summarized,1 and this summary is presented first without any comment on its correctness or applicability. It is assumed that dissolved metal loading to the Coeur d’Alene River (e.g., in pounds/day, the 1   Understanding this appendix will require access to and some familiarity with the PTM.

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin unit used throughout the PTM) at some specific location on the river, can be calculated as follows: (1.1) where2 L = (preremedial) metal loading in the Coeur d’Alene River at the specific location examined (pounds/day); Zref = “loading potential” per unit volume (pounds/day/cubic yard) for the reference source type for the location on the river under examination (averaged over all sources of that type affecting that location);3 RLPj = “relative loading potential” for the location on the river under examination for a contamination source of type j, averaged over the sources of that type that affect the river location under examination (for the reference source type, the RLP is unity); Vij = volume (cubic yards) of a source of type j with index i that affects the river at the location examined, all such sources being indexed; K = number of different types of sources that affect the river at the location examined; and Nj = number of sources of type j that affect the river at the location examined. The contamination sources are generally volumes of contaminated soil, sediment, and rock, categorized by type. The source types used in the PTM (pp. 2-18 to 2-19), for the upper basin, conceptual site model (CSM) Units 1 and 2, are adits (these are treated specially, by using measured flows and concentrations and deriving an effective volume for them), tailings-impacted floodplain sediments, unimpounded tailings piles, impounded tailings piles at inactive facilities, impounded tailings piles at active facilities, waste rock piles in floodplains, waste rock piles in upland areas, and deeper impacted floodplain sediments (unremediated sources). The reference source type (with RLP = 1) is taken to be tailings-affected floodplain sediments. 2   The notation of the PTM is adopted, except that all symbols are italicized to agree to the degree possible with standard notational conventions (which are not observed in the PTM). The only possible confusion is between the symbols L and L which have distinct meanings in the PTM; however I avoid this confusion by using a different symbol, W, for what the PTM calls L. 3   Only ratios of quantities each of which multiplies what I here call Zref are required in the PTM, so no such term is defined anywhere in the PTM. The exposition is made more concise and direct by introducing Zref explicitly.

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin For the lower basin, CSM Unit 3, the source types used are riverbed sediments, banks and levees, wetland sediments, lake sediments, other floodplain sediments, Cataldo/Mission Flats dredge spoils, and a composite of all the source types (unremediated sources). At some time t after remediation (the time of which defines t = 0), the dissolved metal loading F(t) at the same specific location is written as (1.2) where R(t) is a remediation factor at time t.4 R(t) is a moving 1-year time average (it is defined in the PTM, p. B-4, as representing “one-year averages over each water-year”). The immediate effect of remedial action on each contamination source is supposed to be a reduction in the relative loading potential of that source by a “remedial action effectiveness” factor Rij for the source type j and with index i, so that immediately after remediation the remediation factor is given by (1.3) For future times, R(t) is written as R(t) = R0 exp(–βt), (1.4) where the decay rate β is estimated as the ratio of the preremedial “total effective mass” of metal (TEM) available for leaching and the average preremedial rate W at which metal is removed via the river, or as the ratio of the same quantities immediately postremediation5 4   Transient effects due to the remediation efforts themselves (e.g., stirring up sediments during remedial actions) are explicitly ignored and implicitly assumed to have no lasting effect. 5   The PTM uses the symbol L to represent what I here call W. The notation in the PTM becomes confused, particularly in section B.2.2.2 starting on p. B-16, in not distinguishing 1-year time averages from instantaneous values. In Equation 1.5 above, F(0) is used as in the PTM (p. B-18, equation 6), but what is meant is a time-averaged version of F, because F is defined as proportional to L, which is not time-averaged (PTM, p. B-4, equation 1, and Equation 1.2 above).

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin (1.5) where (1.6) and is 1 year, while the total effective mass of metal (TEM′ preremediation and TEM″ postremediation) available for leaching are assumed to be computable as (1.7) where γ = volumetric unit weight of the reference source type, and Cs = volumetric average metal concentration of the reference source type. Finally, the “load ratio”, Lr(t), is defined by (1.8) where CL, the “loading capacity,” is the product of ambient water-quality criterion (AWQC) and river flow rate Q: (1.9) The AWQC is a concentration of the dissolved metal in water and is defined by regulation at a value that is supposed to be protective of fresh-water life. For many metals (and zinc in particular), the AWQC increases with the hardness of the water, and the hardness of the water in the Coeur d’Alene River varies inversely with the flow rate. The AWQC represents the target for most ecological cleanup efforts, in particular for the cleanup of the Coeur d’Alene River, so that a load ratio of unity represents the ultimate cleanup target. The PTM evaluates estimates only for dissolved zinc, claiming that results for other dissolved metals except lead could be obtained approximately from those of zinc by using suitable scaling factors (PTM, p. 1-8, section 1.4). The above summary makes no mention of uncertainties in measurement of the various quantities discussed (for many of the quantities), of

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin their variability due to their unpredictable fluctuations with time, or of the correlations between such uncertainties or variabilities. The PTM attempts to account probabilistically for the uncertainties and variabilities. It does this analytically by assuming wherever necessary that uncertainty and variability distributions are lognormal and matching means and coefficients of variation (equivalently, standard deviations). That is, uncertain or variable quantities included in equations are assumed to have that uncertainty or variability represented by lognormal distributions, and the mean and coefficient of variation for the quantity on the left-hand side of the equation are obtained as the mean and coefficient of variation of the expression on the right-hand side of the equation (even if, strictly speaking, the combination of uncertainty distributions on the right-hand side of the equation does not result in a lognormal distribution). DEFICIENCIES OF THE PTM The PTM suffers from multiple invalidating deficiencies in its formulation and application. The formulation in the PTM goes into considerably more detail (PTM, appendix B) than indicated by the summary given above (which itself contains invalidating deficiencies); however, most of that detail is trivial, in the sense that it is just application to specific cases of the general methodology given in PTM appendix A for combining lognormal distributions. Addition of that detail is unnecessary and substantially reduces the comprehensibility of the PTM. Moreover, there are several instances (described below) where that detail is incorrect either conceptually (through confusion of uncertainty and time variation) or because the equations are incorrect (apparently because of typographical errors in most cases). I did not examine the implementation of the methodology described in the PTM (in the form of the PAT1 and PAT2 spreadsheets;6 PTM, p. 3-1) in sufficient detail to comment on that implementation, because of the deficiencies identified here. It is claimed that: “The analysis results are estimates: engineering approximations based on interpretation and synthesis of information available at this time” (PTM, p. 1-5). It is further claimed that: “The estimates are objective within common standards of engineering practice and applied science. They are scientifically sound and technically defensible within the limits of available information and adequately support informed risk management decisions” (PTM, p. 1-5) (the same claims are made in PTM, p. C-1). 6   The committee was provided with copies of these spreadsheets. It is unclear if they were part of any public record until that time. I believe they should have been.

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin Unfortunately, however, simply stating such claims does not make them true; in this case, they are not true. The analysis presented in the PTM lacks any scientific basis. Four reasons for this conclusion are summarized here: the dependence of the entire analysis on an untested hypothesis; the incorrect treatment of time variation; the use of undocumented, un-validated, and nonreproducible values for parameter values; and incorrect handling of certain probabilistic aspects of the analysis. THE BASIS OF THE ANALYSIS IS AN UNTESTED HYPOTHESIS The analysis in the PTM is based entirely on an untested hypothesis for which no theoretical or experimental evidence is presented. The PTM is explicit in admitting that its entire basis is a hypothesis; for example: The relative load reduction is hypothesized proportional on average to the volume remediated for a given source type and alternative-specific remedial action. This hypothesis generalizes the practical approximation that the load reduction from a given source and remedial action is proportional to the volume remediated. (PTM, p. 1-14, italics in original) It was hypothesized that post-remediation loading reductions for a given source type and remedial action (e.g., removal and placement of impacted sediments into a repository) were proportional, on average, to the volume remediated. (PTM, p. 2-29, italics in original) But there is no attempt to justify the use of this hypothesis in the context of remedial actions either by reference to any experimental data or by presentation of plausible theoretical ideas. The statement that the hypothesis “generalizes the practical approximation” begs the question, because there is no demonstration of any such practical approximation in the PTM. An attempt is made (PTM section B.2.2, pp. B-20 to B-25) to justify the hypothesis as the “most credible,” but that attempt is irrelevant to the hypothesis stated; it addresses a different problem entirely—namely, the time rate of change of loading (which is addressed separately below). The failure to present any evidence for the hypothesis would not necessarily render the claims of the PTM incorrect if the hypothesis were in fact correct or a reasonable approximation. Some theoretical ideas suggest that it is not correct;7 but the lack of any leaching experiments on any of the materials in 7   For example, the PTM at (p. B-11) points out that loading from each source will occur as the result of at least four mechanisms: erosion, infiltration of surface water, leaching caused by groundwater fluctuations, and leaching by groundwater flow. Under certain physical conditions these mechanisms could produce loads proportional to source area for the first three examples (erosion, infiltration, groundwater fluctuations) or source linear dimensions for the

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin the basin, the lack of concentration measurements in groundwater, and the very limited information on groundwater flow deny the information needed to evaluate the hypothesis or propose any more correct one on which to build a plausible analysis. THE EVALUATION OF TIME IS INCORRECT Even if the principal hypothesis used in the PTM was correct and the calculation of the immediate postremediation situation was adequately approximated, the treatment of time variation following remediation is incorrect. This treatment is essentially captured in the summary above by Equations (1.4), (1.5), and (1.7). The PTM claims (PTM, section B.2.2.2, starting at p. B-16) that the decay rate β is the same for all times and all remedial scenarios. Unfortunately, the analysis leading the PTM to such conclusions is incorrect in two ways. (1) The “relative loading potential” (RLP) introduced by the PTM is defined to account for the rate of leaching of metal from source material; it does not in any way represent the total mass of metal ultimately available for leaching or erosion. Even the original definition of (Equation 1.5) thus does not define a decay rate for the available leachable metal. (2) The PTM analysis that purports to show that there exists a constant decay rate, β, is based on (at least) two incorrect assumptions and is itself incorrect. The Time Scale for Loading or Concentrations Varies with Remedial Option The first of these problems is easy to detect. The PTM analysis purports to show that the exponential decay rate for annual average loading or concentration is the same for all remedial actions (including no action). Assuming for arguments sake that the loading and concentration do decrease exponentially, it is obvious that the decay rate cannot be the same for different remedial scenarios. Only one remedial option (chemical fixation) has the potential to substantially change the total amount of metal that ultimately could leach or erode down the Coeur d’Alene River (all other options simply reduce the rate of leaching or erosion). Because the expo-     last (groundwater flow). If all sources were the same depth, the first three might be considered proportional to source volume, but the fourth would not. However, under different physical conditions these mechanisms would produce loads that differed in their relationship to source volume. Even if the physical conditions were just right to produce loading proportional to source volume, it does not follow that loading reduction is proportional to the reduction in source volume due to remediation, because remedial action may alter the relevant physical conditions.

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin nential decay rate is just the ratio of the rate of transport down the river to the total mass ultimately available for leaching, reducing the rate of transport (the aim of the remedial actions) necessarily will decrease the decay rate (unless the only remedy applied is chemical fixation). All the available metal ultimately will leach or erode into the river and be carried downstream; if the rate of leaching and erosion is reduced, the time scale over which leaching or erosion occurs is correspondingly increased. To explain where the fallacy arises in the PTM analysis, recall that the “relative loading potential” (RLP) is introduced (PTM pp. 2-17 to 2-18) in an attempt to take account of the differences between various source materials in the combination of metal concentration and mass, its relative mobility, and its exposure to leaching or erosion. Conceptually, therefore, the RLP is not proportional to metal mass available for leaching or erosion (that is, conceptually at least, two source types with substantially different average metal masses per unit volume available for leaching or erosion may have identical RLP values, and two source types with substantially different RLP values may have identical average metal masses per unit volume available for leaching or erosion; in practice, as discussed below, it is unclear how the RLP values were derived). For example, the RLP for waste rock piles in upland areas may be very low compared with tailings-affected sediments (the PTM, p. C-6, gives an estimate value of 0.001 to 0.005 for upland waste rock, compared with 1 for the reference source, tailings-affected sediments), but that tells us nothing about the relative mass per unit volume ultimately available for leaching in these two source types. The preremedial total effective mass (TEM) introduced in the PTM (Equation 1.7 and PTM p. B-15, equation 8) is thus conceptually related to loading, and the same goes for the postremedial total effective mass (TEM), so that in concept it may be adequate to write the loading as proportional to this total effective mass; that is (PTM, p. B-16, equation 1, but see footnote 5) (1.10) However, even if the preremediation total effective mass (TEM′) were somehow to represent the total mass available for leaching or erosion (as could happen in principle if the metal in all sources were present at the same concentration, equally mobile, and equally exposed to leaching or erosion), the same would not be true of the postremedial total effective mass (TEM″), because this is conceptually obtained by incorporating the remedial action effectiveness factors Rij. These factors measure the extent to which remedial actions reduce the loading potential—that is, the rate of leaching or erosion; in principle, they have nothing to do with changing the mass that is

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin available for leaching or erosion. Only one of the potential remedial actions (chemical fixation) is likely to have any substantial effect on the total mass ultimately available for leaching or erosion. It is therefore incorrect to write (PTM, p. B-16, equation 2) (1.11) The right-hand side represents the loss of metal mass down the river, but the left-hand side bears no relation to the rate of change in total metal mass ultimately available for transport down the river. Equation 1.11 therefore is not a mass balance equation, and all the arguments about mass balance in the PTM (e.g., p. B-18) fail for the same reason—that TEM′ and TEM″, despite the name given to them, have nothing to do with the total metal mass available for leaching or erosion. As a consequence of this failure, the PTM fails to appreciate that the time scale for leaching and erosion will change under different remedial options, and the entire evaluation of the future course of concentrations and loadings is completely incorrect (and even for the unremediated case, the “decay rate” obtained is incorrect). The Timecourse of Loading or Concentration Is Not Exponential Another error in the PTM analysis occurs in the assumption that the time course of loading or concentration will be exponential either before or after remediation. There is a long argument given (PTM, pp. B-20 to B-25) that purports to demonstrate that relationships of the form (1.12) (PTM, p. B-20, equations 12 and 13), with (implicitly) constant n, are sufficiently general to be all that must be examined, and that the value n = 1 is the “most credible” (PTM, p. B-24).8 However, this argument is based on multiple fallacies, among which are the following: 8   I have combined equations 12 and 13 (PMT, p. B-20) because this is the only way in which TEM″(t) is anywhere defined for arbitrary time t; the definition of TEM″(0) (immediately postremediation) is given in Equation 1.7.

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin A Belief in the Generality of Equations 1.12 It is stated that “By varying exponent n, the relationship F(t) = βnTEM″(t)n could allow loading to be any hypothetical yet plausible continuous function of total effective mass” (PTM, p. B-20), and then, after allowing the coefficient βn to be essentially an arbitrary function of time,9 “it would tentatively appear that F(t) = βnTEM″(t)n could approximate the net effect of any plausible theory of geochemical dependence between metal mass and loading.” These statements are either trivial (and useless) or meaningless. At time t = 0, the first equation with n ≠ 1 is incorrect by definition of TEM″(0) (see Equation 1.7), because TEM″(0) was explicitly constructed (all its terms were defined) so that the loading (F) was proportional to it. To make any meaningful statements requires definitions that can support some meaningful interpretation, and no such definitions are provided in the PTM; in this sense, alternatively, the PTM argument is trivial (but useless) in that it can mean whatever anybody wishes. Even if the statements were not meaningless or trivial, they would not be correct as used in the arguments, where βn and n are treated as constants. There are many potential leaching behaviors that cannot be represented by such functional forms (e.g., a constant leaching rate for some period followed by a decline that can be modeled as an error function, as might occur for infiltration of groundwater into a waste pile). The Fallacy of Equating TEM″ to the Mass of Metal Available for Leaching or Erosion This was already pointed out. The second of Equations 1.12 has no physical meaning; it is not the mass balance equation that the PTM assumes. Although the right-hand side could represent the rate of loss of mass (if the definition of TEM″ were to be suitably modified at arbitrary times to account for n ≠ 1), the left-hand side is not the rate of change of mass available for leaching or erosion. The Fallacy That “n = 1 Is the Only Non-zero Value of n That Yields Physical Reasonable Results That Are Independent of Arbitrary Changes in Loading History” (PTM, p. B-24) This obtuse phrase is used to represent the (false) conclusion of the PTM (obtained on p. B-24) that the solution of the second of Equations (which is PTM equation 13 on p. B-20) is multivalued (“for any … arbitrary time periods such that tp1 < tp2 < tp3 < … < tpX … load F(tpX) 9   But subsequently (in the argument) it is treated as a constant.

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin depends on the arbitrary time periods tp1 through tpX”, PTM, p. B-24).10 The error in the PTM probably arises from the careless use of notation—the substitution βn = β/TEM″ on p. B-20 followed by βn = F0/TEM″n on p. B-21 apparently without the realization that this makes β a function of F0. As a result, equation [18] (PTM, p. B-21) would more clearly be written as follows: (1.13) in which form it is immediately apparent that no such problem arises as imagined in the PTM (p. B-24), and the solution F(t) exists and is single-valued for all finite positive real t and all n (including n = 1 and n = 0 as limiting cases). In reality, the time course of loading even from a single uniform homogeneous source need not be exponential. For example, consider the average11 load due to infiltration of rainwater through an initially uniform waste pile, in which there is sufficient time for the infiltrating water to reach equilibrium with the waste before exiting at the bottom of the pile. In this situation, there may be a long period when the average load is constant as the infiltrating water removes contaminant from the upper part of the waste pile, exiting the waste pile with a constant concentration equal to the equilibrium concentration. The location of the dividing line between leached and unleached waste will travel downward through the waste pile until it reaches the bottom, when there may be a relatively rapid drop in loading from that waste pile (that in some circumstances can be modeled by an error function). Many other situations can easily be envisioned, and the physical situation for erosion, infiltration, groundwater leaching, and other mechanisms may all be different. It is then obvious that the time course of loading (in particular to the Coeur d’Alene River) can be extremely complex, as it will be the sum of many components from different sources each (potentially) with a different time behavior. For example, in the unlikely event that all sources do exhibit exponential behavior (but with different decay constants), the overall loading to the river will be a weighted sum of many exponentials with those different decay constants. It is plausible that this weighted sum might behave approximately as a power law with time (e.g., consider the case of 10   Carrying the “argument” of the PTM to its logical conclusion, F(t) is indeterminate for any t > 0 unless n = 1, contrary to a general theorem on the existence and uniqueness of solutions of differential equations! 11   The argument given here applies to a time average over periods of over 1 year. Actual loads of course fluctuate on a shorter time scale due to variation in rainfall, pressure, temperature, variation in covering vegetation, and other conditions.

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin will also be lognormally distributed. An important implication of both pre- and post-remediation loading being lognormal is that the effects of remedial action should also be lognormal (because products and quotients of lognormal distributions are also lognormal…. (PTM, p. 2-10) However, the claim is a complete nonsequitur and simply incorrect. The lognormality of the distribution representing variability in time of concentrations, stream flows, loadings, or other physical quantities has nothing to do with the shape of the uncertainty distribution for the effects of remedial actions. All the probability distributions for remedial actions presented in the PTM are uncertainty distributions (the PTM is not explicit, but no other interpretation is plausible). If, by some chance, the variability in time is what is contemplated in the PTM for one or more of the distributions given for the remedial actions, then there is no implication. In that case, the remedial actions are presumably consistent with the “underlying phenomena” (whatever those are supposed to be). Moreover, as stated elsewhere (PTM, p. A-14), In addition, although theoretically, the sum of independent lognormal distributions is not lognormal, it can be demonstrated by simulation that the sum closely approximates a lognormal PDF. Therefore, the sum of independent lognormal distributions can also be approximated as lognormal. Thus, the analysis is based on approximations anyway; so one might as well admit from the start that it is approximate, the same approximations would apply to the remedial actions, and no such conclusion can be drawn about any distribution for remedial actions. Erroneously Implying a Correlation It is concluded that there is some correlation between L and R(t) (PTM pp. B-37 to B-38): Estimates of the correlation between L and R(t) (as measured by plnL,lnR) were based on professional judgment and interpretation of potential remedial action behavior. Although there is no practical way to quantitatively predict the correlation, it is expected that remedial action will generally be relatively more effective at reducing high loadings (which correlate with high flow conditions) than reducing low loadings (which correlate with low flow conditions) such that L and R(t) will be negatively correlated. The midrange value of plnL,lnR = –0.5 was considered reasonable. Apart from the total lack of basis for any particular numerical value, as explicitly admitted, the whole concept of this correlation is erroneous. L is

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin the loading, with a distribution arising from its variability in time, particularly its variability during the year. R(t) is explicitly defined to be a time average over a year (PTM, p. B-4); there can be no correlation on this basis alone.15 More to the point, the distribution associated with R(t) is an uncertainty distribution, with nothing whatever to do with variability in time, so the concept of correlation does not even apply. What has been done in the PTM is to cancel out (by applying a negative correlation) some of the uncertainty in R(t) with the time variability of L! The “correlation” that is described in the cited paragraph is more accurately a claim that there is a functional relationship between the parameters of the distribution representing the time variability of L and the actual value of R(t)—specifically, that the upper end of the distribution of L is modified by the value of R(t). A potential way of modeling such an effect would be to treat the standard deviation of the distribution of L as a function of R(t). In this case, however, there is no basis provided that the claim is accurate and that “remedial action will generally be relatively more effective at reducing high loadings (which correlate with high flow conditions) than reducing low loadings (which correlate with low flow conditions).” Whether this claim is true depends on details about leaching and erosion from each source, details that are not documented or (apparently) even examined in the PTM in reaching its conclusion. An Attempt to Estimate the Wrong Correlation The load ratio is defined by Equation 1.8 above (PTM, p. B-53, equation 1); that is, (1.14) and the metal loading F(t) is given by Equation 1.2 above (PTM, p. B-4, equation 1); that is, (1.15) so that (1.16) Both L, the preremedial loading, and CL, the loading capacity, vary with time throughout the year, whereas R(t) is defined to be a yearly average. 15   I discount as too unlikely the possibility that the PTM was implying a correlation between the uncertainty distributions for the parameters of the (current) loading and the (future) remedial effectiveness.

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin Associated with R(t) is an uncertainty distribution but no unpredictable time variability (R(t) varies with time, but smoothly and in a predictable fashion), whereas the distributions associated with both L and CL are due to their (unpredictable) time variation (strictly, there are also uncertainty distributions associated with the parameters of the distributions describing their time variation, because of finite numbers of measurements, but these are ignored here, just as they are ignored in the PTM). There is a very high correlation between measured values for L and CL (the correlation coefficient between their logarithms is approximately 0.95)16 but none between R(t) and L or CL (as discussed for L; the same arguments apply to CL as to L). The PTM (p. B-53, equations 1, 2, and 3), however, obtains the uncertainty distribution for F(t) at a random time within about a year of t by combining the time-variability distribution for L with the uncertainty distribution for R(t). It then attempts to argue about the correlation between the resulting uncertainty distribution and the time-variability distribution for CL based on the correlation between L and CL. It states (pp. B-53 to B-54): The future correlation between lnF(t) and lnCL, measured by plnF,lnCL, is expected to be very high. This expectation is based on an almost perfect correlation (p = 1.0) between lnCL and lnQ and a virtually certain high future correlation between lnF(t) and lnQ, just as there has been historically between discharge and loading (which, being a function of discharge, induces correlation). In addition, as further discussed in Section B.3.4.1, and independent statistical analysis of the zinc concentrations, water hardnesses, and discharge data corresponding to that used in developing the TMDL loading capacities for SF271 (EPA 2000) showed a correlation coefficient of 0.95 between the natural logs of zinc loadings (computed as the product of concentration and discharge) and the equivalent loading capacities (computed as the product of the zinc AWQC(H) and discharge). Consistent with this information, a value of plnF,lnCL = 0.9 was used in the analysis. There is no basis for the selection of the particular value 0.9. It is not possible to state whether it is “consistent with this information” without further examination, but in general it is not consistent with that information. The effect of assuming a high correlation between ln(F(t)) and ln(CL) is to substantially cancel the uncertainty in R(t) with the time variability in CL; but this cancellation is purely fictitious. This error compounds the 16   CL is measured by measuring the hardness of the water and the flow rate simultaneously, computing the AWQC from the hardness, and forming the product of AWQC and flow rate. L is measured by measuring the metal concentration and flow rate and forming the product.

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin previous erroneous cancellation of the uncertainty of R(t) by the time variability of L discussed above. The effect of these two incorrect cancellations can be large. This may be illustrated by supposing that what is required is the uncertainty distribution for Lr(t) at a random time, so that it is legitimate to (correctly) combine the uncertainty of R(t) with the time variability of the ratio L/CL in Equation 1.16. For dissolved zinc at location SF271 on the Coeur d’Alene, the measured standard deviation of (the time variability of) ln(L) is 0.525, that of ln(CL) is 0.643, and that of the logarithm of their ratio, ln(L/CL), is 0.22517 (obtained from the joint measurements of concentration, hardness, and flow rate; [EPA 2000, for hardness and flow measurements; URS Greiner Inc. and CH2M Hill Inc. 2001b, for dissolved zinc and flow measurements]).18 With these measured standard deviations for ln(L) and ln(CL), Table F-1 shows the correct calculation of the random-time uncertainty for ln(Lr(t)) compared with that obtained by including the two erroneous correlations introduced in the PTM for various values of the standard deviation of ln(R(t)). The error introduced is clearly substantial for any plausible estimates for uncertainty in ln(R(t)). INCORRECT OR MISLEADING STATEMENTS IN THE PTM The following is an incomplete sampling of various incorrect or misleading statements and equations in the PTM. Attempting to list all such erroneous statements and equations would be too time-consuming, so the failure to list any statement or equation in this list cannot be considered an endorsement of the correctness of any statement or equation not listed here. • The term “power series” is used incorrectly throughout Appendix A. Where “power series” is used, the correct term would be something like “power product.” The discussion is not of power series in one or more random variable, but the product of powers of random variables. • “Minimum statistical assumptions are required” (PTM, p. A-13). No basis is provided for this statement. One can assume anything, but that does not make it correct, or even consistent, or useful. 17   The distributions for ln(L) and ln(CL) are not distinguishable from normal (p = 0.42, 0.44, respectively, Shapiro-Wilk test). The distribution for L/CL is closer to normal than lognormal (p = 0.39, 0.09, respectively; Shapiro-Wilk test) using the available data. 18   Measurements taken on the same day were assumed to be simultaneous, and multiple measurements on the same day were averaged for the analysis. Only the subset of data with simultaneous hardness, flow rate, and concentration data are included in the statistics given here.

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin TABLE F-1 Effect of the Two Erroneous Correlation Calculations Introduced in the PTM Standard deviation of ln(R(t)) Standard Deviation of ln(Lr(t)) Correct PTM 0.0 0.225 0.285 0.3 0.375 0.306 0.6 0.641 0.280 0.9 0.928 0.347 1.2 1.221 0.541 1.5 1.517 0.791 • “A lognormal PDF is believed to be a maximum entropy PDF for the log of variables where only the expected value and coefficient of variation of the distribution is known or estimated. Maximum entropy estimates give the ‘least prejudiced, or least biased, assignment of probabilities’” (Harr 1987) (PTM, p. A-13, footnote 9, italics in original). No connection is proposed between “minimum statistical assumptions” and “maximum entropy.” Nor is any application to the problem at hand proposed; on what basis, for example, is it supposed that only the expected value and coefficient of variation are known for the log of variables, and how does this connect, for example, with the evaluation of probability to exceed the AWQC? • “a correlation coefficient of –1.0 implies perfect inverse linear correlation (i.e., X1 and X2 are inversely proportional)” (PTM, p. A-8). This is incorrect; perhaps what was intended is that if the correlation coefficient between logarithms ln(X1) and ln(X2) is –1.0 then X1 and X2 are inversely related (but not necessarily in direct inverse proportions). • “Unbounded positive values are allowed (which is generally conservative because it tends to overestimate true values)” (PTM, p. A-13). It does not follow that lognormal distributions lead to “generally” conservative estimates, without specifying the universe of discourse. For example, if some variable is (erroneously) assigned a lognormal distribution, and that variable occurs in the denominator of an expression, the result may be an underestimate rather than an overestimate. On the other hand, the inverse of a lognormal distribution is also lognormal, so the preceding example also shows that (erroneously) assigning a lognormal distribution to an expression in the numerator can lead to underestimates—because a lognormal distribution also allows unboundedly small values. • “Any PDF can be conservatively approximated using a lognormal PDF that envelopes the PDF over the range of interest” (PTM, p. A-13). Again, this statement is meaningless without a definition of “conservatively,” “envelopes,” and “range of interest” at the least. Even with such

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin definitions, it is likely to be untrue in general. Indeed, it is quite likely that a converse theorem holds—for any lognormal approximation to a given PDF, there exist statistics of that PDF that are not conservatively estimated by the lognormal approximation. • “Variables CDS and CS are, respectively, the metal (zinc) concentration of the deeper sediments and floodplain sediments having RLP = 1. These sediment concentrations will be positively correlated” (PTM, p. B-47). “Also, because of the way CDS and CS were estimated, they would be positively correlated” (PTM, p. C-9). It is quite plausible that the concentrations of deeper sediments and floodplain sediments are correlated spatially—that is, the concentration would tend to be higher in the deeper sediments beneath floodplain sediments with higher concentrations. Such a spatial correlation is entirely irrelevant, however, for variables CDS and CS, which are defined to be “volumetric average concentrations in the deeper impacted sediments” and “volumetric average concentration in the impacted sediments having an RLP = 1” (PTM, p. B-14). Any spatial correlation is entirely removed by the averaging. What is required is any correlation between the uncertainty distributions for these volumetric averages. No such correlation is induced “because of the way CDS and CS were estimated.” The only documented “estimation methods” are given in section C.2.4, where uncertainty confidence intervals for the values of CDS and CS are supposedly (very loosely) based on observed data in the BHSS and a background estimate based on measurements outside the BHSS. Nothing in the measurements supposedly used or in the described derivation correlates these uncertainty distributions; the fact that the same value is used as the lower uncertainty confidence bound for one and the upper uncertainty confidence bound for the other is the only connection between them, and that has no such effect. The subsequent estimate of a value of 0.5 for the correlation coefficient of this hypothetical, nonexistent correlation is simply incorrect. • “An estimate of CV[M] = 0.5 was used in the analysis” (PTM, p. B-48). There is no basis given for this estimate. Nor is it clear why it was introduced, except to arbitrarily increase the uncertainty estimate. • “Since the estimates for L and TEM′ are independent of each other, plnL,lnTEM, was set to zero in the analysis. This lack of correlation in the estimates should not be confused with the positive correlation that must exist in the true values of L and TEM′, and is otherwise inherent in the data used to make the estimates. To the extent there was (positive) correlation between the estimates of L and TEM′, it would decrease both E[β] and CV[β]” (PTM, p. B-48). This statement demonstrates complete confusion, apparently stemming from a misunderstanding of what “correlation” means or perhaps the confusion in this document between measurement uncertainties, variability in time, and functional relationships. There is obviously

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin no correlation possible between true values of L and TEM′,19 which are single values. • “The BHSS data do not represent the true values of CDS and CS, which are uncertain” (PTM, p. C-10). True values cannot be uncertain, although they may be unknown, so that we are uncertain about what they are. • “For example, the correlation coefficient between the natural logs of Q and H is 0.96 for the SFCDR at SF271 … the correlation coefficient between the natural logs of AWQC and Q at SF271 is also 0.96 for the TMDL data set” (PTM, p. B-26). Both these correlation coefficients are –0.96, not +0.96. • Page 1-1, footnote 1, the conversion factor is actually 0.005394 to 4 significant figures, or 0.00539 to 3 significant figures. The value used should at least be the correct rounding of the exact value. • Page A-24, equation [2] is incorrect. The correct expression is (1.17) and there is no need to introduce the variables Xi′. Indeed, the entire exposition would be greatly clarified by working with statistics of the logarithms of the variables. For example, define Ti = ln(Xi), T = ln(X), and let Xi have mean mi and coefficient of variation ci, Ti have mean μi and standard deviation σi, and similarly for X and T (with no subscripts). Then we have (1.18) and similarly for all subscripted variables. Then equations 1 through 3 of PTM (p. A-24) become the considerably simpler equations: (1.19) where ρij is the correlation coefficient between Ti and Tj, and it is trivial to move between statistics for variables and their logarithms using Equations 1.18. • Page B-10, the second equation for Lj/Vj in the middle of the page is incorrect; that is, 19   The term L here corresponds to our W.

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin (1.20) and the inequality applies except in certain special cases (which do not apply in general in this application). • Page B-12 and C-5, the last two entries in equation 4 of p. B-12, and the same equations repeated in section C.2.3 for RLPj are incorrect if any attempt is made to interpret them according to standard conventions. The first and second entries of equation 4 of p. B-12 correspond to the definitions given. It is just possible to interpret the last two entries in equation 4 of p. B-12 and the same equations in section C.2.3 in the correct sense if the phrases “per unit volume of source type j and FP” and “per unit load of source type j and FP” are interpreted as applying separately to the numerators and denominators of the respective equations, contrary to any standard convention; coming on these equations by themselves (without the correct definition) in section C.2.3 is disconcerting. • Page B-26, “The analysis used the same discharge and H(Q) and AWQC(Q) relationship used in EPA 2000 for the TMDL.” This statement is incorrect, and the approach taken in the PTM is inconsistent with the intent of performing an uncertainty analysis. First, the statement is incorrect because the relationship assumed in EPA (2000) was linear between hardness itself and the logarithm of flow rate, whereas the relationship assumed in the PTM is linear between the logarithm of hardness and the logarithm of flow rate. Second, the approach taken in the PTM is inconsistent, because the PTM analyzed the loading capacities derived for regulatory purposes in EPA (2000). However, those loading capacities already have built into them the results of an uncertainty analysis; the loading capacities are derived as 90th percentiles of an uncertainty distribution.20 That uncertainty analysis should be incorporated in the PTM as part of the overall uncertainty analysis—the PTM should evaluate the original data, not the summary statistics produced by EPA (2000). • Pages A-18 to A-19 and B-33 to B-34, the technique used to estimate parameters (mean and standard deviations of the logarithm) of lognormal distributions by regressing order statistics of the logarithms of measurements against the “plotting points” (p. A-18, equation 8, and p. B-34, equation 1 has nothing to recommend it. The “plotting points” used are 20   EPA (2000) states (p. 22) that the values were lower bounds of a 90th percentile confidence interval, thus at the 95th percentile. This is incorrect, however. The values obtained by EPA (2000) are the 90th percentiles (lower bounds of an 80th percentile confidence interval).

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin only approximations of the expected values of the normal order statistics, so the technique is approximate at best (better approximations of normal order statistics are available (Royston 1993, 1995). The values obtained for mean and standard deviation are almost certainly biased and have unknown statistical properties. On the other hand, simply computing the mean and (sample) standard deviation of the logarithms of measured values gives unbiased estimates with known (and optimal for certain purposes) statistical properties for these parameters. Unless the PTM justifies the methodology used (by demonstrating, for example, superiority in some sense of the estimates obtained), standard (and simpler) approaches should be used. • Page B-35, equation 6, the right-hand side erroneously uses CV[L] where what is required is CV[C]. The expression for Ω erroneously omits p. • Page B-35, equation 8, the right-hand side erroneously uses CV[L] where what is required is CV[C]. • Page B-35, footnote 17, the expressions could be somewhat simplified if the trivial identity (1.21) were applied. Better yet would be adoption of the suggestion discussed in the comment on p. A-24. • Page B-36, first equation on page (carried over from equation 10 of p. B-35), the expression for Ω erroneously omits p. • Page B-37, equation 3, the expression for Ω erroneously omits p. • Page B-43, equations 6 and 7, in both these equations the denominators have been written incorrectly, because (1.22) The left side of Equation 1.22 is what is required inside the square root in the denominator of equations 6 and 7, but the right side is what is written. • Page B-57, “The analysis showed the following principal results: Both the AWQC(H) and the equivalent loading capacities were lognormally distributed with respective r2’s of 0.94 and 0.97.” The list continues with similar statements about ratios of zinc loadings to loading capacities, zinc concentrations, loadings, and hardness. However, the given information is not sufficient to support the conclusion of lognormality for these quantities—some values of r2 would be obtained whether or not any par-

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin ticular distribution was lognormal. It is quite feasible to test whether a set of samples is consistent with lognormality—for example, by using the Shapiro-Wilk test (Royston 1982, 1993, 1995). Applying this test suggests that it is somewhat unlikely that the measured zinc concentrations (p = 0.002), AWQC(H) (p = 0.014), or hardness (p = 0.018) are lognormal, although zinc loading measurements (p = 0.3) and loading capacity (p = 0.5) are consistent with lognormality.21 It is already pointed out in footnote 17 that the ratio of zinc load to the load capacity is more consistent with normality than lognormality. • Page B-52 (section B.3.3.3), “For these reasons and because of its general theoretical and practical basis, Eq 1 was considered a valid and reasonable approximation for estimating CV[R(t)] for the lower basin, with further savings of effort.” But there is no theoretical basis whatever for equation 1, because it is purely an empirical approximation found for the upper basin using the specific values for the upper basin.22 Therefore, there is no basis whatever for extending this empirical approximation to the lower basin (with different source types, different mixes of sources, and so forth)—the results obtained there could be substantially different. REFERENCES EPA (U.S. Environmental Protection Agency). 2000. Total Maximum Daily Load for Dissolved Cadmium, Dissolved Lead, and Dissolved Zinc in Surface Waters of the Coeur d’Alene River Basin, Appendix I: Hardness Data. EPA 910R01006. U.S. Environmental Protection Agency, Region 10, Seattle, WA and Idaho department of Environmental Quality, Boise ID [online]. Available: http://yosemite.epa.gov/.../ac5dc0447a281f4e882569ed0073521f/b57be085215b66658825693b0076f71c/$FILE/CdAtecha.pdf [accessed July 27, 2005]. Kaplan, S. 1992. “Expert information” versus “expert opinions.” Another approach to the problem of eliciting/combining/using expert knowledge in PRA. Reliab. Eng. Syst. Safe. 35(1):61-72. Ridolfi (Ridolfi Engineers Inc.). 2000. Containment Cover Evaluation Using the HELP Model, Coeur d’Alene River Basin Feasibility Study. December 2000 (as cited in URS Greiner, Inc., and CH2M Hill 2001a). Royston, J.P. 1982. Algorithm AS 181: The W test for normality. Appl. Statist. 31(2):176-180. Royston, P. 1993. A toolkit for testing for non-normality in complete and censored samples. Statistician 42(1):37-43. Royston, P. 1995. Remark AS R94. A remark on algorithm AS 181: The W-test for normality. Appl. Statist. 44(4):547-551. 21   These statistics are obtained using all the available data, in the same manner as apparently used in the PTM. Footnote 17 gives similar statistics for the subset of these data for which all of hardness, zinc concentration, and flow were simultaneously measured. 22   It is likely to be substantially incorrect, as indicated by the other comments made here, but that is irrelevant to the current argument.

OCR for page 459
Superfund and Mining Megasites: Lessons from the Coeur d’alene River Basin URS Greiner, Inc., and CH2M Hill. 2001a. Probabilistic Analysis of Post-Remediation Metal Loading Technical Memorandum (Revision 1). URSG DCN 4162500.06778.05.a. Prepared for U.S. Environmental Protection Agency, Region 10, Seattle, WA, by URS Greiner, Inc., Seattle, WA, and CH2M Hill, Bellevue, WA. September 20, 2001. URS Greiner, Inc., and CH2M Hill, Inc. 2001b. Final (Revision 2) Remedial Investigation Report for the Coeur d’Alene Basin Remedial Investigation/Feasibility Study, Volume 9, Appendix C. URSG DCN 4162500.6659.05a. Prepared for U.S. Environmental Protection Agency, Region 10, Seattle, WA, by URS Greiner, Inc., Seattle, WA, and CH2M Hill, Bellevue, WA. September 2001.