Escherichia coli O157:H7 in Ground Beef Review of a Draft Risk Assessment (2002) / Chapter Skim
Currently Skimming:
7 Modeling Approach and Implementation
7 Modeling Approach and Implementation
Pages 97-134
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 97... ...
effort faced a number of substantial methodologic hurdles whose solutions have not been described in textbooks or in literature peculiar to microbial risk assessment. In addition to methodologic hurdles, the FSIS team has been forced to cope with the inadequacy of the knowledge base.
|
From page 98... ...
The dose-response relationships are derived by fitting predicted distributions of exposure to estimates of the population health risk attributable to ground beef as estimated from epidemiologic data. In its final step, however, the model departs from the standard approach to risk assessment in a way that merits careful attention.
|
From page 99... ...
Assessment of the Rationale for the Inverted Assessment Approach The nonstandard treatment of dose-response assessment appears to be based on the judgment that this component of the risk estimation carries the most and least likely to be resolved uncertainty. Furthermore, the uncertainty in the dose-response relationship is judged to be less than the uncertainty in the population risk estimate that typically would be considered the goal of the risk-assessment effort.
|
From page 100... ...
It may be possible to place more faith in the use of Shigella dysenteriae 1 as a surrogate for the best estimate of a dose-response function for EHEC, as is proposed in the Hazard Characterization chapter, than in the implemented exposure model as a surrogate for the reality of the ecology and transmission of EHEC within and between farms, feedlots, slaughterhouses, combo bins, and grinders and through the multitude of potential cooking practices and consumer behaviors. An Example of the Standard Approach After the FSIS draft report was released for public comment, a risk assessment by Nauta and colleagues (2001)
|
From page 101... ...
Those are combined to form a prediction of the population health risk attributable to consumption of steak tartare contaminated with EHEC. The results of the analysis can be compared on a truly independent basis with population risk estimates from the Netherlands.
|
From page 102... ...
Any change in the parameters of the exposure assessment (for example, an improved estimate of the prevalence in a population of cattle) or in the assumptions leading to the baseline population health risk estimate (for example, a change in the etiologic fraction estimates)
|
From page 103... ...
A serious miscommunication could result if readers form the impression that the model propagates evidence forward to generate population health risks and then judge the model to be appropriate on the basis of the quality of the match to what are thought to be independent epidemiologic data. Having departed from the standard approach, the authors have the burden of ensuring that readers do not construct an inappropriate mental model of the approach and thereby form a judgment of its validity.
|
From page 104... ...
Infer the distribution of the proportion of positive samples taken from grinders that would be statistically consistent with the FSIS sampling evidence.
|
From page 105... ...
· The fabrication process is modeled as contributing to grinder loads only in situations in which other uncertain factors may be underestimating the pathogen load in grinders, as opposed to having an independent contribution in each simulation, as might be expected; in this way, the impact of fabrication depends on unrelated factors and not on any explicit assumptions regarding the process of fabrication. · The shift of all grinder-load profiles that fall below the 5th percentile toward a distribution leading to the mean proportion is an arbitrary distortion of the grinder concentration distribution.
|
From page 106... ...
from an uncertainty distribution of the population risk estimate of the annual number of illnesses, on the basis of epidemiologic analysis. The assumed dose-response curve is the beta-Poisson function with parameters or and ID50.
|
From page 107... ...
Alternative Model-Updating Strategies Both the model-updating processes described above appear to lack a formal statistical basis. Given the use of Bayesian updating processes at various points in the model and the overall reliance on Monte Carlo simulation, it seems appropriate to consider using a form of Bayesian Monte Carlo simulation (or some of its more advanced resampling relatives)
|
From page 108... ...
For the grinder-level observational data and any other observational data in the system being simulated, the authors may wish to consider Bayesian Monte Carlo methods to provide a structured method of updating model parameters in light of observational data. MODEL VALIDATION At several points, the FSIS draft report argues that its findings are "comparable" with other estimates or descriptions of outbreaks.
|
From page 109... ...
Assumptions used in a hazard characterization can have a great impact on the ability of a model to represent the expected value of mitigations accurately. Scope and Context Decisions in Hazard Characterization A number of important subassessments are required in hazard identification and hazard characterization: · To describe the evidence for probability of illness as a function of any risk factors (that is, dose, age, disease states or other conditions of the host, sex, and food-matrix effects)
|
From page 110... ...
. The draft risk assessment does not use any risk factors in calculating the likelihood of transition from illness to more-severe outcomes, such as hemolytic uremic syndrome (HUS)
|
From page 111... ...
· An explicit accounting of the various indicators of attributable risk (outbreak data, case-control studies of sporadic cases, passive surveillance, and the like) and their expected inferential value as related to a particular food and pathogen combination.
|
From page 112... ...
of illness, and the attribution is constant among different age groups. A2: Product X accounts for 40% of EHEC illnesses in children but only 10% in the remainder of the population.
|
From page 113... ...
The authors should also review the scope of the model and its documentation to ensure that the full public-health context and thereby the value of potential mitigations can be described and measured by the risk assessment. Attribution of EHEC to Ground-Beef Consumption The FSIS draft risk assessment relies on matching the cases predicted by the broad spectrum of ground-beef production and consumption behaviors (although ignoring, at this point in development, the potential for cross contamination)
|
From page 114... ...
It seems reasonable that the risk attributable to exposures to beef known to be "pink in the middle" should constitute a minimum for the attribution of total risk to ground-beef consumption. Such exposure seems to account for only a subset of the exposures to contaminated ground beef even if the estimation is limited to direct consumption of ground meat as opposed to consumption involving cross contamination.
|
From page 115... ...
SOFTWARE IMPLEMENTATION The FSIS draft risk model consists of the evidence base captured in the documentation and a simulation model implemented with the spreadsheet environment Microsoft Excel (referred to hereafter as Excel)
|
From page 116... ...
. · A version that uses probabilistic sampling functions provided by FSIS, with both the sampling functions and the overall simulation model implemented with VBA (no longer requiring ~RISK)
|
From page 117... ...
That can be managed with VBA code to store and perform intermediate calculations, careful documentation of the spreadsheet, and detailed user manuals. However, beyond some level of complexity, the spreadsheet environment becomes more a problem than a solution for the purposes of model communication.
|
From page 118... ...
The task is made more difficult in the draft model by the use of direct cell references as opposed to the use of named identifiers to refer to another quantity in the model; for example, a formula for microbial growth uses the spreadsheet cell location Temperatures! $AB$82, which is located on another worksheet, instead of the label "CookingTemperature" to refer to the cooking temperature.
|
From page 119... ...
Note that those types of risks and benefits generically apply to all such choices and are not limited to BRISK or the particular custom simulation implementation developed by USDA. It is important to clarify that the decision faced by USDA should not be seen as a choice among the two modeling environments.
|
From page 120... ...
Given the importance of the United States as a trading partner and the potential (but as yet undemonstrated) importance of microbial risk assessments in the international food trade, their choices of modeling environments and software are influential.
|
From page 121... ...
The difficulties in developing and justifying input distributions are well known in the field of risk analysis and have received much attention (Finley et al., 1994; Haimes et al., 1994~. Although there is a considerable literature on the subject of estimating probability distributions from empirical data (sullen and Frey, 1999; Morgan and Henrion, 1990)
|
From page 122... ...
Many analysts consider empirical distributions as the best possible representations of variability because they let the available data "speak for themselves." If all variables were treated this way, the Monte Carlo simulation would amount to a permutation study of the raw data. Risk analysts often prefer that approach because it does not require them to make assumptions about the distribution shapes or to fit distributions to data; the data are the distributions.
|
From page 123... ...
Maximum Entropy Some distribution selections in the FSIS draft risk assessment seem to have been based on appeals to the maximum entropy criterion, which states that when one has only partial information about possible outcomes, one should exploit the available information to the extent practicable and impose as few assumptions as possible on the missing information (Grandy and Schick, 1991; laynes, 1957; Lee and Wright, 1994; Levine and Tribus, 1976; Tilwari and Hobble, 1976~. The use of maximum entropy in selecting input distributions for Monte Carlo analysis is superior to naive conjecture and is considered by many to be the state of the art.
|
From page 124... ...
In the final analysis, the committee believes that there are circumstances in which it would be appropriate to solicit expert opinion regarding point estimates and distributions and that, if found useful, such information should be documented in the text and used in the model until data become available. Dependencies It appears that most of the variables in the FSIS draft risk assessment are assumed to be mutually independent.
|
From page 125... ...
An explicit model of such dependence may not be feasible, but the effect of plausible dependence scenarios should be considered on a case-by-case basis and presumably prioritized through causal reasoning of the plausibility of the dependence relationships. This will assist in better characterizing the potential for high-risk scenarios and may help to explain higher proportions of attributable risk to particular exposure pathways.
|
From page 126... ...
There are several ways this happens: incomplete reconstruction of statistical regressions, overreliance on empirical distributions, use of means other than raw data, and modeling of sampling variation without representing the underlying uncertainty arising from measurement error. Each of those is discussed below.
|
From page 127... ...
88 in the draft risk assessment depicts the modeled frequency distribution of log reductions in E cold abundance caused from cooking.
|
From page 128... ...
177 of Appendix C of the FSIS draft, exemplifies another way in which the uncertainty present in the system is underestimated in the draft model. That equation defines the random variable TR (transformation ratio)
|
From page 129... ...
Potentially Dominant Model Uncertainties Model uncertainty is a class of uncertainty that pertains to the adequacy of a model's representation of reality. Strictly on the basis of qualitative judgment, the committee suggests that the following general model uncertainties may dominate: · The ability to define an appropriate output or set of outputs from the onfarm module that is adequately correlated with the level of risk to the ground-beef supply.
|
From page 130... ...
Appendix D, which contains comments on the model presented to the committee by outside reviewer Edmund Crouch, cites specific examples where references were found to undefined cells in the draft model spreadsheet. Some of these cells had been given a "hatched" format, presumably to indicate that the values were not available.
|
From page 131... ...
The FSIS draft report's Appendix C a partial list of the model equations and code contains some but not all of the information needed to check unit conformance. The committee was thus unable to conduct a rigorous review of the dimensions and units of the equations and variables used in the draft model.
|
From page 132... ...
1992. Development of a Bayesian Monte Carlo method for determining water quality model uncertainty.
|
From page 133... ...
II. Initial condition and parameter specification in terms of maximum entropy distributions.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.