Click for next page ( 35


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 34
4 Statistical Considerations Abstract The Cumulative Sum (CUSUM), a widely accepted method for maintaining process control in manufacturing, is a feasible approach to statistical evaluation of the slaughter process. It can help ensure that procedural or policy changes, personnel turnovers, inadvertent mistakes, poor training, or mechanical deficiencies do not establish an upward trend in defects that are detectable by inspection. CUSUM is a good choice to help control the process and ensure uniformity of processing. However, many people erroneously believe CUSUM as used in SIS-C is intended to ensure that no contaminated or otherwise defective meat reaches the consumer. That is not its purpose, and it is inadequate to provide that assurance in cattle slaughtering operations. Therefore, CUSUM in SIS-C will not directly ensure a pleasing, clean, wholesome, and toxin- or pathogen-free product. As one part of the processing and inspection effort, it monitors system performance by checking that the process is consistent in its ability to detect and remove visible defects and that processing modifications do not establish an upward trend in these defects. The limitations of CUSUM as an inspection tool for assessing a nonuniform product must be clearly recognized. Because of these limitations, greater flexibility must be built into the sampling scheme along with an emphasis on optimizing the quality of the final product rather than simply obtaining acceptable CUSUM scores. CUSUM is appropriate for maintaining process control provided appropriate tolerance limits are set and provided all lots are adequately sampled. However, CUSUM cannot eliminate the need for adequate inspection and for microbiologic and toxicologic testing. The primary statistical considerations related to SIS-C involve three points: (~) the rationale supporting a change from Acceptable Quality Level (AQL) scores to CUSUM for STS-C; (2) the use of CUSUM for process control; and (3) the method of sampling for CUSUM determinations. Rationale Supporting the Change from AQI, to SIS The rationale for favoring SIS-C over traditional inspection came from AQL, scores (see definitions in Appendix G), based on both traditional and STS-C procedures. These data showed that one of the five plants sampled (plant #5) had lower AQL scores when STS-C was in effect (FSIS, 1989; Wesson, 1983~. These data are consistent with the hypothesis that STS-C might produce lower AQL scores in that specific plant (plant #5), but are an inadequate basis for accepting this hypothesis or inferring that SIS-C produces lower AQL scores in other plants. Furthermore, these conclusions should not be drawn from data on this one plant, because there was a downward trend in AQL scores in that plant when traditional inspection was still in place. Other 34

OCR for page 34
variables, such as management changes, that are unrelated to inspection procedures could have affected the results. After the initial improvements, little consistent further improvement was observed. Use of CUSUM for Process Control MUM IS a widely accepted method tor maintaining process control In manufacturing products that must be kept within certain tolerances. At appropriate stages of production, product samples are examined for deviations from specified limits in composition, dimensions, or other features. CUSUM is a gooc] method for monitoring manufacturing processes to detect changes that cause failure of products to meet required specifications. CUSUM is an acceptable method of monitoring the process to discover problems introduced by factors such as personnel turnover, inadequately trained personnel, mechanical deficiencies, plant policy, and changes in procedures. This is the purpose for which FSIS uses CUSUM. However, many people believe erroneously that CUSUM as used in SIS-C is intended to ensure that no contaminated or otherwise defective meat reaches the consumer. CUSUM sampling and calculations are not designed to detect defects that are missed by inspectors or trimmers. ~1 In SIS-C, CUSUM is used to monitor slaughtering of cattle; this is similar to process control in manufacturing facilities. However, unlike manufacturing processes that merge raw materials into some product, in SIS-C the incoming material is modified by separating or removing certain parts and by detecting and removing defects. In STS- C, a nonuniform product, i.e., cattle with various defects and from different sources, enters the processing plant. In addition, certain defects can be introduced by the processing. CUSUM attempts to monitor the system's performance in detecting and removing both types of defects and producing a uniform (with respect to defects) final product. According to "FSTS Data Presentation to the Committee," CUSUM involves use of a tolerance level "set at the national average for the total of the weighted non- conformances for all of the plants tested" (FSIS, 1989b). Data from a randomly selected sample of plants were used to determine the average quality (cleanliness and processing defects) of products produced in the United States. This average is applied to SIS-C plants whose initial performance should have been much better than the national average because of the uniform age, size, and weight of the cattle they slaughter. In CUSUM, as in SIS-C, the tolerance limits might more appropriately be based on what is acceptable for every plant rather than on an average reflecting what plants were doing at any particular time. It would be surprising if these measures were identical. The issue is not whether CUSUM is appropriate; it can be a valuable part of SIS-C. Rather, the important factors are (~) the tolerance limits within which plants can operate and be labeled "in control" or "out of control", (2) the appropriate actions that require more conservative processing practices such as slower line speeds and reprocessing, and (3) the way in which CUSUM is utilized and interpreted within the 35

OCR for page 34
entire framework of SIS-C. Cattle arrive at the processing plant with various defects. Defects may be clustered in herds or lots. They may be related to traumas during transport, to diseases, or to nutritional influences at the feediot. In addition, defects are also introduced by processing. Therefore, defects measured in STS-C can come from two sources: the cattle entering the plant and the process itself. If cattle entering the plant had standard defects, then monitoring the defects would monitor the process. A goal of SIS-C monitoring is to remove defects from both sources without distinguishing between them. One question to be answered is whether the effectiveness of CUSUM monitoring is influenced by the condition of the lot entering the plant. Is process control affected by the number of defects, the number of cattle with defects, the clustering of defects, and the kinds and locations of defects? Is it possible for some lots with high numbers of defects to enter the plant and not be included in the sampling for CUSUM? If the processing of cattle is the same for all lots, the nature and location of defects do not influence the ability of inspectors to discover them, and if all lots are adequately represented in CUSUM, then the clustering of defects within lots might be less important. However, if defects of particular kinds or in certain locations are more difficlit to observe, then clustering of defects in lots is important, and the monitoring method needs to ensure that there is adequate sampling within lots (possibly representing all herds). The feasibility of representing all sources in the sampling is at best questionable. On the other hand, sampling from all lots may be feasible and could be incorporated into the plan for CUSUM in the same way that sampling during certain time intervals is planned. Finally, CUSUM in SIS-C plays an indirect, not a direct, role in ensuring pleasing, clean, wholesome, and toxin- or pathogen-free products as part of the entire processing and inspection effort. It is a check on the process to ensure that processing modifications, inadvertent mistakes, and poor training do not establish an upward trend in the number of defects undetected by the inspection. It does not guarantee that the product is free from defects, but should ensure that the process is consistent in its ability to detect and remove defects. The Sampling Basis for CUSUM If there is substantial variation in the average number of defects among lots, then the sampling for CUSUM needs to include an adequate number of subgroups from each lot. The plan for CUSUM monitoring of carcass and viscera includes the plant's sampling subgroups of three consecutive carcass sides with a frequency of one subgroup sample per hour and FSIS sampling subgroups of three consecutive sides every ~ hours; other products are sampled with different criteria. If carcasses from a lot composed of several herds are distributed randomly along the production line, the probabilities of detecting defects in a particular lot should be determined to understand 36

OCR for page 34
how well sampling will represent the lot. The probability that defects in a given lot will be detected can be studied by examining the converse probability that the sampled units will not include any defects in a lot, when in fact they exist. Figure 4-1 depicts these probabilities based on the following assumptions: I. The distribution of defects in lots is Poisson. 2. The average number of defects per carcass side is 0.~. 3. Three carcass sides are observed each hour. 4. Carcass sides are independent. (This is, of course, an oversimplification.) The probabilities have been calculated and plotted for line speeds from 200 to 400 per hour in increments of 50, and for lot sizes of 250 to 3,000 in increments of 250. The probabilities of observing no defects among sampled units from single lots having an average of 0.1 defects per carcass are very high for the smaller lot sizes and decrease as the lot sizes increase. This is to be expected, since more carcass sides would be sampled from the larger lots for any given line speed. The probabilities are consistently higher for faster line speeds, because a smaller proportion of carcass sides is sampled for faster line speeds, regardless of the lot size. By looking at the points where the lines intersect with a particular probability on the graph, it can be seen that the lot size must increase for each increment in line speed to achieve that probability. 1 0.8 m 0.6 6 m 0 0.4 0.2 i_ O 1 250 500 750 1000 1250 1500 1750 2000 2250 2500 2750 3000 LOT SIZE Lumber of Cable per Hour 200 ~ 250 -a--- 300 - - 350 _ 400 Figure 4-! Probability of Observing No Defects in Various Lot Sizes and Line Speeds When the Average Number of Defects/Sample Unit (Carcass Side) is 0.l 3 ,7

OCR for page 34
For example, the following line speeds and lot sizes were all associated with the same probability (0.223) of observing no defects in the lot (line speed/lot size): 200/1,000, 250/1,250, 300/1,500, 350/1,750, and 400/2,000. Higher line speeds can be used with larger lots to achieve probabilities comparable with those of lower line speeds and smaller lots. However, these probabilities relate only to the discovery of at least one defect in a lot (~.0 minus the probability of not detecting a defect) and do not address adequate numbers to assess the level of defects in a lot. Of course, when faster line speeds are used with small lots, there is a chance that some small lots will not be represented in the sample. In summary, a simple Poisson mode] was used to compute probabilities that some defective product will be accepted using CUSUM. These computations showed that defects among cattle in small lots are more likely to be missed than those in large lots and that faster line speeds are associated with higher probabilities of accepting defective meat. This demonstrated that a CUSUM-based inspection scheme will allow- -with disturbingly high probability--some defective meat to pass undetected in small lots and in plants with high line speeds, and reinforced the position that its use should be limited to process control. A second approach to studying the potential for defective cattle to pass the CUSUM monitoring process is presented in Figure 4-2. This figure presents the operation characteristic curve for CUSUM. When the fraction nonconforming is 0.] or lower, the probability of accepting the lot is 0.73 or higher. Disturbingly high probabilities of acceptance (0.5 i, 0.34, 0.21) remain even when the fraction of nonconforming products is high (e.g., probabilities of 0.51, 0.34, and 0.21 are associated with nonconforming fractions of 0.2, 0.3, and 0.4, respectively). These probability computations demonstrate the need to clearly define the objectives of the inspection program. If the objective is to control the process and ensure uniformity of processing, CUSUM is a good choice. However, if the objective is to ensure that no contaminated or otherwise defective meat reaches the consumer, CUSUM is inadequate. This distinction is crucial. Discussion In a broad sense, the question of the usefulness of the CUSUM is not a statistical one but relates more to the objectives of inspection and quality control in plants. The materials upon which this document is based suggest that the purpose of quality control and the role of meat inspectors in processing plants are viewed differently, depending on whose interests are being served. At least four different interests are involved in the processing of cattle: 1. Industry wants to produce an appealing finished product as efficiently, inexpensively, and quickly as possible. 38

OCR for page 34
1 LU ~7 <: 0.8 111 o 0.6 lo o ~ 0.4 m ~ 0.2 o a: 1~ 0L ' ~ o - - - - 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 FRACTION OF NONCONFORMING PRODUCTS Figure 4-2 Evaluation of CUSUM for SIS-C, Assuming n=3, c=0 2. USDA wants to fulfill its mandate to ensure wholesomeness, protect public health, prevent adulteration, and protect against economic fraud. 3. Consumer groups want a safe final product that has a good appearance. 4. Health organizations want to ensure that the product is not contaminated with disease-causing organisms, toxins, or chemicals. These goals may be an oversimplification of the issues, but they are important in evaluating inspection procedures and designing sampling schemes. There are probably no economically feasible sampling schemes that could satisfy all these concerns. An attempt to sample adequately is built into SIS-C (see Chapter 3) by having plants institute their own quality control programs that are monitored by USDA. According to this plan, USDA samples only for finished product standards and continues 100% inspection of head, carcass, and viscera. After this inspection, the head and viscera are separated from the carcass and company personnel cut and trim 39

OCR for page 34
for the cooler. At another company station, the cuts are again viewed and three carcass sides are sampled once or twice each shift and the products are examined for processing defects only. The inspectors maintain their CUSUM, which should be similar to the plant's score. This plan could be expected to accomplish the following in the best of circumstances: 1. The company quality control program (QC) could be designed to detect visual defects exceeding acceptable limits among the cattle being processed and remove the defects or condemn the defective meat. This would require a sampling scheme that could take into account cattle lots from different sources and other potentially important factors. 2. The 100% inspection of heads, viscera, and carcasses would accomplish exactly what traditional inspection has accomplished, assuming that the inspection is performed with the same thoroughness as in the traditional program. ^. Documentation and monitoring of plant QC would help ensure that deviations from the plant norm would be followed by appropriate actions. 4. CUSUM would allow inspectors to monitor the plant process so that the defects introduced and missed during processing could be monitored, and trends toward increasing numbers of defects could be detected and corrected. However, the method alone could not be expected to ensure that carcasses leaving the plant meet the expectations mentioned above. CUSUM is appropriate for maintaining process control, provided the tolerance levels are set for maintaining an acceptable process for all plants and are not at the level observed in a sampling of plants, and provided al] Jots are adequately sampled. However, CUSUM cannot eliminate the need for the other parts of SIS-C, including inspection and other testing (e.g., microbiological and toxicologic). It is important to distinguish between two objectives that might be part of the SIS-C in order to realize the impact of the probabilities presented in this chapter. One objective might be process control (i.e., ensuring that the process remains consistent in its detection of defects). CUSUM wall be helpful in attaining this objective. The other, and primary, objective should be preventing contaminated or defective meat from being accepted. CUSUM wall not ensure that the final product is free from defects or from microbial and chemical contamination. Recommendations The theoretical and statistical considerations underlying CUSUM should be 40

OCR for page 34
reevaluated with the following points in mind: The limitations of CUSUM as an inspection too] for assessing a nonuniform product must be clearly recognized. Because of these limitations, greater flexibility must be built into the sampling process along with an emphasis on optimizing the quality of the final product rather than simply obtaining acceptable CUSUM scores. For CUSUM, or any comparable sampling system, the statistical basis for setting initial sample sizes and tolerance levels must be validated in a series of ongoing controlled studies. USDA should evaluate CUSIJM's performance with respect to accepting `defective lots of meat. This should be done in two ways: (1) Operating characteristic curves should be developed for various schemes by using CUSUM. These can be used to evaluate performance under various conditions and assumptions by providing probabilities of accepting defective meat. Ideally, the probability of accepting defective meat should be zero or very close to zero. (2) CUSUM should be tested in plants to determine how often it actually accepts defective meat. This evaluation should also include observations on microbial or chemical contamination and should be designed with adequate sample numbers to facilitate valid interpretations of the results. 41