Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Committee on Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System March 13, 2009 Carol Maczka, Ph.D. Assistant Administrator USDA Food Safety and Inspection Service Office of Data Integration and Food Protection South Agriculture Building 1400 Independence Avenue, S.W., Room 3130 Washington, DC 20250 Dear Dr. Maczka, At the request of the Food Safety and Inspection Service (FSIS), the Institute of Medicine (IOM)—under the auspices of the Standing Com- mittee on the Use of Public Health Data in FSIS Food Safety Programs— established the Committee on Review of the Use of Process Control In- dicators in the FSIS Public Health Risk-Based Inspection System to re- view criteria developed by FSIS for ranking establishments based on relative risk. The body of this letter report provides the committee’s find- ings and recommendations regarding whether FSIS has adequately de- fined and identified indicators of process control that will be used to rank establishments and allocate agency inspection resources to protect public health. Specifically, the committee has evaluated how FSIS is proposing to use its available data to develop risk-based criteria for ranking estab- lishments, as described in the technical report Public Health Risk-Based Inspection System for Processing and Slaughter (PHRBIS; FSIS, 2008b). SUMMARY Overall, the committee finds FSIS’s commitment to developing a risk-based inspection system commendable and agrees with the general concept of using process control indicators as part of an algorithm to rank establishments in different levels of inspection. The committee also encourages FSIS to continue to provide the rationale and scientific evi- 1

OCR for page 1
2 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS dence serving as the basis for the proposed system and praises FSIS for its resilience as it improves the proposal with public comments. In gen- eral, the committee found it a challenge to evaluate the adequacy of indi- cators of process control to rank establishments and allocate agency in- spection resources without a clear understanding of the rationale for the general approach. The committee’s deliberations, based on its review of the report PHRBIS, open meetings, and personal communications with FSIS, resulted in the following findings: • The proposed inspection system consists of two components: one based on process control indicators and a second based on public health impact. The committee was tasked to review only the first component, but found it difficult to completely exclude delibera- tions on indicators of public health impact. • The report PHRBIS lacks details that are crucial to its evaluation. For example, the description of the algorithm, the scientific basis for the algorithm, the scientific basis for the use of the process indicators, the description and analysis of data, and the use of the process control indicator algorithm as it is integrated into the overall inspection system are not clearly articulated in the FSIS technical report. • The specific activities assigned to the three levels of inspection are not explicated. Likewise, the process of decision making to transfer a plant into a different level of inspection (LOI) (e.g., from LOI 2 to LOI 1) is not well defined. Further, it is not clear for how long or how frequently a plant in category LOI 2 or LOI 3 will be subject to an in-depth inspection or how these LOI des- ignations relate to current regulatory requirements. • Key terms of the algorithm, such as “process control indicators,” are not well defined. In addition, the proposed algorithm assigns the same weight to all process indicators, even though they vary in their ability to predict loss of process control. For example, some indicators may predict future loss of control (e.g., the rate of health-related noncompliance records [NRs]), but others might only reflect past loss of control (e.g., recalls). For some foods, no adequate process control indicator is proposed. • The statistical analysis that was conducted to find associations between proposed process control indicators—lift analysis—is a data-mining tool appropriate for use in finding initial associa- tions among events that occur infrequently. However, the identi-

OCR for page 1
LETTER REPORT 3 fication of process control indicators to properly categorize plants based on risk to public health requires more complex sta- tistical analysis as well as data that have been collected for the purpose of identifying such indicators. • Although there are limitations on the use of pathogenic organ- isms or Salmonella verification testing results as indicators of process control (e.g., infrequency of events), the committee con- cludes that the use of such testing to categorize plants in differ- ent levels of inspection is appropriate, if the recommendations stated in this report are followed. • FSIS currently tests each product class for different microorgan- isms, for different purposes, and with different underlying as- sumptions. The applicability of these data to the FSIS algorithm is dependent on the specific protocols, assumptions, and statisti- cal characteristics of each testing program. The FSIS technical report did not provide in-depth consideration of the statistics that underlie the specific microbiological testing protocols employed and the assumptions made when using such data (e.g., the mag- nitude of type I and type II errors). • The use of the rate of NR receipt as an indicator of process con- trol is promising but presents limitations based on the nature of the NRs (e.g., they document failure to comply with a regulation but are not always associated with a loss of process control or a public health hazard; NRs are subjective in nature; statistical analysis was conducted by aggregating data from all facilities, which might have biased the results). • Other proposed process control indicators also present limita- tions. The use of public health-related recalls, enforcement ac- tions, and outbreaks to rank establishments in different levels of inspection has been justified based on potential direct public health risk, a valid risk-management decision criterion. How- ever, the initial data analysis has not provided scientific support for these decision criteria as predictive of a loss of process con- trol or for their association with other indicators. The deliberations of the committee resulted in recommendations for improvement in the areas listed below that should be followed prior to implementing this algorithm:

OCR for page 1
4 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS • Definition of key terms used in developing the algorithm, spe- cifically, pointing out the limitations and consequences of using such terms in the context of the proposal; • Design of the algorithm, by conducting a risk-ranking activity to better identify process control indicators and their relative impor- tance; • Collection or retrieval of additional data for the purpose of con- firming the current process control indicators as well as explor- ing the use of new potential process control indicators to im- prove the sensitivity of the algorithm; and • Development of procedures to validate the algorithm. CHARGE TO THE COMMITTEE Responding to the request of the Food Safety and Inspection Service of the U.S. Department of Agriculture (USDA), the Institute of Medicine of the National Academies appointed the nine-member ad hoc Commit- tee on Review of the Use of Process Control Indicators in the FSIS Pub- lic Health Risk-Based Inspection System. Committee members provided expertise in meat and poultry microbiology, molecular biology methods, design and operation of processing establishments, risk analysis and de- cision-making tools, meat and poultry inspection, and foodborne disease epidemiology and public health. The committee met three times during the course of its work. The first meeting (Appendix A: Meeting Agen- das) was held on November 6-7, 2008, in conjunction with a public data- gathering session with FSIS representatives, who provided program background and an in-depth description of the committee’s task (Box 1). The committee’s second meeting on December 17-18, 2008, also in- cluded Dr. Artur Dubrawski, of Carnegie Mellon University, and Dr. Marc Huckabee and Dr. Curtis Travis, consultants to FSIS from Science Applications International Corporation, who conducted the statistical analysis. During an open session of that meeting, these invitees responded to the committee’s questions about the statistical analysis of the data on process control indicators that were used by FSIS to establish the proposed risk-based algorithm. In addition to discussions with FSIS representatives and consultants, the committee formally requested data and information from FSIS through the Freedom of Information Act, as suggested by FSIS representatives. The committee deliberated on the following process control indicators and the data analysis approaches used by FSIS to evaluate their potential inclusion in the algorithm:

OCR for page 1
LETTER REPORT 5 • Salmonella verification testing in raw meat and poultry • Pathogen testing in ready-to-eat (RTE) meat and poultry (Salmo- nella enterica, Listeria monocytogenes, and Escherichia coli O157:H7) and raw ground beef and its components (E. coli O157:H7) • Noncompliance records • Enforcement actions • Class I and II recalls • Pulsed field gel electrophoresis (PFGE) patterns of Salmonella serovars of particular human health concern for isolates derived from the raw meat and poultry Salmonella verification testing program • System for Tracking E. coli O157:H7 Positive Suppliers (STEPS) The committee also discussed the potential use of other indicators that were not included in the FSIS proposal. Findings and recommenda- tions were drafted. A third committee meeting was held on January 13- 14, 2009, in Washington, DC, to finalize its findings and recommenda- tions and to prepare the report for external review. The committee reviewed the data and statistical analysis (Appen- dixes D and E of the technical report Public Health Risk-Based Inspec- tion System for Processing and Slaughter [FSIS, 2008b]) provided for the proposed indicators listed above. Appendix D of that report includes a description of the data used; Appendix E describes the data analysis that was conducted and the conclusions derived thus far. Appendix D and E also include limitations of the data and analysis and the rationale for the design of the algorithm. At the request of FSIS and because another National Academy of Sciences (NAS) committee (Committee on Review of the Food Safety and Inspection Service [FSIS] Risk-Based Approach to Public Health Attribution) was assigned the task, data on volume and food attribution were not reviewed by this committee. FSIS noted that this algorithm would undergo improvements during the committee’s deliberation and, therefore, the proposal should be considered preliminary; since the publi- cation of its technical report, FSIS has slightly modified the selection of process indicators. The committee based its deliberations on the updated version of the algorithm that was presented at its meeting on November 6-7, 2008 (see the indicators of process control and levels of inspection

OCR for page 1
6 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS in Appendix B of this report). The committee additionally reviewed sev- eral other FSIS reports, such as the 2008 technical report on poultry slaughter provided to the National Advisory Committee on Meat and Poultry Inspection (NACMPI) (FSIS, 2008a), to better understand the evolution of the FSIS algorithm. This letter report begins with a background description of the FSIS initiative of a risk-based inspection system. Overall recommendations and findings are followed by recommendations for each specific process control indicator reviewed. The agenda of the workshop held on Novem- ber 6-7, 2008, and the agenda for the open session of the second meeting are provided in Appendix A. Appendix B lists the indicators of process control corresponding to each level of inspection. Appendixes C and D contain a list of acronyms and a glossary, respectively. Appendix E lists the committee members’ biosketches. BOX-1 Statement of Task An ad hoc committee will review whether the Food Safety and Inspection Service (FSIS) has adequately defined and identified indicators of process control to protect public health that will be used to rank establishments and allocate agency inspection resources. Specifically, the committee will evaluate how FSIS is proposing to use its available data to develop a relative risk rank- ing of establishments described in the technical report Public Health Risk- Based Inspection System for Processing and Slaughter, publicly posted at http://www.fsis.usda.gov/Regulations_&_Policies/National_Advisory_Committe e_on_Meat_&_Poultry/index.asp. BACKGROUND Public Health Risk-Based Inspection System for Processing and Slaughter The Food Safety and Inspection Service, the USDA agency respon- sible for ensuring the safety of meat, poultry, and egg products, has ex- amined a number of strategies to develop a risk-based food safety sys- tem. Examples include the development and implementation of the Pathogen Reduction; Hazard Analysis and Critical Control Point (PR/HACCP) Systems; Final Rule in 1996 (FSIS, 1996), the develop- ment of microbiological performance standards (FSIS, 1999), and re- quirements for pathogen testing of ready-to-eat foods (Requirements for specific classes of product. 2008. 9 CFR Part 430).

OCR for page 1
LETTER REPORT 7 In January 1997, President Clinton announced a Food Safety Initia- tive to reduce the incidence of foodborne disease in the United States. Among other changes, government agencies in charge of ensuring food safety were directed to improve inspections and enforce HACCP compli- ance in establishments that process meat and poultry (FDA-USDA-EPA- CDC, 1997). It was anticipated that implementation of the HACCP sys- tem would be accompanied by concurrent changes in inspection proce- dures. In 2003, the IOM Committee on Review of the Use of Scientific Criteria and Performance Standards for Safe Food found that the inspec- tion of FSIS-regulated establishments relied largely on visual and or- ganoleptic observations rather than on risk to public health (IOM, 2003). Although these are important and necessary elements of a plant survey, an improved, risk-based inspection system would assign levels of inspec- tion to establishments according to the magnitude of their product’s risk to the public’s health. Other organizations, including the National Acad- emies (NRC, 1987; IOM, 1990) and the Government Accountability Of- fice (GAO, 1992), have previously emphasized the need for a risk-based inspection system for meat and poultry products. In 2006, FSIS initiated the development of a risk-based inspection system. In its technical report Public Health Risk-Based Inspection Sys- tem for Processing and Slaughter (hereafter referred to as PHRBIS) (FSIS, 2008b), FSIS proposes a decision-making tool to rank establish- ments according to their risk to public health by categorizing them first acccording to their level of process control and then by the impact on public health of the food produced. In addition, FSIS intends to upgrade several other elements of the proposed inspection system. For example, FSIS plans to strengthen its information technology system to enable inspection personnel to enter data on hazard analysis and make subse- quent decisions in a more integrated and objective manner (FSIS, 2008b). FSIS also plans to train its inspection force (inspectors and su- pervisors) in effective use of the proposed inspection system tools. For example, in addition to continuing routine inspection training, a group of in-plant inspectors will receive training to enhance their understanding of establishment food safety systems, including HACCP plans or sanitary requirements. Supervisors will also be trained to use a more streamlined inspection review process (E. Dreyling, FSIS, personal communication, December 13, 2008). As FSIS describes in its technical report PHRBIS, the proposed tool has evolved with input from stakeholder groups as well as the USDA’s National Advisory Committee for Meat and Poultry Inspection. An im- portant innovation of this current proposal is to rely, where possible, on

OCR for page 1
8 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS data collected in conjunction with FSIS’s regulatory programs (FSIS, 2008b). The ultimate aim is the production of an effective tool for achieving the Healthy People 2010 goals of reducing foodborne disease caused by Salmonella, Escherichia coli O157:H7, and Listeria monocy- togenes (HHS, 2000). FSIS concludes that to protect public health most effectively, inspection resources have to be allocated based on the degree of risk to public health presented by each processing plant. Therefore, a key element of the risk-based inspection system is an algorithm for cate- gorizing slaughter and processing plants according to risk so that inspec- tion efforts are focused on those establishments having the greatest im- pact on public health (FSIS, 2008b). The algorithm consists of two consecutive steps to rank an establishment’s risk: a first component to determine the establishment’s level of process control (i.e., identifying attributes that indicate whether the establishment is maintaining control) and a second component to quantify public health impact (i.e., the vol- ume of the commodity produced at the establishment together with pub- lic health attribution of the food produced) (FSIS, 2008b). The commit- tee was charged with reviewing the scientific basis of and rationale for the first component of the algorithm—the data and data analysis that were used by FSIS to identify indicators for categorizing establishments according to their level of process control. A second NAS ad hoc com- mittee (Committee on Review of the Food Safety and Inspection Service [FSIS] Risk-Based Approach to Public Health Attribution) was charged with reviewing the second component, the public health attribution sys- tem. Because the two components are closely related (e.g., the volume of production in an establishment influences the sampling plans for patho- gen testing programs that FSIS proposes to use to indicate process con- trol) and included in an overall inspection system, this committee found it difficult to completely exclude deliberations on indicators of public health impact. OVERALL FINDINGS AND RECOMMENDATIONS This section of the report provides overall findings and recommenda- tions related to strengthening the proposed FSIS risk-based decision tools for ranking establishments. It is followed by a section that provides more specific recommendations for each proposed indicator. Prior to imple- menting the algorithm, the recommendations in this report should be fol- lowed.

OCR for page 1
LETTER REPORT 9 General Approach The committee concurs that a risk-based approach to inspection is essential and commends FSIS for undertaking such a daunting and con- troversial endeavor. The committee found that the development and use of a model (algorithm) to categorize establishments based on risk can ultimately be a systematic approach to realizing and implementing deci- sion criteria in a transparent, predictable manner. However, the commit- tee found it challenging to comprehend the framework, concepts, and rationale that FSIS applied in several segments of the proposed model. The descriptions of the algorithm, the scientific basis for the selection of the proposed process indicators, the analysis of data, and the use of the process control indicator algorithm as it is integrated in the overall in- spection system were not clearly stated in the technical report PHRBIS that was provided to the committee. For example, FSIS uses the term “algorithm” to describe its decision-making tool to categorize plants into levels of inspection. As shown in Table 1, there are various definitions of the term algorithm. However, in the context of a risk-based system, the term algorithm implies a mathematical model. Since FSIS did not construct a mathe- matical model, it would be more precise to use the designation decision tool or framework. To avoid confusion for the reader, the committee de- cided to retain the term algorithm for the purposes of this report. TABLE 1 Definitions of Algorithm Definition Source A set of rules for solving a problem in a finite number http://dictionary. of steps, (e.g., finding the greatest common divisor) reference.com/ A procedure for solving a mathematical problem (e.g., http://www.merriam- finding the greatest common divisor) in a finite num- webster.com/ ber of steps that frequently involves repetition of an operation; broadly: a step-by-step procedure for solv- ing a problem or accomplishing some end especially by computer A precise rule (or set of rules) specifying how to solve http://www.websters- some problem online-dictionary.org/ Continued

OCR for page 1
10 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS Table 1 continued Mathematics. A process, or set of rules, usually ex- Oxford English Dic- pressed in algebraic notation, now used especially in tionary, 2nd edition, computing, machine translation, and linguistics 1982 Medicine. A step-by-step procedure for reaching a clinical decision or diagnosis, often set out in the form of a flow chart, in which the answer to each question determines the next question to be asked Any special method of solving a certain kind of prob- Webster’s New World lem; specifically, the repetitive calculations used in Dictionary, 2nd col- finding the greatest common divisor of two numbers lege edition, 1982 Finding 1: Although the use of a model to categorize plants in levels of inspection is appropriate, the descriptions of the algorithm, the scientific basis for the use of the process indicators, the description and analysis of data, and the use of the process control indicator algorithm as it is inte- grated into the overall inspection system are not clearly articulated in the FSIS technical report. RECOMMENDATION 1: The committee recommends that in addition to the improvements in data collection and analysis presented below, FSIS revise its proposal to improve the transparency and clarity of the description of the overall inspection system—in particular, the process control indicator algorithm, its scientific basis, and the type and analysis of data used. Further, FSIS should consider tailoring the proposal to its target audiences (e.g., plant managers, FSIS inspectors and supervisors, FSIS managers and scientists, outside expert panels) and providing them with supplemental information or reports. Definitions of Process Control and Process Control Indicators The FSIS (2008b) report does not adequately define various terms that are key to evaluating the proposed inspection system (e.g., algo- rithm, process control, process control indicator). The ambiguous use of these terms hampered the ability of the committee to understand the use of data and could result in misinterpretations and unnecessary disputes in the future. To avoid confusing the reader and for the purposes of this re- port, however, the committee opted to retain the terms process control and process control indicators while also pointing out the ambiguity of their usage.

OCR for page 1
LETTER REPORT 11 The committee offers more clearly defined key terms and explains its interpretation of those terms for the purposes of this report. The concept of process control, which applies to all manufacturing companies and can be used broadly to address both quality and safety issues, is used in the context of the current report as a means to quantify how well an estab- lishment is employing control measures to minimize pathogen contami- nation. Examples of definitions of process control are shown in Table 2. TABLE 2 Definitions of Process Control Definition Source At certain points in the processing of a food, con- Scientific Criteria to En- trol measures can be applied to prevent an unac- sure Safe Food (IOM, ceptable increase in a hazard, eliminate it, or re- 2003, p. 94) duce it to an acceptable level Activities involved in ensuring a process is pre- BusinessDictionary.com dictable, stable, and consistently operating at the (http://www.business target level of performance with only normal dictionary.com/) variation The inspection of work-in-progress to provide bnet.com feedback on, and correct, a production process. (http://www.bnet.com/) First developed as a mechanical feedback mecha- nism, process control is now widely used to moni- tor and maintain the quality of output Method by which the input flow of processing Chemicals- plants is automatically controlled and regulated technology.com by various output sensor measurements. Process (http://www.chemicals- control can also describe the method of keeping technology.com/ processes within specified boundaries and mini- glossary/) mizing variation within a process The active changing of a process based on the NIST/SEMATECH results of process monitoring. Once the process e-Handbook of Statistical monitoring tools have detected an out-of-control Methods situation, the person responsible for the process (http://www.itl.nist.gov/ makes a change to bring the process back into div898/handbook/pmc/ control section1/pmc13.htm) The automated control of a process. Process con- PCMag.com trol is used extensively in oil refining, chemical (http://www.pcmag.com/ processing, electrical generation, and the food and encyclopedia_term/0,254 beverage industries where the creation of a prod- 2,t=process+control&i=49 uct is based on a continuous series of processes 753,00.asp) being applied to raw materials In its 2003 report Scientific Criteria to Ensure Safe Food, IOM evaluated the use of scientific criteria and standards in food safety regu- lations (IOM, 2003). That report defines various terms used in food

OCR for page 1
50 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS recalls is further limited by their lack of specificity as well as their lack of timeliness (they may occur too late to identify issues in food process- ing); their use is reactive to a past food safety problem, not necessarily predictive of a future problem. As FSIS states, the statistical analysis that estimates the ability of re- calls to predict a loss of process control should allow for differentiation between recalls that have a public health impact and those that do not. A public health-related recall is often based on a laboratory test result—that is, the result of a microbiological test. Because of the limitations of sam- pling (size, frequency, etc.) and the sporadic nature of contamination, the isolation of a foodborne pathogen may not indicate a breakdown in proc- ess control. In some situations, especially related to E. coli O157:H7 in ground beef, a recall is based on failure to hold a product that has been tested for a pathogen, not a failure of the control process. Finding 12: The use of public health-related recalls to rank establish- ments in different levels of inspection has been justified based on poten- tial direct public health risk, a valid risk-management decision criterion. However, the initial data analysis has not provided scientific support for this decision criterion as being predictive of a loss of process control or for its association with other indicators. RECOMMENDATION 14: Only health-related product recalls should be included in the model for ranking public health risks and assigning inspection resources. FSIS should continue to conduct assessments and take regulatory enforcement actions in plants following a recall. STEPS Database Use and Scientific Evidence The System for Tracking E. coli O157:H7 Positive Suppliers data- base identifies suppliers of trim to grinding operations whose ground beef product tests positive for E. coli O157:H7. FSIS proposes to use this database to categorize LOIs for supplier establishments in the following manner (Dreyling, 2008; FSIS, 2008b; E. Dreyling, FSIS, personal communication, February 18, 2009):

OCR for page 1
LETTER REPORT 51 • LOI 1: Establishment has not been cited in the STEPS database more than once; or if it has, any related FSA and follow-up sampling has been completed more than 120 days previously, any related enforcement actions are closed, and establishment meets all other criteria for LOI 1. • LOI 2: For an establishment in the STEPS database more than once, any related FSA and follow-up sampling has been com- pleted in the previous 120 days, and all related enforcement ac- tions are deferred or in abeyance. • LOI 3: Establishment was in the STEPS database more than once within the previous 120 days. The justification for using the STEPS database is that grinding op- erations lacking an E. coli O157:H7 intervention step that would de- crease the likelihood of the presence of pathogens need a process control step to ensure that incoming trim products are not already contaminated with pathogens. Committee’s Discussion The committee recognizes the potential benefits of this approach and would be interested in seeing the details and data supporting it. As dis- cussed above in relation to testing ground beef for E. coli O157:H7, FSIS should assess the role of testing trim for E. coli O157:H7 as a risk deter- minant. Foodborne Disease Outbreaks A foodborne outbreak is the occurrence of two or more cases of a similar illness resulting from the ingestion of a common food. FSIS pro- poses to use foodborne disease outbreaks to categorize establishments in levels of inspections as follows (Dreyling, 2008; FSIS, 2008b; E. Drey- ling, FSIS, personal communication, December 13, 2008): • LOI 1: An establishment has not been linked to an outbreak; or if it has, any related FSA and follow-up sampling has been com- pleted more than 120 days previously, any related enforcement actions are closed, and establishment meets all other criteria for LOI 1.

OCR for page 1
52 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS • LOI 2: For an establishment that was linked to an outbreak, any related FSA and follow-up sampling has been completed in the previous 120 days, and all related enforcement actions are de- ferred or in abeyance. • LOI 3: Human illness was linked to an FSIS-regulated product from the establishment. Finding 13: The use of foodborne disease outbreaks to rank establish- ments in different levels of inspection has been justified based on their potential direct public health risk, a valid risk-management decision cri- terion. However, the initial data analysis has not provided scientific sup- port for use of this decision criterion to predict loss of process control or for its association with other indicators. RECOMMENDATION 15: The committee recommends including foodborne disease outbreaks in the algorithm to categorize plants in lev- els of inspection. The committee also strongly recommends that FSIS systematically work with other appropriate federal and state agencies to routinely disseminate public reports of the results of the investigations into the plant and process failures associated with these outbreaks. Salmonella Serotypes of Human Health Concern If FSIS is planning to use specific serotypes as indicators of process control, serotypes not often linked to human health should also be con- sidered. Since the potential application of serotype evaluation to the des- ignation of facilities as LOI 1, 2, or 3 is dependent on establishing a clear relationship between individual serotypes and disease attribution, any recommendations by this ad hoc committee await the findings of the NAS Committee on Review of the Food Safety and Inspection Service (FSIS) Risk-Based Approach to Public Health Attribution. Consumer Complaints The use of consumer complaints as a potential indicator of process control was analyzed by FSIS and then dismissed due to the challenge of overcoming its limitations (FSIS, 2008b). The committee agrees that the process currently employed to collect and analyze consumer complaints is not appropriate for use as an indicator of an establishment’s need for a

OCR for page 1
LETTER REPORT 53 higher level of inspection. In addition to having no significant associa- tions with other potential indicators (see Appendix E in FSIS, 2008b), consumer complaints may often incorrectly associate a food with an ad- verse health effect. OTHER POTENTIAL INDICATORS OF PROCESS CONTROL Microbial Test Results The primary goal of process control for raw meat and poultry prod- ucts is to limit the presence of fecal contamination, the source of enteric pathogenic microorganisms. Both Salmonella and generic E. coli are in- dicators of fecal contamination and, as such, indicators of loss of process control, and both were targeted by FSIS in the pathogen reduction HACCP regulation. The ideal process indicator is one that is present at sufficient levels and frequency to be measured on a routine basis. For some commodities (e.g., beef carcasses), Salmonella is currently found so rarely that its usefulness as an indicator is limited. It is envisioned that for other commodities where it is currently useful, Salmonella could be- come equally rare in the future. FSIS would benefit from identifying al- ternative microbial indicators that could augment current indicators of fecal contamination on a commodity-specific basis. Data on generic E. coli are collected by individual plants on a regular basis, but are not used by FSIS. Establishments are not required to send such data to FSIS, only to make them available if requested. According to FSIS, there are two limitations to the collection of data on generic E. coli that prevent FSIS from using them as indicators (C. Travis, Science Applications International Corporation, personal communication, De- cember 13, 2008). One is FSIS’s concern about the comparability of data resulting from a variety of different testing methods. Although the PR/HACCP regulation states that validated methods of testing should be used, there is no required single standard testing methodology or sam- pling procedure. It would appear that this could readily be corrected if FSIS articulated the specific methodological requirements (e.g., sensitiv- ity, specificity, reproducibility, repeatability) of its current standard methods and its expectation that similar performance would be achieved by alternative validated methods. The second limitation is that the agency does not currently have the information technology capability to

OCR for page 1
54 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS retrieve and process such data efficiently. It is worth noting that in its May 2008 technical report to the NACMPI on poultry slaughter, FSIS presented convincing evidence of the potential utility of generic E. coli as a process control indicator and suggested that it was considering a new performance standard based on its use (FSIS, 2008a). These findings were based on a detailed study of various potential indicators of process control performed by FSIS and ARS. FSIS indicated that it has analyzed a 2006-2008 generic E. coli data set from its baseline program, but be- cause this data set is small, the results were inconclusive (FSIS, 2008a). Indicator organisms in RTE foods are used not as indicators of fecal contamination but rather as indicators of other control measures, such as the adequacy of the microbicidal step, prevention of recontamination, and maintenance of proper storage conditions (e.g., refrigeration). The FSIS algorithm does not currently include the use of any indicator mi- croorganism for assessing process control in RTE foods. Given the low frequency of L. monocytogenes or other pathogens, FSIS would benefit from identifying appropriate alternative microbial indicators that could be used to assess applicable process controls in RTE foods. Although generic E. coli might not be an indicator of fecal contamination in RTE products, it is still a valuable indicator of general sanitation, recontami- nation problems, and temperature abuse. Finding 14: Microbes currently used as process control indicators are only rarely found in some commodities and are therefore of limited use- fulness (e.g., Salmonella in ground beef). It is anticipated that in the fu- ture, Salmonella will be even less frequent and therefore less valuable as an indicator. Furthermore, in the proposed algorithm, there are no iden- tified process control indicators for RTE foods. RECOMMENDATION 16: FSIS should investigate the potential util- ity of industry data on generic E. coli as an indicator of process control. The committee recognizes the challenges of this approach, but encour- ages FSIS to act promptly to complete the analysis of the data it has al- ready acquired, collect additional data as necessary, and analyze them for their predictive ability as potential indicators of process control. Use of the HACCP System A HACCP plan is developed by identifying steps in a specific meat and poultry process that are critical to ensuring food safety and is meant

OCR for page 1
LETTER REPORT 55 to include the application of corrective actions when those critical control points are not met. Critical control points in HACCP plans were regarded as points in a process in need of specific interventions that, if failed, might result in an end product with risks to public health. If a commodity that was produced under a process deviation leaves the plant without cor- rective action, this constitutes a loss of control. By regulation, the control point limits are to be validated and verified by the establishments (Haz- ard Analysis and Critical Control Point [HACCP] Systems. Validation, Verification, Reassessment. 2008. 9 CFR § 417.4). Therefore, it may be appropriate for FSIS to study the feasibility of a system in which devia- tions from control point limits are incorporated as NRs and used to cate- gorize plants according to the inspection level required. In fact, for most HACCP plans, critical control points and limits should be similar in na- ture for all facilities processing the same commodity. The committee ac- knowledges that for the processing of raw product, defining the control points is challenging; in these cases, more weight could be allocated to pathogen contamination as a control indicator. RECOMMENDATION 17: The committee recommends that FSIS consider using specific critical control point deviations as indicators of process control. Process deviations should be integrated into an algo- rithm to categorize plants according to the level of inspection needed. Because of inherent problems in the use of NRs described above, the committee recommends redefining public health-related NRs and creat- ing new ones where appropriate so that they reflect the current view of HACCP as a food safety control approach. This approach should identify true science-based indicators of process control. This concept should be included in inspection training programs. USDA should conduct a pilot study in a few plants to determine if the new NRs based on HACCP critical control point adherence are valid and useful parameters to be considered as predictors of loss of process control. This should be fol- lowed by longitudinal studies designed to validate the new NRs. Value of Real-Time In-Plant Data Acquisition The committee supports FSIS’s efforts to explore options for rapid collection and reporting of real-time data that indicate potential failures of process control. The real-time data should focus on objective meas- ures of control (e.g., critical control points) for the process and take ad- vantage of electronic data-capturing tools.

OCR for page 1
56 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS The committee agrees that FSIS should use in-plant inspection per- sonnel to collect real-time data. They would provide immediate input into the algorithm indicating a potential failure of process control. To be effective, objective process performance measures should be defined. For example, real-time tracking of repetitive instances of noncompliance that are related to food safety and that affect process control would be re- ported and used as indicators of process control. Also, whenever feasible, performance measures should allow action to be taken before the process fails. The committee supports FSIS’s current activities to develop such a system and urges that it do so concurrently with carefully designed train- ing of its inspection and supervisory personnel. CONCLUSION The committee recognizes the magnitude of the task of designing a risk-based system to rank meat and poultry slaughtering and processing establishments based on their impact on public health. The committee notes that at the request of FSIS, only the data on and analysis of indica- tors of process control were reviewed. Other components of the algo- rithm (e.g., volume) vital to determining its applicability were not. FSIS should include as part of the proposed inspection system a specific plan for when and how it will evaluate the system. Scientific verification and validation are essential to evaluate the success or failure of the new pro- gram. The committee agrees with the general concept of using process con- trol indicators as part of an algorithm to rank establishments in different levels of inspection. The committee recommends that FSIS continue the collection and analysis of data and, in consultation with stakeholders and expert panels, continue to improve its proposed risk-based inspection system so that it more effectively allocates inspection resources accord- ing to risk. Prior to implementing this algorithm, the recommendations in this report should be followed. Specifically, the committee emphasizes the need to align the process control indicators of a risk-based inspection system with HACCP, a framework required throughout the meat and poultry slaughtering and processing industry that serves to minimize the risk of foodborne illness. The committee also recommends that FSIS improve the clarity and transparency of the algorithm so that its intent, scientific basis, and im- plementation are clearly articulated and understood by all stakeholders. One option for FSIS to communicate effectively with stakeholders would

OCR for page 1
LETTER REPORT 57 be to produce supplemental informative documents targeted to specific audiences (e.g., inspectors, plant managers), in addition to a technical report. Also, because this new algorithm would bring about changes in inspection procedures, a parallel training program for the inspection force would likewise be necessary. The Committee on Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System thanks FSIS for the opportunity to review the technical report Public Health Risk-Based Inspection System for Processing and Slaughter and hopes that its find- ings and recommendations are useful. The committee will be available to FSIS for any clarifications regarding this letter. Sanford Miller, Chair Committee on Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System Attachments Appendix A Meeting Agendas Appendix B Levels of Inspection Appendix C Acronyms Appendix D Glossary Appendix E Biographical Sketches of Committee Members REFERENCES CDC (Centers for Disease Control and Prevention). 2008. Preliminary FoodNet data on the incidence of infection with pathogens transmitted commonly through food—10 states, 2007. Morbidity and Mortality Weekly Report 57(14):366-370. Dreyling, E. 2008. Public health risk ranking for processing and slaughter es- tablishments. Presented at the November 6, 2008, meeting of the IOM Committee on Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System, Washington, DC. FDA-USDA-EPA-CDC (U.S. Food and Drug Administration-U.S. Department of Agriculture-U.S. Environmental Protection Agency-Centers for Disease Control and Prevention). 1997. Food Safety from farm to table: A national food safety initiative. Report to the President. Available at http://www.foodsafety.gov/~dms/fsreport.html (accessed February 20, 2009). FSIS (Food Safety and Inspection Service). 1996. 9 CFR Parts 304, 308, 310, 320, 327, 416, and 417. Pathogen Reduction; Hazard Analysis and Critical

OCR for page 1
58 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS Control Point (HACCP) Systems; Final Rule. Federal Register 61(144):38805-38989. FSIS. 1999. 9 CFR Parts 301, 317, 318, 320, and 381. Performance standards for the production of certain meat and poultry products; Final Rule. Federal Register 64(3):732-749. FSIS. 2008a. Improvements for poultry slaughter inspection. Technical report. Available at http://www.fsis.usda.gov/Regulations_&_Policies/National Advisory_Committee_on_Meat_&_Poultry/index.asp#August (accessed January 26, 2009). FSIS. 2008b. Public health risk-based inspection system for processing and slaughter. Technical report. Available at http://www.fsis.usda.gov/ Regulations_&_Policies/National_Advisory_Committee_on_Meat_&_Poult ry/index.asp#August (accessed January 7, 2009). FSIS. 2008c. Progress report on Salmonella testing of raw meat and poultry products, 1998-2007. Available at http://www.fsis.usda.gov/Science/ Progress_Report_Salmonella_Testing/index.asp (accessed February 23, 2009). FSIS. 2008d. The FSIS Microbiological Testing Program for Ready-to-Eat (RTE) Meat and Poultry Products, 1990-2007. Available at http://www.fsis. usda.gov/Science/Micro_Testing_RTE/index.asp#trends (accessed February 23, 2009). FSIS. 2008e. Current recalls and alerts. Available at http://www.fsis.usda.gov/ Fsis_Recalls/Open_Federal_Cases/index.asp (accessed December 7, 2008). GAO (U.S. General Accounting Office). 1992. Food safety and quality. Uni- form, risk-based inspection system needed to ensure safe food supply. Re- port to the Chairman, Subcommittee on Oversight and Investigations, Committee on Energy and Commerce, U.S. House of Representatives. Re- sources, Community, and Economic Division, Report 92-152. Washington, DC. Giovannacci, I., C. Ragimbeau, S. Queguiner, G. Salvat, J. L. Vendeuvre, V. Carlier, and G. Ermel. 1999. Listeria monocytogenes in pork slaughtering and cutting plants. Use of RAPD, PFGE and PCR-REA for tracing and mo- lecular epidemiology. International Journal of Food Microbiology 53(2- 3):127-140. HHS (U.S. Department of Health and Human Services). 2000. Healthy people 2010, Volume 1. Washington, DC: U.S. Government Printing Office. Avail- able at http://www.healthypeople.gov/Publications/ (accessed January 7, 2009). ICMSF (International Commission on Microbiological Specifications for Foods). 2002. Microorganisms in foods 7. Microbiological testing in food safety management. New York: Klewer Academic/Plenum Publishers. IOM (Institute of Medicine). 1990. Cattle inspection. Washington, DC: The National Academy Press.

OCR for page 1
LETTER REPORT 59 IOM. 1998. Ensuring safe food: From production to consumption. Washington, DC: The National Academy Press. IOM. 2003. Scientific criteria to ensure safe food. Washington, DC: The Na- tional Academies Press. Keto-Timonen, R., R. Tolvanen, J. Lundén, and H. Korkeala. 2007. An 8-year surveillance of the diversity and persistence of Listeria monocytogenes in a chilled food processing plant analyzed by amplified fragment length poly- morphism. Journal of Food Protection 70(8):1866-1873. Lundén, J. M., T. J. Autio, A. M. Sjöberg, and H. J. Korkeala. 2003. Persistent and nonpersistent Listeria monocytogenes contamination in meat and poul- try processing plants. Journal of Food Protection 66(11):2062-2069. NRC (National Research Council). 1987. Poultry inspection: The basis for a risk-assessment approach. Washington, DC: National Academy Press. Peccio, A., T. Autio, H. Korkeala, R. Rosmini, and M. Trevisani. 2003. Listeria monocytogenes occurrence and characterization in meat-producing plants. Letters in Applied Microbiology 37(3):234-238. Thévenot, D., A. Dernburg, and C. J. Vernozy-Rozand. 2006. An updated re- view of Listeria monocytogenes in the pork meat industry and its products. Applied Microbiology 101(1):7-17. WHO-FAO (World Health Organization-Food and Agriculture Organization of the United Nations). 2004. Risk assessment of Listeria monocytogenes in ready-to-eat food: Technical report. Microbiological Risk Assessment Se- ries 5. Geneva, Switzerland: WHO.

OCR for page 1