National Academies Press: OpenBook
« Previous: Front Matter
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 1
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 2
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 3
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 4
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 5
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 6
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 7
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 8
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 9
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 10
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 11
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 12
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 13
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 14
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 15
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 16
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 17
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 18
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 19
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 20
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 21
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 22
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 23
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 24
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 25
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 26
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 27
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 28
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 29
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 30
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 31
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 32
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 33
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 34
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 35
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 36
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 37
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 38
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 39
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 40
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 41
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 42
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 43
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 44
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 45
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 46
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 47
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 48
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 49
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 50
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 51
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 52
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 53
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 54
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 55
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 56
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 57
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 58
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 59
Suggested Citation:"Letter Report ." Institute of Medicine. 2009. Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report. Washington, DC: The National Academies Press. doi: 10.17226/12617.
×
Page 60

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Committee on Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System March 13, 2009 Carol Maczka, Ph.D. Assistant Administrator USDA Food Safety and Inspection Service Office of Data Integration and Food Protection South Agriculture Building 1400 Independence Avenue, S.W., Room 3130 Washington, DC 20250 Dear Dr. Maczka, At the request of the Food Safety and Inspection Service (FSIS), the Institute of Medicine (IOM)—under the auspices of the Standing Com- mittee on the Use of Public Health Data in FSIS Food Safety Programs— established the Committee on Review of the Use of Process Control In- dicators in the FSIS Public Health Risk-Based Inspection System to re- view criteria developed by FSIS for ranking establishments based on relative risk. The body of this letter report provides the committee’s find- ings and recommendations regarding whether FSIS has adequately de- fined and identified indicators of process control that will be used to rank establishments and allocate agency inspection resources to protect public health. Specifically, the committee has evaluated how FSIS is proposing to use its available data to develop risk-based criteria for ranking estab- lishments, as described in the technical report Public Health Risk-Based Inspection System for Processing and Slaughter (PHRBIS; FSIS, 2008b). SUMMARY Overall, the committee finds FSIS’s commitment to developing a risk-based inspection system commendable and agrees with the general concept of using process control indicators as part of an algorithm to rank establishments in different levels of inspection. The committee also encourages FSIS to continue to provide the rationale and scientific evi- 1

2 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS dence serving as the basis for the proposed system and praises FSIS for its resilience as it improves the proposal with public comments. In gen- eral, the committee found it a challenge to evaluate the adequacy of indi- cators of process control to rank establishments and allocate agency in- spection resources without a clear understanding of the rationale for the general approach. The committee’s deliberations, based on its review of the report PHRBIS, open meetings, and personal communications with FSIS, resulted in the following findings: • The proposed inspection system consists of two components: one based on process control indicators and a second based on public health impact. The committee was tasked to review only the first component, but found it difficult to completely exclude delibera- tions on indicators of public health impact. • The report PHRBIS lacks details that are crucial to its evaluation. For example, the description of the algorithm, the scientific basis for the algorithm, the scientific basis for the use of the process indicators, the description and analysis of data, and the use of the process control indicator algorithm as it is integrated into the overall inspection system are not clearly articulated in the FSIS technical report. • The specific activities assigned to the three levels of inspection are not explicated. Likewise, the process of decision making to transfer a plant into a different level of inspection (LOI) (e.g., from LOI 2 to LOI 1) is not well defined. Further, it is not clear for how long or how frequently a plant in category LOI 2 or LOI 3 will be subject to an in-depth inspection or how these LOI des- ignations relate to current regulatory requirements. • Key terms of the algorithm, such as “process control indicators,” are not well defined. In addition, the proposed algorithm assigns the same weight to all process indicators, even though they vary in their ability to predict loss of process control. For example, some indicators may predict future loss of control (e.g., the rate of health-related noncompliance records [NRs]), but others might only reflect past loss of control (e.g., recalls). For some foods, no adequate process control indicator is proposed. • The statistical analysis that was conducted to find associations between proposed process control indicators—lift analysis—is a data-mining tool appropriate for use in finding initial associa- tions among events that occur infrequently. However, the identi-

LETTER REPORT 3 fication of process control indicators to properly categorize plants based on risk to public health requires more complex sta- tistical analysis as well as data that have been collected for the purpose of identifying such indicators. • Although there are limitations on the use of pathogenic organ- isms or Salmonella verification testing results as indicators of process control (e.g., infrequency of events), the committee con- cludes that the use of such testing to categorize plants in differ- ent levels of inspection is appropriate, if the recommendations stated in this report are followed. • FSIS currently tests each product class for different microorgan- isms, for different purposes, and with different underlying as- sumptions. The applicability of these data to the FSIS algorithm is dependent on the specific protocols, assumptions, and statisti- cal characteristics of each testing program. The FSIS technical report did not provide in-depth consideration of the statistics that underlie the specific microbiological testing protocols employed and the assumptions made when using such data (e.g., the mag- nitude of type I and type II errors). • The use of the rate of NR receipt as an indicator of process con- trol is promising but presents limitations based on the nature of the NRs (e.g., they document failure to comply with a regulation but are not always associated with a loss of process control or a public health hazard; NRs are subjective in nature; statistical analysis was conducted by aggregating data from all facilities, which might have biased the results). • Other proposed process control indicators also present limita- tions. The use of public health-related recalls, enforcement ac- tions, and outbreaks to rank establishments in different levels of inspection has been justified based on potential direct public health risk, a valid risk-management decision criterion. How- ever, the initial data analysis has not provided scientific support for these decision criteria as predictive of a loss of process con- trol or for their association with other indicators. The deliberations of the committee resulted in recommendations for improvement in the areas listed below that should be followed prior to implementing this algorithm:

4 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS • Definition of key terms used in developing the algorithm, spe- cifically, pointing out the limitations and consequences of using such terms in the context of the proposal; • Design of the algorithm, by conducting a risk-ranking activity to better identify process control indicators and their relative impor- tance; • Collection or retrieval of additional data for the purpose of con- firming the current process control indicators as well as explor- ing the use of new potential process control indicators to im- prove the sensitivity of the algorithm; and • Development of procedures to validate the algorithm. CHARGE TO THE COMMITTEE Responding to the request of the Food Safety and Inspection Service of the U.S. Department of Agriculture (USDA), the Institute of Medicine of the National Academies appointed the nine-member ad hoc Commit- tee on Review of the Use of Process Control Indicators in the FSIS Pub- lic Health Risk-Based Inspection System. Committee members provided expertise in meat and poultry microbiology, molecular biology methods, design and operation of processing establishments, risk analysis and de- cision-making tools, meat and poultry inspection, and foodborne disease epidemiology and public health. The committee met three times during the course of its work. The first meeting (Appendix A: Meeting Agen- das) was held on November 6-7, 2008, in conjunction with a public data- gathering session with FSIS representatives, who provided program background and an in-depth description of the committee’s task (Box 1). The committee’s second meeting on December 17-18, 2008, also in- cluded Dr. Artur Dubrawski, of Carnegie Mellon University, and Dr. Marc Huckabee and Dr. Curtis Travis, consultants to FSIS from Science Applications International Corporation, who conducted the statistical analysis. During an open session of that meeting, these invitees responded to the committee’s questions about the statistical analysis of the data on process control indicators that were used by FSIS to establish the proposed risk-based algorithm. In addition to discussions with FSIS representatives and consultants, the committee formally requested data and information from FSIS through the Freedom of Information Act, as suggested by FSIS representatives. The committee deliberated on the following process control indicators and the data analysis approaches used by FSIS to evaluate their potential inclusion in the algorithm:

LETTER REPORT 5 • Salmonella verification testing in raw meat and poultry • Pathogen testing in ready-to-eat (RTE) meat and poultry (Salmo- nella enterica, Listeria monocytogenes, and Escherichia coli O157:H7) and raw ground beef and its components (E. coli O157:H7) • Noncompliance records • Enforcement actions • Class I and II recalls • Pulsed field gel electrophoresis (PFGE) patterns of Salmonella serovars of particular human health concern for isolates derived from the raw meat and poultry Salmonella verification testing program • System for Tracking E. coli O157:H7 Positive Suppliers (STEPS) The committee also discussed the potential use of other indicators that were not included in the FSIS proposal. Findings and recommenda- tions were drafted. A third committee meeting was held on January 13- 14, 2009, in Washington, DC, to finalize its findings and recommenda- tions and to prepare the report for external review. The committee reviewed the data and statistical analysis (Appen- dixes D and E of the technical report Public Health Risk-Based Inspec- tion System for Processing and Slaughter [FSIS, 2008b]) provided for the proposed indicators listed above. Appendix D of that report includes a description of the data used; Appendix E describes the data analysis that was conducted and the conclusions derived thus far. Appendix D and E also include limitations of the data and analysis and the rationale for the design of the algorithm. At the request of FSIS and because another National Academy of Sciences (NAS) committee (Committee on Review of the Food Safety and Inspection Service [FSIS] Risk-Based Approach to Public Health Attribution) was assigned the task, data on volume and food attribution were not reviewed by this committee. FSIS noted that this algorithm would undergo improvements during the committee’s deliberation and, therefore, the proposal should be considered preliminary; since the publi- cation of its technical report, FSIS has slightly modified the selection of process indicators. The committee based its deliberations on the updated version of the algorithm that was presented at its meeting on November 6-7, 2008 (see the indicators of process control and levels of inspection

6 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS in Appendix B of this report). The committee additionally reviewed sev- eral other FSIS reports, such as the 2008 technical report on poultry slaughter provided to the National Advisory Committee on Meat and Poultry Inspection (NACMPI) (FSIS, 2008a), to better understand the evolution of the FSIS algorithm. This letter report begins with a background description of the FSIS initiative of a risk-based inspection system. Overall recommendations and findings are followed by recommendations for each specific process control indicator reviewed. The agenda of the workshop held on Novem- ber 6-7, 2008, and the agenda for the open session of the second meeting are provided in Appendix A. Appendix B lists the indicators of process control corresponding to each level of inspection. Appendixes C and D contain a list of acronyms and a glossary, respectively. Appendix E lists the committee members’ biosketches. BOX-1 Statement of Task An ad hoc committee will review whether the Food Safety and Inspection Service (FSIS) has adequately defined and identified indicators of process control to protect public health that will be used to rank establishments and allocate agency inspection resources. Specifically, the committee will evaluate how FSIS is proposing to use its available data to develop a relative risk rank- ing of establishments described in the technical report Public Health Risk- Based Inspection System for Processing and Slaughter, publicly posted at http://www.fsis.usda.gov/Regulations_&_Policies/National_Advisory_Committe e_on_Meat_&_Poultry/index.asp. BACKGROUND Public Health Risk-Based Inspection System for Processing and Slaughter The Food Safety and Inspection Service, the USDA agency respon- sible for ensuring the safety of meat, poultry, and egg products, has ex- amined a number of strategies to develop a risk-based food safety sys- tem. Examples include the development and implementation of the Pathogen Reduction; Hazard Analysis and Critical Control Point (PR/HACCP) Systems; Final Rule in 1996 (FSIS, 1996), the develop- ment of microbiological performance standards (FSIS, 1999), and re- quirements for pathogen testing of ready-to-eat foods (Requirements for specific classes of product. 2008. 9 CFR Part 430).

LETTER REPORT 7 In January 1997, President Clinton announced a Food Safety Initia- tive to reduce the incidence of foodborne disease in the United States. Among other changes, government agencies in charge of ensuring food safety were directed to improve inspections and enforce HACCP compli- ance in establishments that process meat and poultry (FDA-USDA-EPA- CDC, 1997). It was anticipated that implementation of the HACCP sys- tem would be accompanied by concurrent changes in inspection proce- dures. In 2003, the IOM Committee on Review of the Use of Scientific Criteria and Performance Standards for Safe Food found that the inspec- tion of FSIS-regulated establishments relied largely on visual and or- ganoleptic observations rather than on risk to public health (IOM, 2003). Although these are important and necessary elements of a plant survey, an improved, risk-based inspection system would assign levels of inspec- tion to establishments according to the magnitude of their product’s risk to the public’s health. Other organizations, including the National Acad- emies (NRC, 1987; IOM, 1990) and the Government Accountability Of- fice (GAO, 1992), have previously emphasized the need for a risk-based inspection system for meat and poultry products. In 2006, FSIS initiated the development of a risk-based inspection system. In its technical report Public Health Risk-Based Inspection Sys- tem for Processing and Slaughter (hereafter referred to as PHRBIS) (FSIS, 2008b), FSIS proposes a decision-making tool to rank establish- ments according to their risk to public health by categorizing them first acccording to their level of process control and then by the impact on public health of the food produced. In addition, FSIS intends to upgrade several other elements of the proposed inspection system. For example, FSIS plans to strengthen its information technology system to enable inspection personnel to enter data on hazard analysis and make subse- quent decisions in a more integrated and objective manner (FSIS, 2008b). FSIS also plans to train its inspection force (inspectors and su- pervisors) in effective use of the proposed inspection system tools. For example, in addition to continuing routine inspection training, a group of in-plant inspectors will receive training to enhance their understanding of establishment food safety systems, including HACCP plans or sanitary requirements. Supervisors will also be trained to use a more streamlined inspection review process (E. Dreyling, FSIS, personal communication, December 13, 2008). As FSIS describes in its technical report PHRBIS, the proposed tool has evolved with input from stakeholder groups as well as the USDA’s National Advisory Committee for Meat and Poultry Inspection. An im- portant innovation of this current proposal is to rely, where possible, on

8 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS data collected in conjunction with FSIS’s regulatory programs (FSIS, 2008b). The ultimate aim is the production of an effective tool for achieving the Healthy People 2010 goals of reducing foodborne disease caused by Salmonella, Escherichia coli O157:H7, and Listeria monocy- togenes (HHS, 2000). FSIS concludes that to protect public health most effectively, inspection resources have to be allocated based on the degree of risk to public health presented by each processing plant. Therefore, a key element of the risk-based inspection system is an algorithm for cate- gorizing slaughter and processing plants according to risk so that inspec- tion efforts are focused on those establishments having the greatest im- pact on public health (FSIS, 2008b). The algorithm consists of two consecutive steps to rank an establishment’s risk: a first component to determine the establishment’s level of process control (i.e., identifying attributes that indicate whether the establishment is maintaining control) and a second component to quantify public health impact (i.e., the vol- ume of the commodity produced at the establishment together with pub- lic health attribution of the food produced) (FSIS, 2008b). The commit- tee was charged with reviewing the scientific basis of and rationale for the first component of the algorithm—the data and data analysis that were used by FSIS to identify indicators for categorizing establishments according to their level of process control. A second NAS ad hoc com- mittee (Committee on Review of the Food Safety and Inspection Service [FSIS] Risk-Based Approach to Public Health Attribution) was charged with reviewing the second component, the public health attribution sys- tem. Because the two components are closely related (e.g., the volume of production in an establishment influences the sampling plans for patho- gen testing programs that FSIS proposes to use to indicate process con- trol) and included in an overall inspection system, this committee found it difficult to completely exclude deliberations on indicators of public health impact. OVERALL FINDINGS AND RECOMMENDATIONS This section of the report provides overall findings and recommenda- tions related to strengthening the proposed FSIS risk-based decision tools for ranking establishments. It is followed by a section that provides more specific recommendations for each proposed indicator. Prior to imple- menting the algorithm, the recommendations in this report should be fol- lowed.

LETTER REPORT 9 General Approach The committee concurs that a risk-based approach to inspection is essential and commends FSIS for undertaking such a daunting and con- troversial endeavor. The committee found that the development and use of a model (algorithm) to categorize establishments based on risk can ultimately be a systematic approach to realizing and implementing deci- sion criteria in a transparent, predictable manner. However, the commit- tee found it challenging to comprehend the framework, concepts, and rationale that FSIS applied in several segments of the proposed model. The descriptions of the algorithm, the scientific basis for the selection of the proposed process indicators, the analysis of data, and the use of the process control indicator algorithm as it is integrated in the overall in- spection system were not clearly stated in the technical report PHRBIS that was provided to the committee. For example, FSIS uses the term “algorithm” to describe its decision-making tool to categorize plants into levels of inspection. As shown in Table 1, there are various definitions of the term algorithm. However, in the context of a risk-based system, the term algorithm implies a mathematical model. Since FSIS did not construct a mathe- matical model, it would be more precise to use the designation decision tool or framework. To avoid confusion for the reader, the committee de- cided to retain the term algorithm for the purposes of this report. TABLE 1 Definitions of Algorithm Definition Source A set of rules for solving a problem in a finite number http://dictionary. of steps, (e.g., finding the greatest common divisor) reference.com/ A procedure for solving a mathematical problem (e.g., http://www.merriam- finding the greatest common divisor) in a finite num- webster.com/ ber of steps that frequently involves repetition of an operation; broadly: a step-by-step procedure for solv- ing a problem or accomplishing some end especially by computer A precise rule (or set of rules) specifying how to solve http://www.websters- some problem online-dictionary.org/ Continued

10 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS Table 1 continued Mathematics. A process, or set of rules, usually ex- Oxford English Dic- pressed in algebraic notation, now used especially in tionary, 2nd edition, computing, machine translation, and linguistics 1982 Medicine. A step-by-step procedure for reaching a clinical decision or diagnosis, often set out in the form of a flow chart, in which the answer to each question determines the next question to be asked Any special method of solving a certain kind of prob- Webster’s New World lem; specifically, the repetitive calculations used in Dictionary, 2nd col- finding the greatest common divisor of two numbers lege edition, 1982 Finding 1: Although the use of a model to categorize plants in levels of inspection is appropriate, the descriptions of the algorithm, the scientific basis for the use of the process indicators, the description and analysis of data, and the use of the process control indicator algorithm as it is inte- grated into the overall inspection system are not clearly articulated in the FSIS technical report. RECOMMENDATION 1: The committee recommends that in addition to the improvements in data collection and analysis presented below, FSIS revise its proposal to improve the transparency and clarity of the description of the overall inspection system—in particular, the process control indicator algorithm, its scientific basis, and the type and analysis of data used. Further, FSIS should consider tailoring the proposal to its target audiences (e.g., plant managers, FSIS inspectors and supervisors, FSIS managers and scientists, outside expert panels) and providing them with supplemental information or reports. Definitions of Process Control and Process Control Indicators The FSIS (2008b) report does not adequately define various terms that are key to evaluating the proposed inspection system (e.g., algo- rithm, process control, process control indicator). The ambiguous use of these terms hampered the ability of the committee to understand the use of data and could result in misinterpretations and unnecessary disputes in the future. To avoid confusing the reader and for the purposes of this re- port, however, the committee opted to retain the terms process control and process control indicators while also pointing out the ambiguity of their usage.

LETTER REPORT 11 The committee offers more clearly defined key terms and explains its interpretation of those terms for the purposes of this report. The concept of process control, which applies to all manufacturing companies and can be used broadly to address both quality and safety issues, is used in the context of the current report as a means to quantify how well an estab- lishment is employing control measures to minimize pathogen contami- nation. Examples of definitions of process control are shown in Table 2. TABLE 2 Definitions of Process Control Definition Source At certain points in the processing of a food, con- Scientific Criteria to En- trol measures can be applied to prevent an unac- sure Safe Food (IOM, ceptable increase in a hazard, eliminate it, or re- 2003, p. 94) duce it to an acceptable level Activities involved in ensuring a process is pre- BusinessDictionary.com dictable, stable, and consistently operating at the (http://www.business target level of performance with only normal dictionary.com/) variation The inspection of work-in-progress to provide bnet.com feedback on, and correct, a production process. (http://www.bnet.com/) First developed as a mechanical feedback mecha- nism, process control is now widely used to moni- tor and maintain the quality of output Method by which the input flow of processing Chemicals- plants is automatically controlled and regulated technology.com by various output sensor measurements. Process (http://www.chemicals- control can also describe the method of keeping technology.com/ processes within specified boundaries and mini- glossary/) mizing variation within a process The active changing of a process based on the NIST/SEMATECH results of process monitoring. Once the process e-Handbook of Statistical monitoring tools have detected an out-of-control Methods situation, the person responsible for the process (http://www.itl.nist.gov/ makes a change to bring the process back into div898/handbook/pmc/ control section1/pmc13.htm) The automated control of a process. Process con- PCMag.com trol is used extensively in oil refining, chemical (http://www.pcmag.com/ processing, electrical generation, and the food and encyclopedia_term/0,254 beverage industries where the creation of a prod- 2,t=process+control&i=49 uct is based on a continuous series of processes 753,00.asp) being applied to raw materials In its 2003 report Scientific Criteria to Ensure Safe Food, IOM evaluated the use of scientific criteria and standards in food safety regu- lations (IOM, 2003). That report defines various terms used in food

12 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS safety and refers to process control in various contexts. For example, the report refers to control measures as those measures that “can be applied at certain points in the processing of a food to prevent an unacceptable increase in a hazard, eliminate it, or reduce it to an acceptable level.” The report also states that “process control is based on four premises: (1) product quality or product safety must be built into the manufacturing process, (2) the manufacturing process must be monitored and the data must be analyzed using appropriate measurement and statistical tech- niques, (3) the process must be managed to ensure its variation remains stable and predictable, and (4) the process is capable of delivering prod- uct that meets the performance standard” (IOM, 2003). Manufacturing processes inherently possess some degree of variation that is acceptable and considered within the limits as long as it is predictable and stable, as described in Scientific Criteria to Ensure Safe Food (IOM, 2003). Thus, the concepts of validation and verification against one or more articu- lated performance metrics form an integral part of any process control system. The committee adopted the following definition of process control for the purpose of this review: A process is in control when, within the limits of a stable and pre- dictable process variation, all hazards are controlled to an accept- able level. This definition assumes that the process variation is known. It also assumes that there is active monitoring of the process using appropriate metrics, which ideally would allow corrective actions to be taken before a critical safety limit is surpassed. Using this general definition of process control, the committee de- fined a process control indicator for the purpose of this review as: A measurable attribute that indicates whether a process maintains or surpasses an acceptable degree of risk or hazard control. An adequate indicator is an attribute that can be measured with ob- jectivity and for which limits that indicate a need for corrective action can be established. It should be noted that such limits require considera- tion of both the scientific basis for the metric being employed and the societal considerations that were implicit in establishing the performance criterion used for decision making. In the proposed algorithm, FSIS utilizes the term “indicator of

LETTER REPORT 13 process control”; however, no definition is provided. Furthermore, FSIS’s selection of certain process control indicators that are based on a limit of detection or the single occurrence of a process deviation may reduce the primary strength of process control approaches (i.e., signaling the need to take corrective action before a critical limit is exceeded). An ideal indicator of process control is one that can predict future outcomes with some level of certainty. Such indicators allow establishments to take corrective actions before a loss of control represents a threat to public health, thereby advancing FSIS’s goal of reducing the number of high- risk establishments. In the absence of ideal indicators, it is acceptable to select others, as long as their limitations are fully recognized. Among the indicators of process control being proposed by FSIS as part of its algorithm are two basic types: those that may predict a future loss of control (e.g., exceeding a specific rate of NRs) and those that are outcomes of a past loss of control (e.g., finding a pathogen in an RTE food product, recall of a product for safety reasons). Although predictors of future loss of control are better indicators because triggering an in- depth inspection and corrective action could prevent future risk to public health, it is reasonable to conclude, in the absence of any contradictory information, that a plant that has produced contaminated products in the past may not have implemented adequate corrective actions and may therefore need a more comprehensive inspection program until its production process is shown to be in control. Such events are not true indicators of process control; rather, they demonstrate prior failures. This distinction is vital to understanding the algorithm, and therefore future improvements to it, and should be stated clearly along with the definition of process control indicator. Another fundamental limitation of the FSIS proposal is the fact that food process attributes inherently vary; the mere presence of an indicator organism could therefore simply reflect process variation within a threshold and not necessarily demonstrate that a process is out of control. During the open meeting discussions, FSIS staff acknowledged that none of the proposed indicators do, in fact, indicate that process control has been lost; instead, they alert FSIS that a more in-depth inspection is needed. This is a subtle but important distinction to disclose in order to avoid misinterpretations of a plant’s categorization. Indeed, the proposal does not adequately describe how the inspectors will address cases where an in-depth inspection reveals that the system is still under control and following regulatory requirements, but the process repeatedly fails in one indicator.

14 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS Process control indicators provide variable levels of predictability. Identifying and ranking process control indicators that are very different in nature may be challenging, but this should not preclude FSIS from implementing a risk-based inspection system. Statistical and risk-ranking analysis methodologies (e.g., multivariate analysis) can help in determin- ing the relative importance of different predictors by commodity, and these results should be taken into consideration when developing the al- gorithm. Other decision-making tools such as decision trees can help in categorizing plants at different levels of inspection. An evaluation plan to review the validity of the algorithm and to update it as new information becomes available is warranted. Finding 2: The committee finds that the technical report PHRBIS does not adequately define various terms that are key to a clear understanding of the proposed inspection system—specifically, process control and process control indicators. RECOMMENDATION 2: Prior to analyzing the available data and selecting the indicators to develop a risk-based inspection system, the committee recommends that FSIS clearly define the terms (and their limitations) that are critical to the development of the inspection system proposed in the technical report PHRBIS, such as algorithm, process con- trol, and process control indicator. FSIS should seek external advice from experts, especially on risk and risk-ranking, on the reliability and accu- racy of various attributes to predict public health hazards, but also from experts on the subject matter; this information should be used to evaluate the utility of potential indicators of process control. Further, FSIS should distinguish which indicators are suitable for different classes of meat and poultry products. Once a suitable decision-making tool (e.g., a decision tree) has been adopted, it should be validated for its purpose. Levels of Inspection The system proposed by FSIS (FSIS, 2008b) integrates nine process control indicators in a three-tier algorithm (system) that classifies proc- essing and slaughtering establishments into one of three levels of inspec- tion (LOI 1, 2, or 3), with LOI 3 representing the strictest level of inspec- tion (Appendix B). Subcategorization of LOI 1 and LOI 2 plants will be done according to their impact on public health (based on volume and food product public health attribution). Although the nature of the in- spections prescribed for the three different levels is not described explic-

LETTER REPORT 15 itly in the proposal, it was clear from discussions with FSIS representa- tives that the categories will be used to identify establishments to receive near-term for-cause Food Safety Assessments (FSAs), to prioritize rou- tine FSAs that are conducted in all establishments at least once every four years, and to schedule routine hazard assessment verification (HAV). 1 Those establishments designated LOI 1 or LOI 2 facilities will undergo routine inspection procedures or more in-depth inspections, such as more frequent FSAs than are routinely done (every four years) and HAV inspections. Those plants with a suspected loss of process control (those in LOI 3) will receive an immediate for-cause FSA (C. Travis, Science Applications International Corporation, personal communica- tion, December 13, 2008). Although the committee was not specifically asked to comment on the number or thoroughness of the levels of inspection proposed, evalua- tion of FSIS’s use of its available data as process control indicators to rank establishments required that the committee fully understand the de- tails of the concepts and procedures associated with the proposed algo- rithm and decision criteria leading to the various levels of inspection. The committee experienced some confusion over the use of three levels of inspection, specifically the inclusion of an intermediate level, LOI 2, in which an establishment appears to be considered neither in nor out of control. It is the understanding of the committee that LOI 2 is reserved for establishments that have recently been classified as LOI 3 but are implementing corrective actions, clearing an enforcement action, or be- ing inspected through an HAV or FSA. It was not clear to the committee for what length of time or how frequently an LOI 2 establishment will be subject to HAV inspection, or whether this will be decided on a case-by- case basis or by using a decision-making framework. For example, if a slaughtering establishment has failed the Salmonella verification testing percentile cut point, Salmonella verification testing results must remain below that cut point for at least 120 days for an establishment to be re- classified as LOI 1 (FSIS, 2008b). In that case, the committee questions what the frequency of HAV inspections would be, who would make de- cisions about the course of action to take, which process steps would be inspected, and the rationale for the length of time before the plant is eli- gible to be reclassified as LOI 1 (120 days). Furthermore, it is unclear 1 HAV is a proposed inspection activity in which FSIS in-plant inspectors review certain com- ponents of the facility’s process controls (e.g., HACCP monitoring and verification activities). HAV is considered an intensified routine inspection activity to be conducted by specifically trained in- plant inspectors.

16 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS how these level-of-inspection designations relate to the current regula- tory requirements for Salmonella verification testing associated with dif- ferent product classes. For example, for ground turkey, 15 positive sam- ples are allowed in a single 53-day window, whereas beef carcasses can have only one positive sample in a single 83-day window. In both cases (a ground turkey processor having up to 15 positive samples in a single 53-day window and a beef processor having one positive sample in a single 83-day window), the establishments would be in compliance with FSIS process control regulations, and it would be difficult to classify them as being out of control. Similarly, if a Class I recall has occurred during the preceding 120 days and the affected plant receives an HAV inspection, the committee questions by whom and when the adequacy of the process will be confirmed and the decision made to reclassify the establishment to LOI 1. Finding 3: Specific procedures assigned to the three levels of inspection are not clearly described. RECOMMENDATION 3: The committee recommends that FSIS clearly describe in its proposal the nature of the inspection for each dif- ferent level, the decision-making process that would result in a change in inspection level, and the relationship between level-of-inspection desig- nation and the state of process control as specified by current FSIS regu- lations. The FSIS personnel responsible for making such decisions and their expertise should also be designated in the proposal. If the system is completely automatic (e.g., input from an inspector automatically results in a specific LOI decision, involving no subjective judgment on the part of the inspector), the committee recommends that studies be carried out to ensure that the model includes all possible scenarios. Data Collection and Analysis of Proposed and New Indicators of Process Control FSIS used various statistical analyses to correlate proposed process control indicators with recognized process control indicators now in use (e.g., results of Salmonella verification testing) or with other proposed indicators that, based on FSIS regulatory definitions, record the presence of adulterants and therefore imply a failure of control in the system. Based on the data presented, except for the Salmonella verification test- ing results and NRs, the proposed indicators of process control measure a problem that has occurred in the past (they demonstrate an outcome), but

LETTER REPORT 17 they are not statistically associated with other process control indicators (e.g., Salmonella verification testing results). Furthermore, the design of the data collection and analysis to justify the selection and use of the proposed indicators was not based on any specific definition of process control and process control indicators. Some current predictors of process control may no longer be useful in the future. For example, the tendency and purpose of Salmonella veri- fication testing is reduction in the frequency of contamination of raw meat and poultry products with Salmonella (i.e., fewer Salmonella- positive samples will be found over time). If the standard is successful, then Salmonella positives might become so infrequent in the future that the test will lose its utility as a predictor of process control. The committee commends the effort to develop a data-driven risk- based inspection system and provides some comments for consideration to improve future acquisition or analysis of the data. Collection of Data Given the importance of establishing risk-based inspections to the overall effectiveness of FSIS programs, the identification of process con- trol indicators to categorize establishments based on their risk to public health would best be achieved through a data collection approach spe- cifically designed for that purpose. FSIS has used lift analysis, a data- mining method that determines associations between two variables that occur separately in time (i.e., predictability), to identify indicators of process control (see below). While this is a defensible approach, the sys- tem could be improved substantially if a more complete statistical analy- sis was performed and additional data for proposed or new predictors were collected. Also, the proposed algorithm does not currently include an adequate process control indicator for some products (e.g., RTE foods). Microorganisms that are more likely to be found in the environ- ment or the product, such as generic Escherichia coli, may be a better indicator of process control than a microorganism that is normally pre- sent only in low numbers. Information on process control deviations col- lected by inspectors should be tested to determine their usability as pre- dictors of process control; similarly, the FSIS-Agricultural Research Service (ARS) study on poultry slaughter (Technical Report on Im- provements for Poultry Slaughter Inspection, including Appendix H, “Data Analyses Supporting Proposed Performance Standards”) (FSIS, 2008a) should continue and be expanded.

18 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS The committee finds that retrieving data already collected by indus- try or others could help identify other potential indicators of process con- trol. For example, data on generic E. coli that are collected by industry should be analyzed to determine their usefulness in predicting process control. The committee recognizes that there are challenges in acquiring data from industry or others, but it encourages FSIS to act promptly to collect these data and analyze their potential as indicators of process con- trol. At least in the case of poultry slaughter, the FSIS-ARS study sug- gests that FSIS found enough evidence to use generic E. coli as an addi- tional performance standard for process control (FSIS, 2008a). It is also worth noting that the study employed many of the data analysis ap- proaches that are recommended below. For the indicators proposed in FSIS’s technical report (FSIS, 2008b) and potential new indicators, detailed findings and recommendations for data collection and analysis are provided in the following sections. Use of Lift Analysis Lift analysis is a data-mining tool that can identify associations be- tween two variables separated in time. It is the central statistical analysis used by FSIS to estimate the predictability of loss of process control and is explained in detail in Appendix E of the technical report PHRBIS (FSIS, 2008b). Lift is a measure of how much prediction results are im- proved by use of a model over those obtained by chance. Lift indicates how well the model improved the predictions over a random selection, given actual results, and allows a user to infer how a model will perform on new data. It works by converting input and output data accumulated over time into binary streams. Lift analysis is generally considered a relatively imprecise, prelimi- nary data-mining tool that would typically be followed up with more rig- orous evaluation. However, it is appropriate for the analysis of some data sets. For example, it works well as a metric of increased risk, the purpose of the process indicators analyzed here. The use of lift analysis also con- forms to the predictive nature of the modeling task for at least some of the predictors that were evaluated. Multiple combinations of evidence and outcome window sizes were used to empirically select promising configurations. FSIS indicated that the time windows (the length of time separating the two variables) selected were multiples of seven days, to eliminate the strong day-of-the-week effect observed in data. However, this may also have led to a misinterpretation of the data for certain tests as discussed below. Although other alternatives (e.g., logistic regression)

LETTER REPORT 19 could have been employed, this simplified statistical analysis appears useful in the context of the low frequency of occurrence of several of the attributes that FSIS proposes to use for its algorithm. Lift analysis seems to tolerate rare event data better than the alternative, more sophisticated, univariate regression analysis, which might not detect a correlation (even if it existed) due to the rare occurrence of the outcome. FSIS conducted lift analyses forward; that is, they were designed to detect the likelihood that a future outcome would occur, given the occurrence of a particular event in the past. For example, lift analysis was used to evaluate the abil- ity of NRs to predict that an establishment producing raw meat or poultry would fail Salmonella verification testing, which in turn is used by FSIS to determine whether an operation is out of process control. When suffi- cient data were available, FSIS conducted regression analyses among variables. Because of the rare occurrence of some of the proposed indicators, the committee supports the use of lift analysis described in Appendix E of PHRBIS (FSIS, 2008b) for initial identification of relationships. The analysis aggregates data from all establishments to increase statistical power. For example, the Salmonella verification data were aggregated across establishments for the analysis. Aggregating data is a valid ap- proach as an initial assessment of raw data. However, it may produce a biased estimate of association, so further confirmatory analysis with more sophisticated statistical tools is warranted. In conducting the lift analysis, it would seem particularly important to ensure that associations between attributes that are predictive be used in a manner that is consistent with their current use. It is not surprising that the Salmonella verification testing program was among the most effective indicators of process control examined by FSIS, since it is one of the few metrics evaluated that was specifically designed as a process control indicator (i.e., control of fecal contamination). However, what is not adequately stated in the report is that the verification testing program does not regard Salmonella as an adulterant in these products. This seems to have led to substantial confusion in the report, particularly with regard to the time window that should be employed between the issuance of an NR and when the presence of a positive Salmonella finding is indicative of a loss of process control. In the Salmonella verification testing pro- gram, each raw meat and poultry product has its own unique criterion for the level of control deemed acceptable. This led to the committee’s being unclear about the basis of the lift analyses for these commodities. The technical report PHRBIS (FSIS, 2008b) indicates that the lift analysis was based on the occurrence of the first positive Salmonella sample dur-

20 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS ing various 7- to 84-day windows following the occurrence of an NR. However, the presence of a single positive sample is not consistent with the way the Salmonella verification testing program is conducted—it examines whether the number of Salmonella-positive samples in the specified moving window exceeds the current standard for that specific commodity. Thus, it would not be surprising to see a positive sample after an NR for ground turkey, a product class with a baseline frequency of Salmonella occurrence of approximately 50 percent; however, this would not be indicative of a process that is out of control. Conversely, one would not expect to see a positive Salmonella sample from a beef carcass, a commodity that has a baseline frequency of 1 percent; in this case, the occurrence of a positive result might be predictive of a loss of process control. It is also worth noting that the original lift analyses did not take into account the variance in the Salmonella verification testing regime or the confidence levels associated with this microbiological cri- terion. The current performance standards for Salmonella verification testing in raw products are based on an 80 percent confidence interval, that is, a 20 percent probability that failing a Salmonella verification set occurred by chance; therefore, even if an attribute correlated with a fail- ure to meet the Salmonella process control standard, that relationship may have occurred by chance alone. Not until an establishment has failed three consecutive Salmonella verification testing sets is it considered, from a regulatory standpoint, to be out of process control. Finding 4: Lift analysis is a data-mining tool that is appropriate to use for finding initial associations among events that occur infrequently. However, the identification of process control indicators requires more complex statistical analysis as well as data that have been collected for the purpose of identifying such indicators. The committee emphasizes that although mining or extracting data from currently existing data sets to design the inspection system is commendable and the use of lift statis- tics for data mining is justified, the system could be significantly im- proved if more complete statistical analyses were conducted in addition to the lift analysis and if additional data were collected for more useful predictors. Also, the proposed algorithm does not currently include an adequate process control indicator for some foods (e.g., RTE foods). RECOMMENDATION 4: The committee recommends that FSIS per- form further statistical analysis for the purpose of validating proposed indicators of process control as well as exploring the utility of new proc- ess control indicators through new studies, expert consultation, and lit-

LETTER REPORT 21 erature review. In some instances, FSIS should take advantage of data for other potential process indicators generated by industry or others. After a preliminary association with an outcome is established (predictability is demonstrated statistically), FSIS should conduct further analysis to con- firm the utility of product-based process indicators and ultimately con- clude the analysis with a multivariate model or similar method. FSIS should then modify the algorithm as new predictors are identified and test the adequacy of its current (and future) algorithm. Microbiological Testing Several of the process control indicators included in the proposed al- gorithm are based on the results of microbiological testing programs. As discussed above, Salmonella verification testing appears to be particu- larly well suited as a predictor of loss of process control and an alert to take corrective actions prior to exceeding a public health limit. However, its application within the algorithm needs to be consistent with its current use in the FSIS regulatory framework and should take into account the statistical characteristics associated with sampling programs and the op- erational assumptions made when establishing microbiological perform- ance standards. For the sake of transparency and to confirm the scientific basis of an attribute that is being used to categorize an establishment based on risk, FSIS should ensure that it has fully articulated the statisti- cal operating characteristics of its Salmonella verification testing when it is being used in a framework other than the current regulatory frame- work. FSIS should also identify, to the degree feasible, the sources of type I and type II errors associated with the testing regime and the rela- tive sensitivities of the analytical methods. There is also a need through- out the technical report and in its accompanying analysis to carefully dif- ferentiate the assumptions in the Salmonella verification testing program, where it serves as an indicator of control of fecal contamination in raw products, from its detection during the testing of RTE foods, where it is considered a pathogen whose presence indicates that the process has al- ready failed. In addition to the Salmonella verification testing of raw products, the results of microbiological testing of ground beef for E. coli O157:H7 and RTE foods for L. monocytogenes, Salmonella, and E. coli O157:H7 were examined as potential indicators of process control. For these microbi- ological testing activities, detection of a positive sample is considered by definition an indication of loss of process control (i.e., there is “zero tol- erance”). However, these testing programs are based on an evaluation of

22 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS individual lots of product and are not specifically designed to measure process control (ICMSF, 2002). The decision criteria for the assignment of establishments based on these testing results have operationalized the lot-by-lot sampling program by effectively assuming there are no type II errors (false positive results). While this is a practical risk-management approach for the implementation of regulatory programs, its application in the algorithm suggests that FSIS has not fully considered the general concepts underlying process control indicators and the statistical basis for microbiological testing. This could be corrected by FSIS’s articulat- ing its underlying assumptions regarding the interpretation of microbi- ological testing programs. The FSIS technical report could benefit greatly from a more in-depth consideration of the statistics that underlie microbiological testing and the assumptions made in using such data as decision criteria in the FSIS algorithm. For greater transparency of the statistical basis for FSIS’s interpreta- tion of the testing programs in which detection of a microorganism is assumed to represent a loss of process control, FSIS should include a discussion of type I errors (the incidence of false negative results). As pointed out by the International Commission on Microbiological Specifi- cations for Foods (ICMSF, 2002), the probability of detecting a pathogen in a food depends on the concentration (mean and variance) of the patho- gen in the food, the assumed distribution, the number and size of the samples examined, and the sensitivity of the analytical method. Thus, the absence of a positive result does not necessarily indicate that a food is free of the pathogen of concern and that the process is therefore in con- trol. For example, the current protocol of examining a 25-g sample effec- tively provides 95 percent confidence that a contaminated lot would be detected if the mean concentration is 1 CFU (colony forming unit)/3 g. However, if a ground beef lot had an E. coli O157:H7 concentration of approximately 3 CFU/100 g, there is 50 percent likelihood that the con- tamination would not be detected using the current protocol for this pathogen. Transparency in the role of microbiological sampling pro- grams as process control metrics requires information on the confidence that a positive sample will be detected, based on the mean concentration and standard deviation that were assumed in designing the sampling pro- tocols included in the report PHRBIS. Finding 5: FSIS currently tests different classes of products for different microorganisms, for different purposes, and with different underlying assumptions. The applicability of these data to the FSIS algorithm is de- pendent on the specific protocols, assumptions, and statistical character-

LETTER REPORT 23 istics of each testing program. The FSIS technical report did not provide in-depth consideration of the statistics that underlie the specific microbi- ological testing protocols used and the assumptions made when using such data (e.g., the magnitude of type I and type II errors, assumed pathogen concentration means and standard deviations, confidence in- tervals, the specificity and sensitivity of the microbiological protocols). RECOMMENDATION 5: The FSIS technical report should describe the characteristics of the microbiological criteria being used as determi- nants of loss of process control. These characteristics include in-depth consideration of the statistics underlying the specific microbiological testing protocols used and the assumptions that are made in using such data (e.g., the magnitude of type I and type II errors; assumed pathogen concentration means and standard deviations). As recommended in the following sections, FSIS should also consider investing in research to find and validate alternative microbiological indicator tests whose target microorganism occurs at a substantially greater frequency than those cur- rently in use. If successful, this would provide FSIS with a better process control indicator that could be used to analyze trends and to take actions (e.g., perform an in-depth inspection) before public health limits are ex- ceeded. FINDINGS AND RECOMMENDATIONS ON SPECIFIC PROCESS INDICATORS Salmonella Testing Results Use and Scientific Evidence In accordance with the PR/HACCP rule, FSIS has set Salmonella testing standards to be met by establishments producing certain raw products. As introduced above, the standards were derived from national estimates of prevalence by product and were calculated so that an estab- lishment operating at the national baseline Salmonella prevalence has an 80 percent probability of meeting the standard. To assess compliance with the standards, FSIS implemented the Salmonella verification testing program in 1998, in which establishments are monitored by testing sam- pling sets for Salmonella at a specific frequency and comparing them with performance criteria based on product class (FSIS, 2008b). The time

24 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS required to complete a sampling set ranges from two months to one year. FSIS monitors eight classes of raw meat and poultry products, and test data are available for about 80 percent of establishments. The sampling protocol is described in Appendix D of the technical report PHRBIS (FSIS, 2008b). In the current Salmonella verification program, FSIS categorizes establishments producing those eight product classes into three categories by comparing their Salmonella verification results to the Salmonella prevalence rates within each class of product: • Category I—Establishment achieved Salmonella prevalence rates <50 percent of the performance standard (based on the na- tional estimate baseline for a given product) in the two most re- cent Salmonella sets. • Category II—Combinations of results for two most recent sets do not qualify as Category I, but establishment has not failed the most recent Salmonella set. • Category III—Establishment failed most recent Salmonella set. Table 3 shows the national prevalence rates, number of samples per set, and number of positives to be categorized as Category I, II, or III. Based on 2006 results, only 3 percent of establishments were in Category III.

LETTER REPORT 25 TABLE 3 Cut Points of Set Results Defining Salmonella Verification Categories by Product Class Number Number of Positives Relative to Baseline of Sam- Standard Prevalence ples per Product (%) Set ≤50% >50% Exceeds Steers, 1.0 82 0 1 2 or more heifers Cows, bulls 2.7 58 1 or fewer 2 3 or more Ground 7.5 53 3 or fewer 4-5 6 or more beef Market 8.7 55 3 or fewer 4-6 7 or more hogs Broilers 20.0 51 6 or fewer 7-12 13 or more Ground 44.6 53 13 or 14-26 27 or more chicken fewer Ground 49.9 53 15 or 16-29 30 or more turkey fewer Young 19.6 56 7 or fewer 8-13 14 or more turkeys SOURCE: Pathogen Reduction; Hazard Analysis and Critical Control Point (PR/HACCP) Systems; Final Rule, Section 310.25 (b) 2 (meat), Section 381.94 (b) 2 (poultry). Although a decrease in the incidence of foodborne infections is often cited as evidence that the Salmonella verification testing program is an effective process control indicator, various factors may confound the value of that association. The ability to relate these results to improve- ment in public health is limited by the lack of a good food attribution model that can directly connect cases of salmonellosis to specific food products. However, in the absence of direct measures of attributable pub- lic health outcomes, the data available on the exposure of the public to raw meat and poultry products containing Salmonella provide a reason- able measure of the relative risk reduction associated with those prod- ucts. One factor limiting the utility of the data is that although FSIS has published Salmonella results since 1996, because of changes in sampling designs and the segments of the industry being reviewed, data are not always comparable from year to year. Progress was clearly evident when FSIS tracked the percentage of Salmonella positives in verification sam-

26 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS ples by product class. For example, in 2006, 100 percent of the sample sets submitted by ground turkey establishments, 88.6 percent of those from broiler establishments, 94.5 percent of those from market hog es- tablishments, 91.2 percent of those from cow or bull establishments, and 93.9 percent of those from steer or heifer establishments passed the Sal- monella standard (Category I or II). By the end of 2007, 100 percent of ground turkey establishments, 97.2 percent of broiler establishments, 97.3 percent of market hog establishments, 95.2 percent of cow or bull establishments, and 94.7 percent of steer or heifer establishments passed the standard (FSIS, 2008c). The FSIS proposes the use of establishment categorization based on Salmonella testing as an indicator of process control (FSIS, 2008b; E. Dreyling, FSIS, personal communication, February 18, 2009). The level of inspection to which an establishment would be assigned would depend on its Salmonella testing results: • LOI 1: Establishment was below the Salmonella percent positive LOI 1 percentile cut point on most recent sample set, unan- nounced sampling, or other Salmonella testing programs • LOI 2: Establishment was above the Salmonella percent positive LOI 1 percentile cut point on most recent sample set, unan- nounced sampling, or other Salmonella testing programs and not in Salmonella Category III • LOI 3: Establishment is in Salmonella Category III FSIS proposes to determine the percentile cut point for the three lev- els by analyzing the number of Salmonella verification positive results for each specific class of product (e.g., broilers, ground beef) and finding the inflection points in the curve representing the number of establish- ments versus the rate of positive Salmonella verification testing results over a period of three months. An example given by FSIS for ground beef is shown in Figure 1.

LETTER REPORT 27 40 Salmonella Cut Point for Ground Beef 35 30 % positive 25 Salmonella 20 verification 15 10 LOI1 Cut Point 5 0 1 47 93 139 185 231 277 323 369 415 461 507 553 599 645 691 737 783 829 Number of Plants FIGURE 1 Inflection points in the Salmonella verification testing data used to determine cut points in the proposed algorithm for ground beef. SOURCE: E. Dreyling, FSIS, personal communication, December 13, 2008. Committee’s Discussion Salmonella verification testing was designed to be a process control indicator in raw products, so its use in the algorithm is appropriate, as long as positive Salmonella test results occur with sufficient frequency and at high enough levels. As the prevalence of Salmonella decreases, alternatives to Salmonella testing should be sought. Also, the number of products for which Salmonella testing is an indicator (i.e., there are cur- rently only eight raw meat and poultry product classes in the program) is appropriately limited, so alternative indicators based on an objective measurement (e.g., other microbiological testing approaches) will have to be identified for RTE foods. The current use of discrete testing sets decreases the overall power of the testing program as an indicator of process control compared to the daily testing required by the generic E. coli testing program. As mentioned above, there does appear to be a fundamental problem in the way the association of these data with other indicators was evalu- ated using lift analysis. FSIS would also benefit from an analysis of how the results of the first Salmonella verification testing set relate to the po- tential use of this metric as an enforcement tool (i.e., the failure of three Salmonella verification testing sets). Additional explanations related to

28 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS the method for differentiating Category I from Category II should be provided—for example, whether the 50 percent frequency cut point is better determined by halving the number of positive samples or by short- ening the length of the sampling window. FSIS also needs to provide a better explanation of the utility of these data as an indicator of process control when the frequency of Salmonella detection falls below ap- proximately 10 percent. Alternatively, FSIS should explore whether the Salmonella verification testing data would provide greater discriminatory power if they were quantitative instead of qualitative. Finding 6: The use of Salmonella verification testing results to rank es- tablishments in different levels of inspection is justified, but could be en- hanced by additional explanation and characterization. RECOMMENDATION 6: The committee recommends that FSIS pro- vide a more detailed analysis of how it will employ the results of the Salmonella verification testing program, including a consideration of the underlying statistics of its application. FSIS would also benefit from the following data collection and research activities to alleviate some of the limitations of Salmonella verification testing as an indicator of process control: • Sponsor research programs to develop and validate faster, quan- titative testing methodologies for Salmonella. Inclusion of newer, molecular-based methods for typing and subtyping Sal- monella isolates may also help distinguish the underlying rea- sons for loss of process control (see also below). • Continue to develop a process- and commodity-specific national baseline for Salmonella levels to verify the effectiveness of FSIS efforts to ensure food safety. • Collect data on Salmonella serotypes by raw product and at dif- ferent steps throughout the process, including the incoming step. Salmonella serotype data could help determine whether a loss of control has occurred within an establishment. It could provide evidence of the source of contamination (by traceback investiga- tions) and a contamination pattern in an establishment. In addi- tion to their potential value in foodborne disease attribution (i.e., determining which products are more likely to be associated with foodborne disease), FSIS should evaluate the use of Salmonella serotype data as a potential indicator of process control.

LETTER REPORT 29 • Explore the use of prevalence and load of Salmonella in the in- coming raw material as an indicator of process control. • FSIS should provide a more in-depth description of the sampling and testing statistics that are the basis for the Salmonella verifi- cation testing program, as well as how these characteristics and assumptions influence the use and interpretation of the data for categorizing establishments. This should include consideration of the magnitude of type I and type II errors, assumed pathogen concentration means and standard deviations, specificity and sensitivity of the microbiological protocols, and so forth. Listeria monocytogenes Testing Results for RTE Products Use and Scientific Evidence FSIS considers the presence of a pathogen in a ready-to-eat food product an indicator of a public health risk; therefore by definition, it indicates a potential loss of process control. The levels of inspection pro- posed by FSIS relating to Listeria monocytogenes (Dreyling, 2008; FSIS, 2008b; E. Dreyling, FSIS, personal communication, February 18, 2009) are as follows: • LOI 1: Establishment has not had a positive FSIS test result for L. monocytogenes in RTE products or a positive L. monocyto- genes food contact surface sample; or if it has, any related FSA and follow-up sampling has been completed more than 120 days previously, all related enforcement actions are closed, and estab- lishment meets all other LOI 1 criteria. • LOI 2: For an establishment that has had an FSIS positive L. monocytogenes test result in an RTE product or an L. monocyto- genes-positive food contact surface sample, any related FSA and follow-up sampling has been completed in the previous 120 days and all related enforcement actions are deferred or in abeyance. • LOI 3: Establishment has had an FSIS positive L. monocyto- genes test result in an RTE product or an L. monocytogenes- positive food contact surface sample.

30 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS The relative rates of L. monocytogenes infection in 2007 (CDC, 2008) were 42 percent lower than in 1996-1998, which might be inter- preted as a success for plant hygiene and sanitation controls and regula- tions. However, there was no change in the 2007 incidence compared to rates reported in 2004-2006. There were continuing reductions in the rate of L. monocytogenes contamination in RTE meat products during 1990- 2007 (FSIS, 2008d). Any finding of this pathogen in an RTE food immediately places the establishment into LOI 3, requiring an in-depth inspection (FSIS, 2008b). As part of its RTE regulatory sampling, FSIS has four L. monocytogenes testing programs: ALLRTE, RTE001, Routine L. monocytogenes (RLm), and Intensified Verification Testing (IVT). Under the ALLRTE program, inspectors sample products at random (except for products that do not support the growth of L. monocytogenes, such as fats and oils, dried soup mixes, and popped pork skins). RTE001 is a risk-based program in which establishments are selected for testing based on risk factors identi- fied in FSIS’s L. monocytogenes risk assessment. Products identified as presenting a higher risk are sampled more often than products considered to be less risky. Four variables are used to determine relative risk: prod- uct type; production volume; alternative 1, 2, or 3 and the processing plant’s history of L. monocytogenes testing results (E. Dreyling, FSIS, personal communication, December 13, 2008). Committee’s Discussion FSIS considers the presence of L. monocytogenes in a finished prod- uct or on a food contact surface after the posttreatment process an indica- tion of a loss of process control. Comments about this basic assumption and others related to the use of microbiological testing as a means of verifying process control have been made earlier in the section on micro- biological testing. As indicated above, this metric is primarily an out- come but is being used as a predictor of loss of process control in the proposed FSIS algorithm. Furthermore, as observed in the lift analysis, the low incidence of positive L. monocytogenes samples makes it diffi- cult to establish a significant association with other potential predictors of loss of process control (see FSIS, 2008b; Appendix E). FSIS’s deci- sion, based on public health concerns, to operationalize its testing pro- grams to identify any isolation of L. monocytogenes from RTE foods as a loss of process control is a valid risk-management decision; however, FSIS should fully explore the rationale for and impact of that decision in terms of the achieved risk reduction and the assignment of establish-

LETTER REPORT 31 ments to risk categories. It is also worth noting that several risk assess- ments of RTE foods have indicated that the risk of listeriosis in such products is highly dependent on whether the product supports the growth of L. monocytogenes (WHO-FAO, 2004). FSIS has not indicated how this risk factor was considered in the designation of establishments based on the results of the L. monocytogenes testing programs or what percent- age of RTE food products whose positive tests resulted in source estab- lishments being categorized as LOI 3 were foods that supported growth of the pathogen. As with other pathogens, not all meat and poultry product processors are tested for L. monocytogenes, only those producing RTE foods (FSIS, 2008d). The incidence of listeriosis decreased during the decade between 1996-1998 and 2007 (CDC, 2009), presumably due in part to the various preventive controls applied to meat and poultry processing plants. The current low frequency of events (for example, only 0.37 percent of 2,963 ALLRTE samples in 2007 tested positive for L. monocytogenes; E. Dreyling, FSIS, personal communication, December 13, 2008) presents a challenge to using the presence of a pathogen as an indicator of process control, since there may be no detectable correlation with loss of process control because of the low number of positives. As mentioned above, the testing protocol for L. monocytogenes is restricted in sample size and frequency, limiting the ability to directly relate the presence of the pathogen to ongoing processing—that is, absence of the pathogen does not necessarily indicate that the process is in control, and vice versa. Finding 7: The use of L. monocytogenes testing results in RTE foods to rank establishments in different levels of inspection has been justified based on the potential direct health risk of the pathogen, a valid risk- management decision criterion, particularly for specific production lots. However, the initial data analysis has not provided scientific support for use of this decision criterion to predict loss of process control or for its association with other indicators. As previously mentioned, FSIS has not discussed the sampling and related statistics that should be considered when using any microbiological sampling program to verify process con- trol. The FSIS algorithm does not consider that all RTE products do not present the same level of risk to public health. RECOMMENDATION 7: Given the limitations of the use of L. mono- cytogenes testing results as a process indicator, the committee recom- mends that FSIS do the following:

32 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS • Consider redesigning the testing protocols by prioritizing inspec- tion of RTE products according to product risk, that is, with con- sideration of a product’s ability to support the growth of L. monocytogenes (e.g., the food’s acidity level, the use of pre- servatives). This risk-based approach is being adopted by others (e.g., the Codex Alimentarius Commission) and merits consid- eration by FSIS. • Consider analyzing industry data on L. monocytogenes or Lis- teria spp. in the environment and/or Listeria spp. on food contact surfaces or in the final product to determine whether these data could serve as a useful indicator of process control. For example, FSIS could use data collected more frequently for routine sam- pling of processing environments that may be reservoirs of L. monocytogenes. Although unpublished results of PFGE analyses of L. monocytogenes isolates from samples taken from 127 plants suggested that contamination of product or contact sur- faces did not originate in the plant environment (E. Dreyling, FSIS, personal communication, December 13, 2008), further analysis is needed to confirm or refute this finding due to the small sample size. This is particularly important when consider- ing that the scientific literature is replete with examples suggest- ing that controlling harborage sites in the processing environ- ment is critical to managing this foodborne pathogen (Giovannacci et al., 1999; Lundén et al., 2003; Peccio et al., 2003; Thévenot et el., 2006; Keto-Timonen et al., 2007). • Conduct lift analysis and other appropriate analyses by product class to determine whether there is a correlation between L. monocytogenes and specific NRs for products with inherently high public health risks. This will allow comparisons with statis- tical analyses already conducted for all products. • Sponsor research programs to develop and validate improved and quantitative testing methodologies for L. monocytogenes as a potential means of increasing the discriminatory power of this indicator. Improved sampling and testing methods might in- crease the confidence in the methodologies and decrease the number of false positives and false negatives. Rapid methodolo- gies will decrease the temporal gap between the loss of control and the inspection, and therefore it is more likely that the associ- ated problem (and the solutions) could be found. These kinds of improvements will enhance the reliability of the algorithm.

LETTER REPORT 33 • FSIS should provide a more in-depth description of the sampling and testing statistics that are the basis for L. monocytogenes regulatory testing programs, as well as how the characteristics and assumptions of the sampling and testing statistics influence the use and interpretation of these data for categorizing estab- lishments based on this metric. This should include consideration of the magnitude of type I and type II errors, assumed pathogen concentration means and standard deviations, specificity and sensitivity of the microbiological protocols, and so forth. E. coli O157:H7 Testing Results in Raw Ground Beef or Its Components Use and Scientific Evidence The testing programs for E. coli O157:H7 are targeted primarily at raw ground beef and, more recently, the trim used in this raw product. Testing of trim and/or finished product is used extensively by industry as a control measure for diverting contaminated meat to other uses that have a lethal treatment step. The following criteria for E. coli O157:H7 have been proposed by FSIS (Dreyling, 2008; FSIS, 2008b; E. Dreyling, FSIS, personal communication, February 18, 2009) to define the level of in- spection categories: • LOI 1: Establishment has not had a positive FSIS E. coli O157:H7 verification result; or if it has, any related FSA and fol- low-up sampling has been completed more than 120 days previ- ously, any related enforcement actions are closed, and establish- ment meets all other criteria for LOI 1. • LOI 2: Establishment had an FSIS positive test for E. coli O157:H7 in RTE products or ground beef or components, and any related FSA and follow-up sampling has been completed in the previous 120 days and all related enforcement actions are de- ferred or in abeyance. • LOI 3: Establishment has had an FSIS positive test for E. coli O157:H7 in RTE products or ground beef or components. As mentioned above, major declines in the incidence of certain food- borne diseases occurred between 1996 and 2004, including Shiga toxin-

34 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS producing E. coli O157 (STEC O157) (CDC, 2008). For STEC O157, there has been no significant decline in cases since 2004, despite inter- ventions to reduce ground beef contamination (CDC, 2008). No statistical analysis was performed by FSIS to correlate the pres- ence of the pathogen with process control. Lift statistics were conducted by FSIS between E. coli O157:H7 and consumer complaints, NRs, re- calls, and enforcement actions. The only suggestive results for predictors of the presence of E. coli were obtained for NRs and enforcement actions restricted to a 14-day time window (see FSIS, 2008b; Appendix E). In the case of ground beef, E. coli O157:H7 is by regulation consid- ered a pathogen even if the product is intended to be further processed or cooked by the consumer. FSIS’s rationale for including this pathogen in the algorithm is that any level of E. coli O157:H7 presents a high risk to the public. Committee’s Discussion The limitations of this indicator are similar to those that apply to the use of other pathogens as indicators of process control. Prior comments on the interpretation of microbiological results for pathogens with a zero tolerance as defined by a standardized testing program are valid for the testing of raw ground beef for E. coli O157:H7. While the occasional detection of a positive sample cannot definitively establish a loss of process control, the use of testing results for E. coli O157:H7 for this purpose is a valid risk-management decision to operationalize the deci- sion criterion for the testing program and to safeguard the public from a pathogen that is capable of infecting individuals at a low dose. However, FSIS should be prepared for instances in which the subsequent FSA find- ings cannot document a loss of process control, particularly given the potential for E. coli O157:H7 to be spread in grinding operations. The low frequency of contamination events is a major challenge, par- ticularly in establishing an association with other potential indicators of a loss of process control. As noted for L. monocytogenes, because of these sampling limitations, a failure to detect E. coli O157:H7 does not neces- sarily demonstrate that a process is in control. As mentioned above, many producers of raw ground beef routinely test incoming raw ingredients for E. coli O157:H7 as a control measure to reduce the potential presence of the pathogen in the final product. However, because of the probabilistic nature of microbiological testing, there is a distinct possibility that the raw ingredients could test negative for E. coli O157:H7, but samples of the final product would be positive.

LETTER REPORT 35 Again, this reflects the characteristics and limitations of sampling and its dependence on the random nature of the contamination. To address the possibility that contaminated trim may not be detected, FSIS includes in the LOI 3 category those establishments that appeared in the STEPS da- tabase more than once in the preceding 120 days (FSIS, 2008b). How- ever, it is not clear that including the grinding establishment in the LOI 3 category and therefore increasing its inspection would solve the loss of process control. The basis for the number of times that a trim provider is listed in the STEPS database (i.e., why two instead of one or three) or the duration of time (i.e., 120 days) selected as a criterion for inclusion in the database is also not clear. It is not evident to the committee whether the algorithm considers in- stances in which a ground beef product has been found by the grinding facility to be adulterated by E. coli O157:H7, but subsequently sent for further processing and pathogen elimination. In this case, the grinding establishment would not be in loss of process control if it appropriately diverts product that has tested positive for the pathogen. Finding 8: The use of E. coli O157:H7 testing results for ground beef and its components to rank establishments in different levels of inspec- tion has been justified based on the potential direct health risk of the pathogen; this is a valid risk-management decision criterion, particu- larly for specific production lots. However, the initial data analysis has not provided scientific support for the ability of this decision criterion to predict loss of process control or for its association with other indica- tors. As previously mentioned, FSIS has not discussed the use of this in- dicator in relation to the sampling and related statistics that should be considered when using any microbiological sampling program to verify process control. RECOMMENDATION 8: The committee recommends improving the use of the presence of E. coli O157:H7 as an indicator of process control in raw ground beef by the following measures: • Provide a more in-depth description of the sampling and testing statistics that are the basis for the E. coli O157:H7 regulatory testing program and of how the characteristics and assumptions of the sampling and testing statistics influence the use and inter- pretation of the data for categorizing establishments based on this metric. This should include consideration of the magnitude

36 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS of type I and type II errors, assumed pathogen concentration means and standard deviations, specificity and sensitivity of the microbiological protocols, and so forth. • Assess the association of the practice of trim testing with the fre- quency of E. coli O157:H7 in final product to evaluate the use of trim testing as a risk determinant. This can be done by using ap- propriate study designs to address potential confounders and in- teractions. If such an association is found, incorporate this into the algorithm as applied to ground beef as a simple predictive criterion based on whether an establishment tests incoming trim to a sufficient degree. • Because of the low frequency of E. coli O157:H7 isolations in ground beef, evaluate data on other potential indicators of fecal contamination such as generic E. coli (see below). • Support research to develop and validate improved sampling and testing methodologies for E. coli O157:H7. Improved sampling and testing might increase confidence in the methodologies and decrease the number of false positives and false negatives. Rapid methodologies will decrease the temporal gap between the loss of control and the inspection, and therefore it is more likely that the associated problem (and the solutions) could be found. These kinds of improvements will enhance the reliability of the algorithm. E. coli O157:H7 and Salmonella Testing Results for RTE Products Use and Scientific Evidence In addition to testing raw ground beef, FSIS also tests specific RTE foods for the presence of E. coli O157:H7 and Salmonella. The following criteria for Salmonella and E. coli O157:H7 (Dreyling, 2008; FSIS, 20008b; E. Dreyling, FSIS, personal communication, February 18, 2009) define the level of inspection categories: • LOI 1: Establishment has not had a positive FSIS test result for Salmonella or E. coli O157:H7 in RTE products; or if it has, any related FSA and follow-up sampling has been completed more than 120 days previously, any related enforcement actions are closed, and establishment meets all other criteria for LOI.

LETTER REPORT 37 • LOI 2: For an establishment that has had a positive FSIS test for Salmonella or E. coli O157:H7 in RTE products, any related FSA and follow-up sampling has been completed in the previous 120 days and all related enforcement actions are deferred or in abeyance. • LOI 3: Establishment had an FSIS positive test for Salmonella or E. coli O157:H7 in a RTE product. Unlike the Salmonella verification testing program for raw meat and poultry products where it is used as an indicator of fecal contamination, Salmonella in RTE products is considered an adulterant. In summary, RTE foods are intended to be consumed without further processing, so any level of Salmonella or E. coli O157:H7 contamination is regarded as a risk to consumers, especially those that are more vulner- able. FSIS considers RTE products adulterated if either of these patho- gens is detected and therefore infers a loss of process control. Committee’s Discussion The presence of Salmonella or E. coli O157:H7 in an RTE product that has undergone a lethal treatment step (e.g., cooking) that substan- tially reduces levels of both pathogens is indicative of posttreatment re- contamination and thus a loss of process control. In other RTE products that receive a less stringent treatment (e.g., semidry fermented sausage), the effectiveness of the treatment depends on a number of factors, includ- ing ensuring a low level of contamination in raw ingredients. In these products, there is a small likelihood that a positive sample might be de- tected when the process was under control, but such a finding would more likely indicate either a loss of control or an inadequately validated process. While the occasional detection of a positive sample cannot sci- entifically be stated as a definitive loss of process control, using this de- tection as an indicator of process control is a valid risk-management de- cision to operationalize the decision criterion for the testing program and to safeguard the public from a pathogen capable of infecting individuals at a low dose. However, FSIS should be prepared for instances in which subsequent FSA findings cannot document a loss of process control, par- ticularly for RTE foods that have less stringent inactivation treatments. As with E. coli O157:H7 in ground beef and L. monocytogenes in RTE meats and poultry, testing for Salmonella or E. coli O157:H7 in RTE foods has all the advantages and limitations discussed earlier in the

38 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS general section on the use of microbiological testing. They include the possibility that products that test negative are not under control, as well as difficulties in demonstrating statistically significant associations with other predictive indicators due to the rarity of detecting these pathogens in RTE foods. Finding 9: The use of Salmonella and E. coli O157:H7 testing results in RTE foods to rank establishments in different levels of inspection has been justified based on the potential direct health risk of these patho- gens; this is a valid risk-management decision criterion, particularly for specific production lots. However, the initial data analysis has not pro- vided scientific support for use of this decision criterion to predict loss of process control or for its association with other indicators. As previously mentioned, FSIS has not discussed the sampling and related statistics that should be considered when using any microbiological sampling program to verify process control. The current use of these data to de- termine the risk associated with establishments producing RTE foods does not take into account the inherent differences in risk associated with the different classes of meat and poultry products that fall within the broad designation of RTE foods. RECOMMENDATION 9: The committee recommends improving the use of the presence of Salmonella and E. coli O157:H7 as an indicator of process control in RTE products by the following measures: • Provide a more in-depth description of the sampling and testing statistics that are the basis for the Salmonella and E. coli O157:H7 testing programs in RTE foods as well as how the characteristics and assumptions of the sampling and testing sta- tistics influence the use and interpretation of these data for cate- gorizing establishments based on this metric. This should include consideration of the magnitude of type I and type II errors, as- sumed pathogen concentration means and standard deviations, specificity and sensitivity of the microbiological protocols, and so forth. • Because of differences in the inherent risk of various subcatego- ries of RTE products, the product classes should be subdivided to determine whether better predictors can be identified for spe- cific products.

LETTER REPORT 39 • Because of the low frequency of Salmonella and E. coli O157:H7 isolations in RTE products, evaluate data on other po- tential indicators of process control. For example, and in con- formance with the committee’s recommendation below, devia- tions from control point limits in an RTE HACCP plan may be better suited as indicators of process control. • Support research to develop and validate improved sampling and testing methodologies for Salmonella and E. coli O157:H7. Im- proved sampling and testing methods might increase the confi- dence in the methodologies and decrease the number of false positives and false negatives. Rapid methodologies will decrease the temporal gap between the loss of control and the inspection, and therefore it is more likely that the associated problem (and the solutions) could be found. These kinds of improvements will enhance the reliability of the algorithm. Noncompliance Records Use and Scientific Evidence FSIS personnel perform thousands of inspection procedures each day in federally inspected slaughter and processing establishments to deter- mine whether the plants are in compliance with regulatory requirements. An NR is written to document noncompliance, and the establishment is notified so that it takes action to remedy the situation and prevent future recurrence. The issuance of an NR is prompted by any one of more than 500 citation violations, all of which relate to adherence to regulatory re- quirements (FSIS, 2008b). A committee review of NRs selected by FSIS and industry representatives reveals that some, but not all, NRs address public health risks. Because many NRs are not related to food safety or public health, FSIS performed the statistical analysis using the totality of NRs and also using exclusively health-related NRs to determine any im- provement of predictability with use of only the selected NRs. Two sets of health-related NRs were created and analyzed separately: a group of nine FSIS experts with diverse backgrounds in the regulation of meat, poultry, and egg products assigned each NR a weight of 3, 2, 1, or 0 in- dicating the degree of loss of process control it represented, and the me- dian score of each was used to identify those with a weight of three (W3NR). A second set of health-related NRs was identified by an indus- try coalition.

40 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS FSIS proposes to categorize plants according to the level of inspec- tion needed (LOI 1, LOI 2, or LOI 3) (Dreyling, 2008; FSIS, 2008b; E. Dreyling, FSIS, personal communication, February 18, 2009) in response to their NR rates in the following way: • LOI 1: An establishment whose public health NR rate (over a rolling three-month average) is less than the LOI 1 percentile cut point, when all other indicators suggest that the process is in control. • LOI 2: An establishment whose public health NR rate (over a rolling three-month average) is greater than the LOI 1 percentile cut point but less than the LOI 3 percentile cut point. • LOI 3: An establishment with health-related NR rates (over a rolling three-month average) higher than the highest percentile of health-related NR rates (e.g., those citing specified risk material [SRM], insanitary dressing, zero tolerance, residue). FSIS included a statistical analysis to justify the use of NRs as pre- dictors of loss of process control in Appendix E of the technical report PHRBIS (FSIS, 2008b). FSIS proposes to determine the percentile cut point for the three levels by analyzing the number of health-related NRs (W3NRs, as identified by FSIS) issued to each specific type of estab- lishment (e.g., broiler processing, beef slaughter, beef grinding) and find- ing the inflection points in the curve representing the number of estab- lishments versus the percentage of NRs over a period of three months. An example for beef slaughter establishments given by FSIS is shown in Figure 2.

LETTER REPORT 41 20 W3NR Cut Points for Beef Slaughter 18 16 14 W3NR % 12 10 8 6 4 LOI3 Cut Point 2 LOI1 Cut Point 0 1 11 21 31 41 51 61 71 81 91 101 111 121 131 141 151 161 171 Number of Plants FIGURE 2 Inflection points in the W3NR data used to determine cut points in the proposed algorithm for beef slaughter establishments. SOURCE: E. Dreyling, FSIS, personal communication, December 13, 2008. Once the NRs were selected, FSIS performed a lift analysis of the various sets of NRs (i.e., W3NRs and those selected by an industry coali- tion as described above) to estimate their power to predict the loss of process control. The lift analysis was performed using aggregated data, that is, data from all types of establishments, against the following vari- ables: pathogen test results, consumer complaints, food safety recalls, enforcement actions, and RTE L. monocytogenes alternatives (FSIS, 2008b). Results from the lift analysis indicate that the rate of W3NRs received by an establishment could be used to predict positive Salmo- nella verification test results. As an example, the lift analysis showed that receipt of one W3NR within the previous seven days correlated with a threefold increase in the likelihood of recording a positive test result for Salmonella within the following two weeks. Different periods of time, or time windows, were used and a decrease in lift (or predictability) was noted with an increase in time window. The lift was greatest if the W3NR group was used for the analysis rather than either the aggregated NRs or the industry-proposed NRs. Therefore, W3NRs may be predic- tive of loss of process control and of an unsafe product. None of the analyses with other outcome variables found a significant lift, suggesting that health-related NRs would not necessarily be a good predictor of con- sumer complaints, food safety recalls, enforcement actions, RTE L.

42 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS monocytogenes alternatives, or future product contamination with E. coli O157:H7 or L. monocytogenes. The significant association between NRs and Salmonella verification testing was the FSIS’s basis for using se- lected health-related NRs as an indicator of process control and therefore including this indicator in the FSIS proposed algorithm. Lift analysis was also performed using only those individual NRs that were issued most frequently (more than 1,000 times over seven days), and the results were reviewed by the committee (FSIS, 2008b). This analysis showed that some of the NRs clearly contribute to the pre- dictive ability of W3NRs much more than others. These promising re- sults show that an even more limited number of NRs may be identified as significantly predictive. For example, an NR issued due to visible fecal material entering the chiller in a poultry operation is 4.8 times more likely to be followed by a positive test for Salmonella, with a highly sig- nificant association (p < 0.001; 95 percent CI: 4.251-5.513), compared to an NR resulting from lack of compliance with general rules, which had a nonsignificant likelihood of association of 2.4 (p < 0.075; 95 percent CI: 0.671-4.364). These values were for an evidence and outcome window of seven days. Committee’s Discussion Several limitations could affect the ability of NR data to predict loss of process control. These relate to the subjective nature and purpose of NRs and the statistical analysis used to determine their predictive accu- racy, as well as human factors. NRs were developed not to indicate a loss of process control but rather to document failure to comply with USDA regulations. It appears that many of the current public health-related NRs are not aligned with the HACCP plan, the food safety system required in FSIS-inspected es- tablishments. For example, some critical control points are not refer- enced in the current NRs. In selecting the public health-related NRs (i.e., W3NRs), FSIS assigned them equal weight in the assessment and the algorithm. It is likely that some of these NRs are more closely associated with public health risk than others, as evidenced by data presented by FSIS (E. Dreyling, FSIS, personal communication, December 13, 2008). Some of the regulations underpinning the issuance of NRs are nonspe- cific (e.g., section 416.4[d] accounts for more than 50 percent of the NRs written during all of 2006, three months in 2007, and one month in 2005, according to data provided by FSIS [E. Dreyling, FSIS, personal com- munication, December 13, 2008]) and aggregate failures that may not be

LETTER REPORT 43 related to health. In addition, some current NRs document outcomes of control failure so they are already used independently to classify levels of inspection. For example, NRs issued due to a positive result for E. coli O157:H7 or L. monocytogenes in RTE foods do not predict future loss of control but are instead an outcome of past loss of control and therefore, by definition, put an establishment in LOI 3. To address these current limitations, a more focused, commodity-based analysis excluding those NRs that are already used independently to classify levels of inspection would help identify true predictors of process control. Also, future public health-related NRs should be closely aligned to the pertinent HACCP plan, especially the critical control points. Another limitation derives from the subjective nature of some NRs. There are two types of cause for writing an NR: (1) visual and or- ganoleptic evidence that a regulatory requirement is not being met, as observed by inspection personnel, and (2) laboratory findings (microbi- ological data, product composition, etc.) demonstrating that a regulatory requirement is not being met. The decision to issue an NR is not always based on quantitative data, but often relies on observation and is there- fore a subjective decision on the part of inspection personnel. It is impor- tant to note that the levels of technical experience and training of the in- spection personnel who write NRs are very diverse. It is the committee’s observation that significant variation can be expected in the interpreta- tion of regulations and in individual inspector’s criteria for justification of a specific NR. Supervisory review by the inspector-in-charge may likewise be variable or subject to bias and, therefore, unreliable. As part of the in-plant performance system (IPPS), supervisors are to assess the quality of the NRs written by inspectors at least once in a rating year (E. Dreyling, FSIS, personal communication, December 13, 2008). The dis- tricts also randomly select a number of NRs every month to review for quality and to provide feedback to supervisors. FSIS has indicated that the quality assessment it currently uses does not focus on the factual ac- curacy of the NR, but rather on whether it includes all of the administra- tive elements required (e.g., regulation violated, type of product and process, corrective action taken) (E. Dreyling, FSIS, personal communi- cation, December 13, 2008). The results of the IPPS and the district as- sessments are captured in AssuranceNet, but they focus on the profi- ciency of the employee, not the quality and substance of individual NRs. The lift analysis performed by FSIS suggests that the likelihood of a positive Salmonella verification test is higher when specific NRs have previously been written for an establishment. The lift analysis shows only preliminary associations and will obviously be applicable only to

44 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS commodities for which Salmonella verification testing is required. For example, the analysis provides no justification for the use of NR rates to categorize RTE establishments where Salmonella verification testing is not conducted. In addition, the data used for the analysis were not segre- gated by type of commodity; instead, the data were aggregated across all commodities, which increased the statistical power but possibly gener- ated a bias in the association. That is, because Salmonella is more likely to occur in a poultry-processing establishment than in a beef operation, data from raw poultry may drive the analysis. NRs for an establishment that processes raw chicken might be different from or less frequently generated than NRs for a plant that processes ground beef or RTE meat. More accurate use of these data would require the selection of NRs that reflect a loss of process control for each type of product. Another limitation is what appears to be an arbitrary selection of threshold points to classify an establishment as LOI 1, LOI 2, or LOI 3. FSIS proposes to use the inflection points of the curve of the number of plants with a specific NR rate, which might not be related to a particular risk differential between establishments, as the threshold point to assign establishments to a level of inspection (Figure 2). This may be especially problematic for commodities with a small positive sample size. There are also numerous ways of determining inflection; to evaluate the scientific adequacy of the determination method, FSIS should describe it in the technical report PHRBIS. Finally, the committee concludes that the data indicate that reliance on use of the rate of NRs as a measure of process control might present a logistical and economic disadvantage for small plants, which may lack the legal or financial resources to appeal an NR and whose rate of NR receipt may therefore appear misleadingly high. For example, during 2008, the rate of NRs was higher in small and very small plants than in large plants (E. Dreyling, FSIS, personal communication, December 13, 2008). The committee questions this difference and recommends that it be investigated because it could distort a plant’s NR rate and consequent categorization into an LOI. Finding 10: The use of selected NRs as process control indicators in a risk-based inspection system offers potential. However, because current NRs are written to document failure to comply with regulations, not all of them are predictors of loss of process control. The subjective nature of the issuance of NRs also limits their use as process control indicators. The description of the association between NRs and other measures of process control would benefit from a more effective communication of

LETTER REPORT 45 which NRs are employed for specific commodities and which ones are pertinent to all meat and poultry products. RECOMMENDATION 10: The committee strongly recommends that FSIS conduct the following activities to improve the scientific basis for using NRs as process control indicators: • Stratify lift statistics on Salmonella verification test results and NRs by plant size and commodity. • Perform further appropriate statistical analysis to identify and rank which public health-related NRs among the 66 W3NRs have the greatest predictive value for various product classes. • Convene a panel of qualified external scientific experts to review the results of these analyses and other potential factors; this panel would also make recommendations on weighting factors for the different product classes. • Validate the use of NRs as predictors of positive Salmonella verification test results by conducting a pilot test to ensure that positives are not occurring simply as a result of the expected baseline load of Salmonella in the products or the variability of Salmonella load. • Investigate the utility of surrogate indicators (e.g., generic E. coli or others) to identify noncompliance records that might be pre- dictors of public health risk and loss of process control. RECOMMENDATION 11: In concurrence with previous NAS reports (NRC, 1987; IOM, 1990, 1998, 2003), the committee recommends fo- cusing on those inspection activities that foster the implementation of and compliance with HACCP systems and sanitary requirements. The committee strongly recommends that additional NRs be developed to reflect indicators of process control, instead of relying entirely on NRs that were created for purposes of regulatory compliance. The committee recommends that FSIS identify, validate, and adopt those NRs that are truly predictive of future contamination problems—for example, those being triggered by process deviations from HACCP plan critical control point limits. This exercise should be conducted under the guidance of a non-FSIS expert panel. RECOMMENDATION 12: To reduce the subjectivity implicit in cur- rent NRs, the committee recommends supporting the improvement of the

46 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS inspection force (inspectors and supervisors) by strengthening oversight of the writing of NRs to determine not only that the appropriate informa- tion is provided on regulatory citations, but that the information is both factual and properly documented to support the noncompliance; and im- proving the training and testing of inspection personnel themselves, with special emphasis on the quality and consistency of NRs and on any new NRs to be developed. Enforcement Actions Use and Scientific Evidence Enforcement actions are taken by FSIS against an establishment that fails to comply with regulatory requirements. As with NRs, not every enforcement action is related to a food safety problem; some may be is- sued in response to regulatory requirements of a different nature (e.g., nutritional labeling). FSIS Rules of Practice define the type of adminis- trative enforcement action that FSIS takes under a given condition and the procedures to follow. The administrative actions include regulatory control action, withholding action, and suspension. When there is an imminent threat to public health, FSIS takes immediate action. In other situations, FSIS provides prior notification of intended enforcement ac- tion to the establishment. FSIS may defer an enforcement decision based on corrections submitted by the establishment, or it may place a suspen- sion action in abeyance if an establishment presents and puts into effect corrective and preventive action. Examples of food safety-related reasons for enforcement action are failure of the establishment to comply with its HACCP plan or with regulatory requirements for generic E. coli testing. Enforcement actions are typically initiated by the Enforcement Investiga- tions and Analysis Officer (EIAO) during an FSA and after a lack of ef- fective corrective action leads to repeated NRs, indicating that the prob- lem has not been adequately addressed, or after a severe violation (FSIS, 2008b). For the most severe violations, enforcement actions may not be preceded by an NR. FSIS proposes to use enforcement actions as indicators of loss of process control in cases when enforcement actions remain open (LOI 3) or are deferred or in abeyance as a result of a process control failure (LOI 2) (E. Dreyling, FSIS, personal communication, February 18, 2009). If an enforcement action is taken as the result of an FSA in an LOI 3 estab-

LETTER REPORT 47 lishment, the establishment cannot be recategorized to LOI 2 until an enforcement action is deferred or in abeyance. If an enforcement action is taken for a reason other than an FSA, the establishment will also be placed into LOI 3. A lift analysis was performed between enforcement actions and NRs. As presented in Appendix E of the technical report PHRBIS (FSIS, 2008b), the results suggest that NRs are not predictive of future enforce- ment actions. The ability of enforcement actions to predict future patho- gen contamination was also statistically analyzed. The lift calculations show that enforcement actions do not appear to be good predictors of process control (Appendix E of PHRBIS). Nevertheless, FSIS justifies the use of enforcement actions as proc- ess control indicators because by definition they arise from a failure to abide by regulations, some of which are related to food safety (FSIS, 2008b). FSIS proposes to use those enforcement actions that are related to food safety issues as indicators of loss of process control. Committee’s Discussion Based on the statistical analysis and due to the potential time lag be- tween a process control failure and the consequent enforcement action (several weeks may pass before an action is initiated), enforcement ac- tions are likely not to be predictive but instead only indicative of past problems. Enforcement actions are, in fact, reactive processes. In addition, some enforcement actions are further delayed to ensure that sufficient legal evidence has been collected to substantiate an action related to a public health concern, possibly further decreasing their value as predic- tors. An enforcement action is an outcome of failure, that is, it may indi- cate that the process is already out of control, but not necessarily predict future loss of control. If the NRs were weighted and specific outcome-based NRs were re- moved, there might be a correlation with enforcement actions. However, enforcement actions occur at a low frequency (e.g., 222, 217, and 308 enforcement actions were issued in 2006, 2007, and 2008, respectively (E. Dreyling, FSIS, personal communication, December 13, 2008), so finding a statistical correlation with other indicators of loss of process control might not be possible. In fact, results of the lift analysis per- formed by FSIS did not suggest any likelihood that enforcement actions are related to other process control indicators, such as pathogen or indi- cator organism test results (FSIS, 2008b). Therefore, as the technical re-

48 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS port PHRBIS indicates, it is not statistically justifiable to use enforce- ment actions as a predictor of process control. As indicated above, differences in the ability to appeal NRs might predispose certain plants to receive enforcement actions, thereby intro- ducing a facility size-based bias to this algorithm. Finding 11: The use of enforcement actions to rank establishments in different levels of inspection has been justified based on their suggesting a past loss of control, a valid risk-management decision criterion. How- ever, the initial data analysis has not provided scientific support for use of this decision criterion to predict a loss of process control or for its association with other indicators. Enforcement actions currently prompt regulatory action, so they already result in categorization in LOI 3. RECOMMENDATION 13: The committee recommends using the en- forcement actions that result from a failure of process control to catego- rize establishments in levels of inspection, not as predictive indicators of loss of process control. Recalls Use and Scientific Evidence There are three types of recalls that an establishment voluntarily conducts in response to detection of a problem in a food product that has already reached the market (FSIS, 2008b): • Class I recall: Prompted by a situation in which there is a reasonable probability that the use of or exposure to a violative product will cause serious adverse health consequences or death • Class II recall: Prompted by a situation in which use of or exposure to a violative product may cause temporary or medically reversible adverse health consequences, or where the probability of serious adverse health consequences is remote • Class III recall: Prompted by a situation in which use of or exposure to a violative product is not likely to cause adverse health consequences

LETTER REPORT 49 The FSIS proposes to use recalls to categorize plants by level of in- spection (Dreyling, 2008; FSIS, 2008b; E. Dreyling, FSIS, personal communication, February 18, 2009) as follows: • LOI 1: Establishment has not shipped adulterated or misbranded product (includes recalls related to human illness); or if it has, any related FSA and follow-up sampling was completed more than 120 days previously, any related enforcement actions are closed, and establishment meets all other criteria for LOI 1. • LOI 2: For an establishment that has shipped adulterated or mis- branded product (includes recalls related to human illness), any related FSA and follow-up sampling was completed in the previ- ous 120 days, and all related enforcement action (e.g., Notice of Intended Enforcement [NOIE]) is deferred or in abeyance. • LOI 3: Establishment has shipped adulterated or misbranded product or is undergoing enforcement action (e.g., NOIE) that is not the result of an FSA. As with other indicators of loss of process control, not every recall is prompted by public health concerns; by definition, Class I and II recalls are (or might be) related to public health, whereas Class III recalls are not. FSIS conducted lift statistics to determine whether the occurrence of a public health recall could be used to assess NRs as predictors of loss of control. It also assessed whether recalls could predict enforcement ac- tions or pathogen test results (FSIS, 2008b). In general, except for an association of Class I and II recalls with the finding of L. monocytogenes in RTE products, the results of these analyses were not statistically sig- nificant (see FSIS, 2008b; Appendix E). According to FSIS, those recalls related to issues affecting public health (e.g., linked to product failure due to the presence of microbial pathogens) would be, by definition, indicative of a potential process con- trol failure and therefore already identify the need for in-depth inspection at the offending establishment (FSIS, 2008b). Committee’s Discussion One limitation shared with other indicators of process control is the low frequency of occurrence of recalls. For example, FSIS reports that the total number of recalls in 2008 was 48 (there were 39 Class I recalls, 9 Class II recalls, and no Class III recalls) (FSIS, 2008e). The utility of

50 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS recalls is further limited by their lack of specificity as well as their lack of timeliness (they may occur too late to identify issues in food process- ing); their use is reactive to a past food safety problem, not necessarily predictive of a future problem. As FSIS states, the statistical analysis that estimates the ability of re- calls to predict a loss of process control should allow for differentiation between recalls that have a public health impact and those that do not. A public health-related recall is often based on a laboratory test result—that is, the result of a microbiological test. Because of the limitations of sam- pling (size, frequency, etc.) and the sporadic nature of contamination, the isolation of a foodborne pathogen may not indicate a breakdown in proc- ess control. In some situations, especially related to E. coli O157:H7 in ground beef, a recall is based on failure to hold a product that has been tested for a pathogen, not a failure of the control process. Finding 12: The use of public health-related recalls to rank establish- ments in different levels of inspection has been justified based on poten- tial direct public health risk, a valid risk-management decision criterion. However, the initial data analysis has not provided scientific support for this decision criterion as being predictive of a loss of process control or for its association with other indicators. RECOMMENDATION 14: Only health-related product recalls should be included in the model for ranking public health risks and assigning inspection resources. FSIS should continue to conduct assessments and take regulatory enforcement actions in plants following a recall. STEPS Database Use and Scientific Evidence The System for Tracking E. coli O157:H7 Positive Suppliers data- base identifies suppliers of trim to grinding operations whose ground beef product tests positive for E. coli O157:H7. FSIS proposes to use this database to categorize LOIs for supplier establishments in the following manner (Dreyling, 2008; FSIS, 2008b; E. Dreyling, FSIS, personal communication, February 18, 2009):

LETTER REPORT 51 • LOI 1: Establishment has not been cited in the STEPS database more than once; or if it has, any related FSA and follow-up sampling has been completed more than 120 days previously, any related enforcement actions are closed, and establishment meets all other criteria for LOI 1. • LOI 2: For an establishment in the STEPS database more than once, any related FSA and follow-up sampling has been com- pleted in the previous 120 days, and all related enforcement ac- tions are deferred or in abeyance. • LOI 3: Establishment was in the STEPS database more than once within the previous 120 days. The justification for using the STEPS database is that grinding op- erations lacking an E. coli O157:H7 intervention step that would de- crease the likelihood of the presence of pathogens need a process control step to ensure that incoming trim products are not already contaminated with pathogens. Committee’s Discussion The committee recognizes the potential benefits of this approach and would be interested in seeing the details and data supporting it. As dis- cussed above in relation to testing ground beef for E. coli O157:H7, FSIS should assess the role of testing trim for E. coli O157:H7 as a risk deter- minant. Foodborne Disease Outbreaks A foodborne outbreak is the occurrence of two or more cases of a similar illness resulting from the ingestion of a common food. FSIS pro- poses to use foodborne disease outbreaks to categorize establishments in levels of inspections as follows (Dreyling, 2008; FSIS, 2008b; E. Drey- ling, FSIS, personal communication, December 13, 2008): • LOI 1: An establishment has not been linked to an outbreak; or if it has, any related FSA and follow-up sampling has been com- pleted more than 120 days previously, any related enforcement actions are closed, and establishment meets all other criteria for LOI 1.

52 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS • LOI 2: For an establishment that was linked to an outbreak, any related FSA and follow-up sampling has been completed in the previous 120 days, and all related enforcement actions are de- ferred or in abeyance. • LOI 3: Human illness was linked to an FSIS-regulated product from the establishment. Finding 13: The use of foodborne disease outbreaks to rank establish- ments in different levels of inspection has been justified based on their potential direct public health risk, a valid risk-management decision cri- terion. However, the initial data analysis has not provided scientific sup- port for use of this decision criterion to predict loss of process control or for its association with other indicators. RECOMMENDATION 15: The committee recommends including foodborne disease outbreaks in the algorithm to categorize plants in lev- els of inspection. The committee also strongly recommends that FSIS systematically work with other appropriate federal and state agencies to routinely disseminate public reports of the results of the investigations into the plant and process failures associated with these outbreaks. Salmonella Serotypes of Human Health Concern If FSIS is planning to use specific serotypes as indicators of process control, serotypes not often linked to human health should also be con- sidered. Since the potential application of serotype evaluation to the des- ignation of facilities as LOI 1, 2, or 3 is dependent on establishing a clear relationship between individual serotypes and disease attribution, any recommendations by this ad hoc committee await the findings of the NAS Committee on Review of the Food Safety and Inspection Service (FSIS) Risk-Based Approach to Public Health Attribution. Consumer Complaints The use of consumer complaints as a potential indicator of process control was analyzed by FSIS and then dismissed due to the challenge of overcoming its limitations (FSIS, 2008b). The committee agrees that the process currently employed to collect and analyze consumer complaints is not appropriate for use as an indicator of an establishment’s need for a

LETTER REPORT 53 higher level of inspection. In addition to having no significant associa- tions with other potential indicators (see Appendix E in FSIS, 2008b), consumer complaints may often incorrectly associate a food with an ad- verse health effect. OTHER POTENTIAL INDICATORS OF PROCESS CONTROL Microbial Test Results The primary goal of process control for raw meat and poultry prod- ucts is to limit the presence of fecal contamination, the source of enteric pathogenic microorganisms. Both Salmonella and generic E. coli are in- dicators of fecal contamination and, as such, indicators of loss of process control, and both were targeted by FSIS in the pathogen reduction HACCP regulation. The ideal process indicator is one that is present at sufficient levels and frequency to be measured on a routine basis. For some commodities (e.g., beef carcasses), Salmonella is currently found so rarely that its usefulness as an indicator is limited. It is envisioned that for other commodities where it is currently useful, Salmonella could be- come equally rare in the future. FSIS would benefit from identifying al- ternative microbial indicators that could augment current indicators of fecal contamination on a commodity-specific basis. Data on generic E. coli are collected by individual plants on a regular basis, but are not used by FSIS. Establishments are not required to send such data to FSIS, only to make them available if requested. According to FSIS, there are two limitations to the collection of data on generic E. coli that prevent FSIS from using them as indicators (C. Travis, Science Applications International Corporation, personal communication, De- cember 13, 2008). One is FSIS’s concern about the comparability of data resulting from a variety of different testing methods. Although the PR/HACCP regulation states that validated methods of testing should be used, there is no required single standard testing methodology or sam- pling procedure. It would appear that this could readily be corrected if FSIS articulated the specific methodological requirements (e.g., sensitiv- ity, specificity, reproducibility, repeatability) of its current standard methods and its expectation that similar performance would be achieved by alternative validated methods. The second limitation is that the agency does not currently have the information technology capability to

54 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS retrieve and process such data efficiently. It is worth noting that in its May 2008 technical report to the NACMPI on poultry slaughter, FSIS presented convincing evidence of the potential utility of generic E. coli as a process control indicator and suggested that it was considering a new performance standard based on its use (FSIS, 2008a). These findings were based on a detailed study of various potential indicators of process control performed by FSIS and ARS. FSIS indicated that it has analyzed a 2006-2008 generic E. coli data set from its baseline program, but be- cause this data set is small, the results were inconclusive (FSIS, 2008a). Indicator organisms in RTE foods are used not as indicators of fecal contamination but rather as indicators of other control measures, such as the adequacy of the microbicidal step, prevention of recontamination, and maintenance of proper storage conditions (e.g., refrigeration). The FSIS algorithm does not currently include the use of any indicator mi- croorganism for assessing process control in RTE foods. Given the low frequency of L. monocytogenes or other pathogens, FSIS would benefit from identifying appropriate alternative microbial indicators that could be used to assess applicable process controls in RTE foods. Although generic E. coli might not be an indicator of fecal contamination in RTE products, it is still a valuable indicator of general sanitation, recontami- nation problems, and temperature abuse. Finding 14: Microbes currently used as process control indicators are only rarely found in some commodities and are therefore of limited use- fulness (e.g., Salmonella in ground beef). It is anticipated that in the fu- ture, Salmonella will be even less frequent and therefore less valuable as an indicator. Furthermore, in the proposed algorithm, there are no iden- tified process control indicators for RTE foods. RECOMMENDATION 16: FSIS should investigate the potential util- ity of industry data on generic E. coli as an indicator of process control. The committee recognizes the challenges of this approach, but encour- ages FSIS to act promptly to complete the analysis of the data it has al- ready acquired, collect additional data as necessary, and analyze them for their predictive ability as potential indicators of process control. Use of the HACCP System A HACCP plan is developed by identifying steps in a specific meat and poultry process that are critical to ensuring food safety and is meant

LETTER REPORT 55 to include the application of corrective actions when those critical control points are not met. Critical control points in HACCP plans were regarded as points in a process in need of specific interventions that, if failed, might result in an end product with risks to public health. If a commodity that was produced under a process deviation leaves the plant without cor- rective action, this constitutes a loss of control. By regulation, the control point limits are to be validated and verified by the establishments (Haz- ard Analysis and Critical Control Point [HACCP] Systems. Validation, Verification, Reassessment. 2008. 9 CFR § 417.4). Therefore, it may be appropriate for FSIS to study the feasibility of a system in which devia- tions from control point limits are incorporated as NRs and used to cate- gorize plants according to the inspection level required. In fact, for most HACCP plans, critical control points and limits should be similar in na- ture for all facilities processing the same commodity. The committee ac- knowledges that for the processing of raw product, defining the control points is challenging; in these cases, more weight could be allocated to pathogen contamination as a control indicator. RECOMMENDATION 17: The committee recommends that FSIS consider using specific critical control point deviations as indicators of process control. Process deviations should be integrated into an algo- rithm to categorize plants according to the level of inspection needed. Because of inherent problems in the use of NRs described above, the committee recommends redefining public health-related NRs and creat- ing new ones where appropriate so that they reflect the current view of HACCP as a food safety control approach. This approach should identify true science-based indicators of process control. This concept should be included in inspection training programs. USDA should conduct a pilot study in a few plants to determine if the new NRs based on HACCP critical control point adherence are valid and useful parameters to be considered as predictors of loss of process control. This should be fol- lowed by longitudinal studies designed to validate the new NRs. Value of Real-Time In-Plant Data Acquisition The committee supports FSIS’s efforts to explore options for rapid collection and reporting of real-time data that indicate potential failures of process control. The real-time data should focus on objective meas- ures of control (e.g., critical control points) for the process and take ad- vantage of electronic data-capturing tools.

56 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS The committee agrees that FSIS should use in-plant inspection per- sonnel to collect real-time data. They would provide immediate input into the algorithm indicating a potential failure of process control. To be effective, objective process performance measures should be defined. For example, real-time tracking of repetitive instances of noncompliance that are related to food safety and that affect process control would be re- ported and used as indicators of process control. Also, whenever feasible, performance measures should allow action to be taken before the process fails. The committee supports FSIS’s current activities to develop such a system and urges that it do so concurrently with carefully designed train- ing of its inspection and supervisory personnel. CONCLUSION The committee recognizes the magnitude of the task of designing a risk-based system to rank meat and poultry slaughtering and processing establishments based on their impact on public health. The committee notes that at the request of FSIS, only the data on and analysis of indica- tors of process control were reviewed. Other components of the algo- rithm (e.g., volume) vital to determining its applicability were not. FSIS should include as part of the proposed inspection system a specific plan for when and how it will evaluate the system. Scientific verification and validation are essential to evaluate the success or failure of the new pro- gram. The committee agrees with the general concept of using process con- trol indicators as part of an algorithm to rank establishments in different levels of inspection. The committee recommends that FSIS continue the collection and analysis of data and, in consultation with stakeholders and expert panels, continue to improve its proposed risk-based inspection system so that it more effectively allocates inspection resources accord- ing to risk. Prior to implementing this algorithm, the recommendations in this report should be followed. Specifically, the committee emphasizes the need to align the process control indicators of a risk-based inspection system with HACCP, a framework required throughout the meat and poultry slaughtering and processing industry that serves to minimize the risk of foodborne illness. The committee also recommends that FSIS improve the clarity and transparency of the algorithm so that its intent, scientific basis, and im- plementation are clearly articulated and understood by all stakeholders. One option for FSIS to communicate effectively with stakeholders would

LETTER REPORT 57 be to produce supplemental informative documents targeted to specific audiences (e.g., inspectors, plant managers), in addition to a technical report. Also, because this new algorithm would bring about changes in inspection procedures, a parallel training program for the inspection force would likewise be necessary. The Committee on Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System thanks FSIS for the opportunity to review the technical report Public Health Risk-Based Inspection System for Processing and Slaughter and hopes that its find- ings and recommendations are useful. The committee will be available to FSIS for any clarifications regarding this letter. Sanford Miller, Chair Committee on Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System Attachments Appendix A Meeting Agendas Appendix B Levels of Inspection Appendix C Acronyms Appendix D Glossary Appendix E Biographical Sketches of Committee Members REFERENCES CDC (Centers for Disease Control and Prevention). 2008. Preliminary FoodNet data on the incidence of infection with pathogens transmitted commonly through food—10 states, 2007. Morbidity and Mortality Weekly Report 57(14):366-370. Dreyling, E. 2008. Public health risk ranking for processing and slaughter es- tablishments. Presented at the November 6, 2008, meeting of the IOM Committee on Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System, Washington, DC. FDA-USDA-EPA-CDC (U.S. Food and Drug Administration-U.S. Department of Agriculture-U.S. Environmental Protection Agency-Centers for Disease Control and Prevention). 1997. Food Safety from farm to table: A national food safety initiative. Report to the President. Available at http://www.foodsafety.gov/~dms/fsreport.html (accessed February 20, 2009). FSIS (Food Safety and Inspection Service). 1996. 9 CFR Parts 304, 308, 310, 320, 327, 416, and 417. Pathogen Reduction; Hazard Analysis and Critical

58 REVIEW OF THE USE OF PROCESS CONTROL INDICATORS Control Point (HACCP) Systems; Final Rule. Federal Register 61(144):38805-38989. FSIS. 1999. 9 CFR Parts 301, 317, 318, 320, and 381. Performance standards for the production of certain meat and poultry products; Final Rule. Federal Register 64(3):732-749. FSIS. 2008a. Improvements for poultry slaughter inspection. Technical report. Available at http://www.fsis.usda.gov/Regulations_&_Policies/National Advisory_Committee_on_Meat_&_Poultry/index.asp#August (accessed January 26, 2009). FSIS. 2008b. Public health risk-based inspection system for processing and slaughter. Technical report. Available at http://www.fsis.usda.gov/ Regulations_&_Policies/National_Advisory_Committee_on_Meat_&_Poult ry/index.asp#August (accessed January 7, 2009). FSIS. 2008c. Progress report on Salmonella testing of raw meat and poultry products, 1998-2007. Available at http://www.fsis.usda.gov/Science/ Progress_Report_Salmonella_Testing/index.asp (accessed February 23, 2009). FSIS. 2008d. The FSIS Microbiological Testing Program for Ready-to-Eat (RTE) Meat and Poultry Products, 1990-2007. Available at http://www.fsis. usda.gov/Science/Micro_Testing_RTE/index.asp#trends (accessed February 23, 2009). FSIS. 2008e. Current recalls and alerts. Available at http://www.fsis.usda.gov/ Fsis_Recalls/Open_Federal_Cases/index.asp (accessed December 7, 2008). GAO (U.S. General Accounting Office). 1992. Food safety and quality. Uni- form, risk-based inspection system needed to ensure safe food supply. Re- port to the Chairman, Subcommittee on Oversight and Investigations, Committee on Energy and Commerce, U.S. House of Representatives. Re- sources, Community, and Economic Division, Report 92-152. Washington, DC. Giovannacci, I., C. Ragimbeau, S. Queguiner, G. Salvat, J. L. Vendeuvre, V. Carlier, and G. Ermel. 1999. Listeria monocytogenes in pork slaughtering and cutting plants. Use of RAPD, PFGE and PCR-REA for tracing and mo- lecular epidemiology. International Journal of Food Microbiology 53(2- 3):127-140. HHS (U.S. Department of Health and Human Services). 2000. Healthy people 2010, Volume 1. Washington, DC: U.S. Government Printing Office. Avail- able at http://www.healthypeople.gov/Publications/ (accessed January 7, 2009). ICMSF (International Commission on Microbiological Specifications for Foods). 2002. Microorganisms in foods 7. Microbiological testing in food safety management. New York: Klewer Academic/Plenum Publishers. IOM (Institute of Medicine). 1990. Cattle inspection. Washington, DC: The National Academy Press.

LETTER REPORT 59 IOM. 1998. Ensuring safe food: From production to consumption. Washington, DC: The National Academy Press. IOM. 2003. Scientific criteria to ensure safe food. Washington, DC: The Na- tional Academies Press. Keto-Timonen, R., R. Tolvanen, J. Lundén, and H. Korkeala. 2007. An 8-year surveillance of the diversity and persistence of Listeria monocytogenes in a chilled food processing plant analyzed by amplified fragment length poly- morphism. Journal of Food Protection 70(8):1866-1873. Lundén, J. M., T. J. Autio, A. M. Sjöberg, and H. J. Korkeala. 2003. Persistent and nonpersistent Listeria monocytogenes contamination in meat and poul- try processing plants. Journal of Food Protection 66(11):2062-2069. NRC (National Research Council). 1987. Poultry inspection: The basis for a risk-assessment approach. Washington, DC: National Academy Press. Peccio, A., T. Autio, H. Korkeala, R. Rosmini, and M. Trevisani. 2003. Listeria monocytogenes occurrence and characterization in meat-producing plants. Letters in Applied Microbiology 37(3):234-238. Thévenot, D., A. Dernburg, and C. J. Vernozy-Rozand. 2006. An updated re- view of Listeria monocytogenes in the pork meat industry and its products. Applied Microbiology 101(1):7-17. WHO-FAO (World Health Organization-Food and Agriculture Organization of the United Nations). 2004. Risk assessment of Listeria monocytogenes in ready-to-eat food: Technical report. Microbiological Risk Assessment Se- ries 5. Geneva, Switzerland: WHO.

Next: Appendix A Meeting Agendas »
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The United States Department of Agriculture's Food Safety and Inspection Service (FSIS) is the government agency responsible for ensuring the safety of America's supply of meat, poultry, and egg products. In an effort to improve its inspection system, FSIS has proposed to modify the allocation of its inspection resources by establishing criteria to rank, based on public health risk, slaughtering and processing establishments. Before implementing the proposed inspection system, FSIS asked the Institute of Medicine (IOM) to evaluate the system, particularly the criteria for ranking slaughtering and processing establishments. In its 2009 letter report Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System, the IOM committee concurs with the use of the risk-based inspection system but makes several recommendations to improve the process.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!