REVIEW OF THE CENTERS FOR DISEASE CONTROL AND PREVENTION’S SMALLPOX VACCINATION PROGRAM IMPLEMENTATION

Letter Report #6

July 6, 2004

Dr. Julie Gerberding

Director

Centers for Disease Control and Prevention

1600 Clifton Road, NE Atlanta, GA 30333

Dear Dr. Gerberding:

The Institute of Medicine (IOM) Committee on Smallpox Vaccination Program Implementation is pleased to offer you the sixth in a series of brief reports.

This report may seem like a departure from the committee’s previous work, which focused on smallpox vaccination as a part of public health preparedness. However, this report responds to a CDC request for guidance as the agency moves toward comprehensive preparedness for bioterrorism and other public health disasters, and toward broad smallpox preparedness efforts. The committee was asked to look specifically at preparedness exercises, which are required by CDC grant guidance and are being conducted by public health agencies. In general, the public health community has somewhat limited experience with exercises, so the committee was asked to describe the state of the science in evaluation of exercises, to identify leadership and experience to build on, and to identify issues or concerns about the use of exercises as a means to performance measurement.

At its fifth meeting, on March 29, 2004, the committee heard presentations about: CDC’s recent efforts in public health preparedness; the modeling workgroup of the Department of Health and Human Services (DHHS) Secretary’s Council on Public Health Preparedness; the theory and science related to preparedness and exercises1 from both a sociological and a disaster management and response perspective; the perspective of a Center for Public Health Preparedness; and the Department of Homeland Security’s experience with planning, conducting, and evaluating exercises. This letter report contains the committee’s findings and recommendations based on information from that meeting, and additional though limited (given time constraints) review of what public health may learn from disaster research and from the practice of disaster response.

1  

The Federal Emergency Management Agency (FEMA) defines exercise as “a focused practice activity that places the participants in a simulated situation requiring them to function in the capacity that would be expected of them in a real event” (FEMA/EMI, 2003).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 REVIEW OF THE CENTERS FOR DISEASE CONTROL AND PREVENTION’S SMALLPOX VACCINATION PROGRAM IMPLEMENTATION Letter Report #6 July 6, 2004 Dr. Julie Gerberding Director Centers for Disease Control and Prevention 1600 Clifton Road, NE Atlanta, GA 30333 Dear Dr. Gerberding: The Institute of Medicine (IOM) Committee on Smallpox Vaccination Program Implementation is pleased to offer you the sixth in a series of brief reports. This report may seem like a departure from the committee’s previous work, which focused on smallpox vaccination as a part of public health preparedness. However, this report responds to a CDC request for guidance as the agency moves toward comprehensive preparedness for bioterrorism and other public health disasters, and toward broad smallpox preparedness efforts. The committee was asked to look specifically at preparedness exercises, which are required by CDC grant guidance and are being conducted by public health agencies. In general, the public health community has somewhat limited experience with exercises, so the committee was asked to describe the state of the science in evaluation of exercises, to identify leadership and experience to build on, and to identify issues or concerns about the use of exercises as a means to performance measurement. At its fifth meeting, on March 29, 2004, the committee heard presentations about: CDC’s recent efforts in public health preparedness; the modeling workgroup of the Department of Health and Human Services (DHHS) Secretary’s Council on Public Health Preparedness; the theory and science related to preparedness and exercises1 from both a sociological and a disaster management and response perspective; the perspective of a Center for Public Health Preparedness; and the Department of Homeland Security’s experience with planning, conducting, and evaluating exercises. This letter report contains the committee’s findings and recommendations based on information from that meeting, and additional though limited (given time constraints) review of what public health may learn from disaster research and from the practice of disaster response. 1   The Federal Emergency Management Agency (FEMA) defines exercise as “a focused practice activity that places the participants in a simulated situation requiring them to function in the capacity that would be expected of them in a real event” (FEMA/EMI, 2003).

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 INTRODUCTION Charge to the Committee One way to measure public health agencies’ performance in achieving preparedness is by performing and evaluating exercises.2 Whereas exercises have been conducted and evaluated in the emergency management field for many years, public health has had less experience with exercises and is currently beginning to assess their value for relationship-building, training, and performance measurement. To place the role of exercises appropriately into the broader definition of what it means to be prepared and to identify specific aspects for which measures can be developed, CDC asked the Committee on Smallpox Vaccination Program Implementation to: Describe the state of the science in exercises and related preparedness strategies; Identify leadership and experience to build upon, from other fields and other federal agencies; and Identify issues or concerns about this approach to performance measurement (Sosin, 2004). To meet the charge presented by CDC, the committee has endeavored to: (1) examine conceptual issues and challenges related to integrating public health into disaster preparedness and response; (2) review some of the evidence base from disaster research and practice that is germane to public health preparedness; (3) learn from the public health response to proxy events; (4) discuss the usefulness of modeling; and (5) discuss the usefulness of exercises, including a description of some of the exercise activities occurring in the federal government. Summary of Recommendations The report’s recommendations revolve around the issues of interagency and intersectoral coordination, learning from experience and research, and continuously improving performance. Recommendation 1: The committee recommends that all federal entities concerned with bioterrorism preparedness (e.g., CDC, the Health Resources and Services Administration, the Office of Domestic Preparedness) should more actively coordinate guidance and funding activities. Federal agencies should also work together to develop mechanisms that facilitate coordination and 2   Initially, the committee’s discussion was concerned with both exercises and drills, as they are related categories along a spectrum of possible activities used for training, performance measurement, etc. However, since drills tend to be very narrowly focused and they typically take place within a single agency, their usefulness is more easily verified. Therefore, they are less relevant to the present broad discussion of preparedness exercises and evidence of their usefulness.  

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 collaboration among their grantees at the state and local levels. Such mechanisms may include, but are not limited to, regular meetings to familiarize CDC and ODP program staff with each other’s program priorities and activities, a database for informing ODP and other partners of exercises planned by CDC grantees, etc. Federal coordination efforts should also include the clarification of primary responsibility and authority in bioterrorism events, to ensure that CDC can fulfill its unique role as the nation’s public health agency. Recommendation 2: The committee recommends that CDC should collaborate with all of its partners to strengthen preparedness by applying research findings and experience in public health emergency response, bioterrorism preparedness, and disaster management. In order to strengthen the evidence base for public health preparedness, CDC should: Strengthen the link between public health research and practice; Participate in and promote interdisciplinary research about preparedness; Support a system to assure the ongoing collection, synthesis, and sharing of lessons learned and best practices from public health preparedness exercises and public health response to proxy events; and In coordination with the appropriate federal-level partners, such as the Agency for Healthcare Research and Quality, evaluate the effectiveness, design, and opportunity costs of preparedness strategies, such as exercises. Recommendation 3: The committee recommends that CDC should use the Evidence-Based Performance Goals for Public Health Disaster Preparedness to develop standards against which CDC, states, and localities may regularly measure their performance in exercises and in response to proxy events. Public health agency performance in exercises and proxy events should be used to identify gaps in preparedness and to improve planning, communication, and coordination at the agency and interagency levels, as part of a process of continuous quality improvement in preparedness planning and response. Preparedness drills and exercises should not be evaluated individually, but their cumulative and long-term impact on preparedness, such as generalizability to other potential hazards, must be considered in the evaluation process

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 INTEGRATING PUBLIC HEALTH INTO DISASTER PREPAREDNESS AND RESPONSE: CONCEPTUAL ISSUES The public health community has become an active partner in the world of emergency and disaster preparedness and response, joining other members in the traditional emergency management and response field that have defined roles and established ways of doing work (Landesman et al., 2001). Although public health workers and agencies have played active roles after many emergency events (and in some states, the emergency medical services [EMS] entity is part of the state public health agency), public health workers have not necessarily counted themselves or been counted among emergency responders (Kahsai and Kare, 2002; Landesman et al., 2001). Some important conceptual issues must be considered in the process of more effectively integrating public health into the disaster preparedness and response field. These issues include (1) the history of public health disaster response, and its relevance to contemporary public health preparedness; (2) the unique role of public health in disasters, and primary role in disasters that involve biological agents; and (3) the heterogeneity which characterizes the field of emergency and disaster preparedness and response. A History of Public Health Disaster Response History provides myriad examples of public health emergencies and disasters (e.g., cholera outbreaks, toxic spills), that wreaked destruction akin to or greater than that of major natural disasters, and to which the evolving discipline of public health responded. Epidemiological and other public health skills and knowledge have also been advanced through lessons learned in such responses (Landesman et al., 2001). The threat of bioterrorism has mobilized the engagement of many disciplines and government agencies both to prevent and to respond. The re-emergence of infectious diseases in part related to demographic change and globalization has elevated interest in public health’s role as both a responder to and a preventer of epidemics and infectious disease outbreaks. Public health agencies have the ongoing responsibility to prevent disease outbreaks and other emergencies through measures such as immunization, sanitation, and community education. In cases where preventive measures are not successful, or there are barriers to their implementation, or an unexpected threat causes disease, public health becomes a responder, conducting surveillance, controlling the spread of disease, conducting mass immunization, etc. At the same time, public health agencies continue prevention, to limit secondary public health problems. The current integration of public health preparedness efforts with those of more traditional “responder” disciplines is based on a growing acknowledgement of public health’s singular capabilities and importance in preparing for and responding to bioterrorism, as well as the health aspects of a range of disasters. These include deliberate attacks with non-biological weapons, natural disasters that may result in the contamination of food or

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 water supplies and lead to infectious diseases, and technological disasters that may endanger population health with radiation or chemical hazards. Unique Role of Public Health in Disasters, and Primary Role In Response to Bioterrorism Public health generally does not have a formal tradition of disaster preparedness and response. However, notable and instructive exceptions are found in the experience of the following types of public health agencies, some of which have developed varying levels of expertise in planning and exercising for disasters and in managing disasters (e.g., the experience of the state of Georgia described by Werner et al., 1998): Public health agencies located in the vicinity of nuclear facilities and involved in federally-mandated training and exercise programs; Public health agencies located in areas with frequent natural disasters (hurricanes, floods, or tornadoes); Public health agencies at sites of one-time or recurring major events or entertainment venues (e.g., auto racing, Olympics, amusement parks); and State public health agencies in states where emergency medical services (EMS) are integrated into the public health agency. The role of public health in disaster preparedness and response is unique, and is not performed by any of the other disciplines that typically respond to disasters and which differ from public health in mission, services provided, and personnel training (e.g., emergency medical services, clinical medicine). Therefore, the role of public health as a responder needs to be formalized and become an indispensable and recognizable part of comprehensive response to disasters. One common thread characterizes the work of public health agencies in relation to most types of disasters: they possess the knowledge and skills required to safeguard the health of the public by limiting morbidity and mortality, whether an event poses a threat to health from the outset (i.e., bioterrorism), or creates secondary threats to health, as in the case of natural disasters. The public health community’s role before, during, and after the occurrence of disasters is to some extent anchored in its capacity to conduct routine, non-crisis activities, and is consistent with public health’s assessment, policy development, and assurance functions, but varies with community resources and interagency agreements, and service provision roles (Salinsky, 2002). Carrying out these functions requires public health agencies to collect, evaluate, and disseminate information; cooperate and collaborate with other disciplines (including, but not limited to, the health care sector); and to prevent disease and ensure the continuity of health care (Landesman et al., 2001; IOM, 2003d). In addition to the public health impacts of most other types of disasters, attacks with biological agents, as exemplified by the anthrax attacks of 2001, require that governmental public health agencies serve as primary responders. Events that involve biological agents are different from other types of disasters because their emergence is likely to go unnoticed for some time; biological agents are microscopic and may be more likely to be introduced silently (e.g., through airborne droplets) rather than with

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 explosions, and become evident over time. Also, the fallout from attacks with biological agents may not remain confined to a specific physical space, in other words, there may not be a “scene” or a “ground zero” (Perry, 2003) and their impact may not be contained, but may ripple outward for some time due to contagion. Preparedness for biological agents therefore involves at least some different requirements from other types of agents, and requires the unique knowledge and skills of trained public health personnel (e.g., case identification and containment), and the unique capabilities (e.g., laboratories, surveillance, communication, community education) and statutory responsibilities (e.g., quarantine) of public health agencies, as well as the complementary facilities, skills, and resources of the health care community (Perry, 2003). In order to integrate the preparedness and response efforts of public health most appropriately with those of the traditional emergency management and response field, some key differences need to be identified. For example, disaster preparedness and response is the central mission of local, state and national civilian and military response organizations and they train and exercise regularly to test and maintain their response capabilities. They have the dual role of responding to disasters and to routine emergencies in their communities. For public health agencies, responding to major crises has been the exception from their usual work, therefore, conducting regular drills and training to prepare for disaster response has generally not been a common practice. Also, even when public health agencies have gained experience dealing with disease outbreaks, these events do not typically reach the scale of a disaster, and response is largely limited to the public health and health care communities. Figure 1: Contrasting roles of public health and traditional responders in bioterrorism and other disasters. Given the statutory responsibilities and special capabilities of public health, and CDC’s leadership role in the provision of essential public health services under all circumstances, it is clear that CDC and the public health community must be ready to fulfill their primary role in responding to bioterrorism, and support roles in other types of disasters, including terrorism with chemical, nuclear and other types of weapons (see Figure 1). The Diverse Field of Emergency and Disaster Preparedness and Response Public health is not entering a monolithic or homogeneous field of emergency and disaster management. Disasters involve people, physical structures, and the broader environment, and they may be caused by a wide range of natural, technological, and deliberately introduced agents. This variety of factors explains the complex array of disciplines and organizations involved in the emergency and disaster response field. The category of first responders has typically included personnel from the firefighting, emergency medical services, and law enforcement fields, along with state emergency

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 management agencies and federal agencies (e.g., Federal Emergency Management Agency, Environmental Protection Agency), and non-governmental organizations, such as the Red Cross and the Salvation Army. Other disciplines involved in preparedness include structural engineers, civic planners, public administrators, etc. Clearly, the set of contributors to emergency and disaster preparedness and response is vast and includes a patchwork of methods, cultures, and disciplines which are in some cases themselves struggling to integrate their activities (Tang and Fabbri, 2003; Kahsai and Kare, 2002). In addition to being multidisciplinary, the field of emergency and disaster preparedness and response is undergoing change toward increased professionalization and an all-hazards3 approach, and evaluating its assumptions and modes of practice, as discussed elsewhere in this report (NRC, 2003; Alexander, 2003). CHALLENGES AND OPPORTUNITIES INHERENT IN INTEGRATING PUBLIC HEALTH INTO A BROADER FIELD The integration of a relative newcomer into the large and complex field of emergency and disaster preparedness and response presents challenges and tensions. Disasters require rapid decisions and quick action, which may bring about cross-jurisdictional conflicts, professional differences, and questions about authority, expertise, and the appropriate chain of command. Coordination Issues In its fifth report (IOM, 2003a), the committee discussed at some length the importance of close collaboration between the public health and health care communities, from the level of federal agencies such as HRSA and CDC, to local public health agencies and their health care counterparts (health care organizations, hospitals, private providers, long-term care facilities, etc.). Previous reports by this committee have also called for public health and health care organizations and workers to coordinate and collaborate with agencies, disciplines, and entities with which they were previously not well acquainted, including, but not limited to fire authorities, law enforcement, emergency medical services, voluntary organizations, and communities. Research and practical experience show that coordination among all agencies involved is one of the fundamental requirements of effective disaster response, and that the lack of adequate coordination is one of the major problems encountered in the field (Tierney et al., 2001; Auf der Heide, 1989). Given the large number of federal, state, and local agencies involved in preparedness efforts, establishing adequate coordination across federal, state, and local levels is proving to be a challenge (Clements and Evans, 2004; Advisory Panel to Assess Domestic Response Capabilities for Terrorism Involving Weapons of Mass Destruction, 2003; GAO, 2003a). Within the federal government, preparedness and response activities are coordinated through the Department of 3   The term “all-hazards” refers to the full spectrum of causes of disasters, which now includes not only natural and technological, but also deliberate, i.e., terrorist-induced (Landesman et al., 2001).

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 Homeland Security (DHS). Coordination at the top levels of the federal government occurs through the Homeland Security Council (HSC), which is charged with ensuring coordination of all homeland security-related activities among executive departments and agencies and promoting the effective development and implementation of all homeland security policies (White House, 2001). Day-to-day coordination of homeland security issues—both within the federal government and among federal, state, and local government agencies—is meant to occur through the Policy Coordination Committees (PCCs) of the HSC (White House, 2001). There are eleven Policy Coordination Committees for different functional areas, including a Medical and Public Health Preparedness PCC. The committee was unable to obtain sufficient information to determine whether and how Medical and Public Health Preparedness PCC actions or policy decisions shape CDC’s preparedness program and whether the PCC plays a role in strengthening CDC’s relationship with DHS. Despite the existence of mechanisms for coordination at the top departmental level, such as the PCCs, it is not evident to the committee that adequate coordination and information-sharing are occurring formally at the level of federal program staff involved in the day-to-day work of public health preparedness (GAO, 2003b). Although the creation of DHS holds the promise of streamlined oversight and funding, there are concerns that coordination between DHS and key preparedness functions in DHHS remains a significant challenge (GAO, 2003a). At the committee’s March 2004 meeting, conversation among presenters from federal agencies and the committee revealed that personal relationships and serendipity may be credited with some coordination and information-sharing across agencies, but it was not immediately evident that there are sufficient and functioning formal mechanisms for coordination and collaboration between DHS and DHHS. Coordination must be planned with forethought and deliberation, not left simply to chance and the goodwill of program staff. Coordination must also be planned and implemented during the preparedness or pre-event phase, beginning with effective communication about funding objectives and activities. For example, it is important for CDC staff to be familiar with relevant activities occurring in DHS and its programs funded and/or administered through FEMA (now in DHS) and ODP, and for DHS staff to be aware of CDC priorities and activities, to ensure the best use of limited federal preparedness resources. State and local public health agencies receive funding through CDC’s Cooperative Agreement on Public Health Preparedness and Response for Bioterrorism and health care entities are funded through HRSA’s National Hospital Bioterrorism Preparedness Cooperative Agreements. These cooperative agreement programs require that grantees conduct exercises that test public health and health care preparedness (and the integration between them) for an attack with biological or chemical agents. Through the DHS Office of Domestic Preparedness Fiscal Year (FY) 2004 Homeland Security Grant Program and FY 2004 Urban Area Security Initiative Grant Program, states and some local emergency management offices receive funding to conduct exercises that test many of the same capacities and interagency collaborations expected by HRSA and CDC (DHS, 2003). Furthermore, FEMA, which is now under DHS although its activities seem not yet fully coordinated with those of ODP, also oversees exercises relevant to chemical

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 and radiation emergencies, which include public health components. The committee learned that sometimes states pool different sets of resources to conduct a larger drill or exercise involving a larger number of state and local agencies and community partners, and in other cases, the different funding streams are used to fund separate exercises (Schweitzer, 2004). ODP has released guidelines for exercises and their evaluation through the Homeland Security Exercise and Evaluation Program (HSEEP). Although the committee is not aware of the nature and extent of CDC’s involvement in the development of the HSEEP guidelines, the committee believes it is important that both CDC and DHS/ODP work to ensure a reasonable level of compatibility and coordination. This is necessary because of the functional overlap between public health and other state agencies, and because some state public health agencies already plan and execute their bioterrorism preparedness exercises in conjunction with their state emergency management offices. While public health preparedness exercises are needed to assess the unique functions and goals of public health, they will ideally be coordinated with other types of exercises, where appropriate. Since state emergency management offices will be following the HSEEP guidelines, and some state public health agencies may be participating in exercises that follow these guidelines, a certain level of coordination is necessary between CDC’s public health preparedness exercise guidelines and the HSEEP guidelines. In order to maximize the knowledge, skills, and relationship-building that states and local jurisdictions gain from participating in preparedness exercises supported by limited federal resources, the committee encourages CDC to work closely with ODP (as well as HRSA) to coordinate, where appropriate and consistent with agency goals, the funding and guidelines for exercises provided by all federal agencies to states, local jurisdictions, and to private sector entities, such as hospitals. Responding to a public health disaster, such as a smallpox attack, will require coordination with other organizations in the private sector and within the health care community. At the March 2004 meeting, the committee heard about the initiatives of the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) to engage communities in preparedness planning and exercises. The committee believes it is important that CDC identify other organizations which, like JCAHO, require and set standards for preparedness activities including exercises, and interact with communities in the area of bioterrorism and disaster preparedness. This is needed to help avoid duplicative efforts as well as ensure the best coordination of preparedness efforts. The range of partners in preparedness should be conceived broadly, to include local community, health care institutions, voluntary organizations, and others. The committee also heard that state grantees funded by the DHS ODP FY 2004 Homeland Security Grant Program are encouraged to share exercise calendars with other partners and to coordinate or integrate efforts with other state and local exercises (Schweitzer, 2004). The committee suggests that CDC develop and maintain a list or database of exercises funded under the current (and future) cycle of the Cooperative Agreement on Public Health Preparedness and Response for Bioterrorism and to share this resource with ODP. Also, regular communication between CDC and ODP would

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 inform both about planned exercises, and would provide opportunities for coordination of exercises within a state and between states. CDC and DHS guidance to grantees makes some reference to the need for interdisciplinary and intersectoral coordination (CDC, 2003, 2004a, 2004b; DHS, 2003; CDC and ODP, 2003). However, it is not clearly spelled out how these linkages function at the federal, state, and local levels, and it is unclear whether the need for coordination is more specifically confirmed with and reinforced with grantees. For example, the CDC guidance for FY 2004 calls for integrating efforts and closely coordinating with “activities funded by the Department of Homeland Security and/or other federal agencies” (CDC, 2004a, 2004b). The guidance does not specifically identify relevant programs funded by DHS, and the committee was unable to locate more detailed explication of the formal linkages and coordination mechanisms that exist or are desirable in the relationship between CDC and DHS grantees. The committee was unable to find a comprehensive resource describing all of the funding streams available for emergency preparedness activities, their purpose, funding amounts, and intended recipients. Such a tool would aid coordination of funding at the state and local level, and would also facilitate coordination of all-hazards preparedness activities among national, state, and local partners in the academic, non-profit, and business sectors. The committee did find a useful matrix of federal all-hazards grants from the International Association of Emergency Managers (IAEM, 2003). If a similar federal resource exists, that has been verified for accuracy and timeliness by the relevant federal agencies, the committee encourages that such a document be shared widely to facilitate coordination among all participants in emergency preparedness. The committee urges all federal agencies to plan for and implement adequate collaboration and communication to ensure the long-term sustainability and effectiveness of an interagency approach to funding, developing, implementing, and evaluating public health preparedness in general, and exercises in particular. An Example of Intersectoral Tension and Collaboration The relationship between public health and law enforcement in responding to bioterrorism illustrates some of the potential tensions inherent in the coming together of different cultures and approaches to address a crisis. An attack with biological agents would put into motion two major and divergent systems (in addition to many others): public health, which attempts to deal with consequences and spread of infectious disease, and law enforcement, which targets the commission of a crime implicit in a deliberate introduction of a biological agent. In the anthrax attacks of 2001, differences between public health and law enforcement became apparent. These included different investigative approaches (inductive versus deductive, respectively), evidentiary standards (scientific versus legal), and communication objectives. Public health tried to share complete and accurate information with the public

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 in a timely manner, while law enforcement sought to disclose little or nothing pertaining to an investigation in order to maintain the integrity of a potential legal case (Butler et al., 2002; Ornstein, 2001; Gerber, 2002). Given these very different objectives and approaches, bioterrorism events would challenge each set of responders to do its own work while allowing the other to carry out its responsibilities. Preparedness efforts must include discussion and clarification of roles and responsibilities in a way that meets the needs of both public health and law enforcement professionals and undermines neither the disease prevention goal of public health nor the evidentiary standard required by law enforcement (Butler et al., 2002; Richards, 2002). In the wake of the anthrax attacks of 2001, the position of CDC liaison to the FBI was created (GAO, 2003c; Butler et al., 2002). This seems to be a step in the right direction, but it would be useful only as long as the liaison unit is considered a priority by both agencies, and it is given an adequate scope of work and level of authority. The Forensic Epidemiology training program, a joint effort between CDC and Department of Justice to facilitate mutual understanding between law enforcement and public health is an example of successful and productive collaboration between public health and law enforcement in the area of bioterrorism preparedness and response (CDC, 2003b). Collaboration between seemingly disparate government agencies and disciplines is not a new need, and there is some history on which to draw to help clarify and streamline these relationships. In the early 1980's, the CDC and the FBI created an interagency bioterrorism unit, located at CDC with secure communication capacity in the wake of a botulinum hoax. Plans were developed for the defense of the civilian population in the event of a bioterrorism incident. This unit was later disbanded (Foege W., personal communication, March 30, 2004). This is an example of the type of collaboration that must be initiated and sustained to help address deliberate threats with health consequences. Common Definitions and Terminology Are Needed The emergency and disaster management field and federal agencies associated with it have developed a great deal of experience planning for disaster response and designing and conducting exercises to promote relationship-building and training (Landesman, 2001; GAO, 2001; FEMA, 2003; Kuhr and Hauer, 2001). As disaster response becomes increasingly interdisciplinary, a common language is needed for good communication and interagency coordination in preparing for and responding to a chemical, biological, radiological, nuclear, or explosive incident. The committee found that similar terms do not always have the same meaning in documents created by different federal agencies (e.g., CDC, HRSA, FEMA and ODP) or in the way they are used by the many disciplines conducting disaster research (CBACI, 2002; Hilhorst, 2003). The terms “exercise,” “drill,” and “simulation” in particular can mean different things to different agencies and disciplines.

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 For models to have utility, sensitivity analyses should be conducted for each input parameter, to assess which factor has the greatest impact on the outcome or outcomes of interest. Once it is determined how sensitive the outcome is to the different input parameters, preparedness efforts can be focused on the factor that is estimated to have the greatest impact on the outcome. This also will help policymakers determine the factors for which indicators should be developed. Considering the limited resources currently available for public health preparedness activities, knowing which factors potentially have the greatest influence on the course of an outbreak will be extremely valuable to those who decide how to allocate limited resources. Without sensitivity analyses, models are of limited value to policymakers. Role of Modeling in Exercise Development Modeling can help inform planning for exercises by elucidating the critical factors that affect the outcome, which in turn helps in designing exercises that stress that particular part of the system. For an exercise testing the response to smallpox, findings from some of the models described below could help inform the focus that should be placed on vaccination of the public compared to contact tracing and containment, the number of staff in mass vaccination clinics, and the need for post-event vaccination of healthcare workers. A number of models created in the past few years have examined the potential spread of smallpox under varying scenarios (Meltzer et al., 2001; Kaplan et al., 2003; Halloran et al., 2002; Bozzette et al., 2003). These models used slightly different assumptions for most of the key input parameters, resulting in different conclusions. Meltzer et al., found from their model that a combination of quarantine and vaccination would be the best option for stopping a smallpox epidemic (Meltzer et al., 2001). Mass vaccination is found to be the best option for limiting mortality and reducing the time it takes to end a smallpox epidemic in the model produced by Kaplan et al. (2003). In the Halloran et al., model, mass vaccination could produce better outcomes than targeted vaccination, and vice versa, depending upon specific components of the particular outbreak, such as preexisting immunity, rate of transmission to contacts, and vaccine supply (Halloran et al., 2002). Bozzette et al. found that the net benefits of vaccination depend upon the probability of an attack, with prior vaccination of health care workers being favored unless the probability of an attack is very low, and mass vaccination being favored only if the probability of a national attack or multiple attacks is high (Bozzette et al., 2003). Ferguson et al., (2003) compared these four different models and offered reasons why the conclusions that could be drawn from each model were different. Models have been used to examine different aspects of a smallpox outbreak. Some models have examined the speed of different components of a response to a bioterrorism incident, and how this affects outcomes (Giovachino and Carey, 2001; Hupert et al., 2002). Other models have examined an individual’s risk-benefit profile for pre-or post-exposure smallpox vaccination (Meltzer, 2003), estimates of historical

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 transmission rates (Gani and Leach, 2001), the course of historical smallpox epidemics (Duncan et al., 1994; Eichner and Dietz, 2003), and the effect of isolation of overt cases of smallpox and surveillance of contacts on the progression of a smallpox epidemic (Eichner and Dietz, 2003). Each of these models has utility in examining particular aspects of a smallpox outbreak and the corresponding response options, but before a model is used to help make important decisions about a smallpox exercise or a smallpox response, the model’s assumptions and input parameters must be deemed reliable and realistic. This is also another important reason why sensitivity analyses are necessary. They are used to study the effect of varying the range of assumptions that have been made. This is the importance of the model—not its overall conclusion. There also may be opportunities to learn from other modeling efforts. The military has extensive experience with modeling, potentially providing a rich knowledge base that could aid smallpox modeling. Knowledge gained from modeling other communicable diseases (e.g., measles) could also inform the population dynamics and transmission aspect of modeling smallpox, West Nile virus, and SARS outbreaks, in particular. Recent efforts to model an intentional release of anthrax could shed light on factors that are unique to modeling a bioterrorism event. The committee encourages CDC to draw upon the knowledge and experience of other modeling efforts when developing models for smallpox or any other biological agent. Smallpox Modeling Working Group Recognizing the potential value of modeling in informing policy decisions, the DHHS Secretary’s Council on Public Health Preparedness recently formed a Smallpox Modeling Working Group (Borio, 2004). The Smallpox Modeling Working Group was created to “explore a range of policy options related to smallpox preparedness and response” (Borio, 2004). Three modeling groups were selected to model the effects of different response strategies. To overcome some of the reasons for differing conclusions of previous models (Ferguson et al., 2003), the Smallpox Modeling Working Group decided that a standardized set of biologically realistic input parameters for smallpox natural history and transmission needed to be agreed upon (Borio, 2004). In addition to the standardized input parameters, the working group also developed outbreak scenarios, policy options regarding outbreak containment measures, and outcome measures of interest to DHHS (Borio, 2004). The three modeling groups ran their models using the agreed upon parameters. Based on the results of these models, the Smallpox Modeling Working Group reached the following interim conclusions: “Surveillance and containment alone is sufficient to effectively contain an intentional smallpox release.

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 There is relatively small marginal benefit in pre-vaccination of hospital workers or mass vaccination of the population after an outbreak begins. Reactive mass vaccination may have additional value in bringing an epidemic under control. In the absence of any interventions, the strongest controlling factor is people withdrawing to the home when they become ill” (Borio, 2004). As was mentioned earlier, models can offer illustrative guidance as to the factors which have the greatest influence on the outcomes of interest, but models have their limitations and should not be used alone for making policy decisions. Of the three models created under the aegis of the Smallpox Modeling Working Group, one is a deterministic model since it uses single point estimates for each of the input parameters, whereas the other two are stochastic models, using probability estimates for the different input parameters (Borio, 2004:52, 61-62). Since recent data on smallpox transmission rates, incubation period, case fatality rate, vaccine efficacy, vaccination adverse event rates, and population dynamics of the current U.S. population are limited, stochastic models may be more illustrative of the range of outcomes that are possible due to a smallpox outbreak. However, to accurately portray the role of modeling in policy decision-making, the sensitivity of particular input parameters on the model’s outcomes must be provided, and the limitations of the model and uncertainties in the data must be conveyed (Ferguson et al., 2003). USEFULNESS OF EXERCISES The Use of and Rationale for Exercises Exercises are believed to be effective in enhancing preparedness, and are widely utilized by local, state, and national disaster response agencies (GAO, 2001). The emergency and disaster response field’s assumptions that exercises work to improve preparedness have been reinforced by experience that has suggested a link between exercises and good performance in an emergency or disaster (FEMA/EMI, 2003). According to FEMA’s Emergency Management Institute (FEMA/EMI, 2003), exercising reveals flaws in planning, clarifies roles, improves individual performance, and tests and evaluates plans, policies and procedures. Moreover, exercises have become an institutionalized strategy for planning in homeland security. In fact, the DHS HSEEP materials assert that exercises “provide a risk-free environment for jurisdictions to assess if they have the plans, policies, procedures, resources, and agreements in place to enable homeland security personnel to perform critical tasks required to prevent, respond to, or recover from a terrorist attack” (DHS/ODP, 2003:iii). Exercises contribute to preparedness by fostering relationship-building; by providing a context and tool for training; and by providing a method for evaluating performance. The use of exercises for training may originate in the military experience, but they are conducted as part of preparedness in a variety of contexts, from the nuclear plants required by the Department of Energy to use exercises to prepare for the possibility

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 of an accident, to local firefighters training to deal with major fires or with natural disasters, to regional exercises required by FEMA to respond to hazardous materials (HazMat) and natural disasters. When exercises are conducted in order to educate, train, or develop interorganizational and interjurisdictional relationships, the underlying assumptions may be easily validated. Disasters are complex events that require many different types of responders, therefore, having partnerships is preferable to working in isolation. Furthermore, some level of organization and coordination is essential to help avoid chaos; rehearsing processes may lead to smoother functioning of complex response systems, and in the event of an emergency, for example, a smallpox attack, having personnel that possess certain knowledge and skills (e.g., smallpox diagnosis, vaccination, and search and containment) is better than having personnel that did not receive such education and training. Exercises which test communication across jurisdictions or test certain skills and processes may provide some indication that certain things are likely to work well in a disaster. Research on Exercises Although the assumptions made about exercises are reasonable, and exercises seem like a practical strategy in many circumstances, there are at least two reasons to seek more objective study: the need to compare multiple types of exercises for more targeted use, and the potential costs posed by exercises. As noted above, more research is needed on preparedness (Tierney et al., 2001), but exercises and other means for evaluating performance (and for improving preparedness) form a particularly neglected subset of preparedness. The overall effectiveness of exercises as a preparedness strategy has not been well demonstrated, and research is needed to determine, for example, whether exercises could be considered predictors of successful response, what type of exercise would have high impact on preparedness, what exercises are most cost-effective, and the best way to assess opportunity costs posed by conducting exercises (NRC, 2003). The use of scenarios, which may serve as a component of exercises, for training and planning purposes has not been well-researched either, but there are “indications that they are an excellent method of teaching rapid response-style thinking, decision-making and the development of managerial skills” (Simpson, 2002:56; Alexander, 2000). Potential research questions would include: how do reality-based scenarios compare with entirely fictional scenarios, and under what circumstances would the use of one be preferable to the other? In the disaster literature, mention of exercises seems limited to descriptions of how they were utilized by responder agencies and disciplines (EMS, emergency departments, fire departments, etc.), the lessons learned, and changes in operations made as a result (Tierney et al., 2001). A brief review of the medical and health peer-reviewed literature (using the National Library of Medicine’s PubMed search engine) shows that hospitals and public health agencies conduct exercises and find them useful in evaluating

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 the quality of training, the smoothness of emergency operations, and other aspects of disaster response, but there seems to be little or no empirical study of the validity or effectiveness of exercises themselves as a strategy for public health and health care preparedness. The Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center recently conducted a review of the literature on hospital exercises and drills, and concluded that “the evidence was insufficient to support firm conclusions about the effectiveness of specific training methods because of the marked heterogeneity of studies, weaknesses in study design, and the limited number of exercises that have been reported in the literature” (AHRQ, 2004:2). The committee hopes that the experience of public health agencies with exercises and drills will not only be reported to CDC, but that there will be increased emphasis on more in-depth studies of the effectiveness of public health preparedness exercises, and more frequent publication of such studies in the literature. The growing partnerships between public health agencies and schools of public health (i.e., the Academic Centers for Public Health Preparedness) would certainly contribute to such an effort. Figure 2. HSEEP’s Building Block Approach to Planning and Training, Source: DHS/ODP, 2004b. At the committee’s March 2004, meeting, DHS ODP speakers presented a graphic describing a cyclical or building block approach to planning and training (see Figure 2). This graphic outlines an incremental set of techniques or methods for preparedness planning and training, from the minimal complexity of a seminar to the significant complexity of functional or full-scale approaches. On the one hand, this building-block approach seems logical, and it is reasonable to select a training and capability testing method based on and scaled to match the complexity of the objective (e.g., a workshop to train for a particular role and then to test it; a functional exercise to rehearse and evaluate a complex interagency process). However, the committee believes this approach requires evaluation. For example, by what means are games determined to be more or less complex than tabletops, and in what circumstances is one method preferable to another? This type of evaluation research would help ensure the implementation of the most effective, evidence-based means for strengthening preparedness. Exercise-Related Activities of the Department of Homeland Security (DHS) To help ensure that the country is prepared for a possible terrorist attack or other emergency, the Department of Homeland Security has primary responsibility in the federal government for organizing and evaluating preparedness exercises and drills. In DHS, the Office for Domestic Preparedness (ODP) is responsible for providing training, funds for the purchase of equipment, support for the planning and execution of exercises,

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 technical assistance, and other support to assists states and local jurisdictions in preventing, planning for, and responding to acts of terrorism (DHS, 2004a). In addition to ODP, FEMA has also been incorporated into DHS, and it has brought over its expertise in the area of exercises. Together, ODP and FEMA are responsible for the Homeland Security Exercise and Evaluation Program (HSEEP), Radiological Emergency Preparedness (REP) Program, Community Hazards Emergency Response Capability Assurance Program (CHER-CAP), and the Chemical Stockpile Emergency Preparedness Program (CSEPP). These programs and others offer a rich history of exercise experience that could inform public health preparedness. HSEEP, mentioned in preceding pages, details DHS’s comprehensive exercise doctrine, and is “a program of financial and direct support designed to assist state and local governments with the development and implementation of a state exercise and evaluation program to assess and enhance domestic preparedness” (DHS/ODP, 2004a:9). HSEEP resources include four volumes of reference materials to assist state and local jurisdictions with the design, development, conduct, and evaluation of exercises. The first volume of HSEEP includes a uniform approach for exercise design, development, conduct, and evaluation (DHS/ODP, 2004a). The second volume includes a methodology for conducting evaluation of homeland security exercises and implementing an improvement program (DHS, 2004b). The third volume is an exercise development manual, outlining a standardized planning process, adaptable to any type of exercise or scenario (Schweitzer, 2004). The fourth volume consists of sample exercise documents (DHS/ODP, 2004a). As the committee learned at its March meeting, FEMA has extensive experience with executing and evaluating exercises as part of the agency’s mission to prepare the country for disasters, and its work contributes to the practical knowledge base on exercises and drills (Kelkenberg, 2004; FEMA/EMI, 1995; FEMA, 2002). For example, FEMA has been involved with the REP program, which was established as a direct result of the Three-Mile Island incident, and ensures adequacy of emergency plans and preparedness for areas near commercial nuclear power plants (Kelkenberg, 2004). Whereas the Nuclear Regulatory Commission is responsible for ensuring the adequacy of emergency plans onsite, FEMA is responsible for reviewing and evaluating offsite radiological emergency response plans developed by state and local governments and for evaluating exercises conducted by state and local governments to determine if radiological emergency plans can be implemented (Kelkenberg, 2004). FEMA also plays the significant role of educating the public about radiological emergency preparedness (Kelkenberg, 2004:228). FEMA has a memorandum of agreement with the U.S. Army for the CSEPP program. Similar to REP, the Army is responsible for onsite preparedness and response, and FEMA is responsible for offsite preparedness and response in the surrounding community (Kelkenberg, 2004:235-236). Some useful documents have been created through this program that may be applicable to other preparedness programs. For example, Oak Ridge National Laboratories has developed training materials on sheltering in place that would be applicable to other emergency preparedness scenarios

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 (Kelkenberg, 2004:236-237). CSEPP also includes an exercise component, which involves both the site itself and the surrounding community (e.g., emergency management agency, health department, hospitals). FEMA evaluates the local community’s performance in the exercise and, based on the findings, recommends how the community partners’ emergency response plans should be improved (Kelkenberg, 2004:237-240). FEMA’s Community Hazards Emergency Response Capability Assurance Program (CHER-CAP) consists of a planning, training assessment, and exercise process for all-hazards response operations (Kelkenberg, 2004). It consists of looking at a community’s emergency response plans, coupling the plan with a risk assessment, identifying training to fill in the gaps in the plan, and then doing a table-top exercise and peer-evaluated full-scale exercise to identify areas for improvement (Kelkenberg, 2004:226-227). Sample Questions, Strategies, and Methodologies for Evaluation Research on Public Health Preparedness Exercises and Proxy Events The committee has identified several possible questions, strategies, and methodologies that could be considered by CDC in evaluation and experimental research in public health preparedness. These include: Examining the impact of a proxy event on two similar communities with different public health infrastructure and capabilities. Conducting “placebo-controlled” trials comparing the response of two similar communities to a proxy event, false alarm, or to an unannounced exercise. One community previously conducted one or more exercises (or employed other methods) to test its preparedness, while the other did not. Compare response times, smoothness of interagency coordination, functionality of communication channels, and other aspects of their response. Conducting an unannounced exercise and compare the performance of three different groups of personnel (with the same qualifications and functional roles) who have undergone one of the following: (1) preparedness-related training only; (2) training and a table top exercise only; (3) training and participation in a comprehensive functional exercise. Randomly assigning educational materials to various types of health personnel to determine whether the type and quality of educational materials have an impact on exercise outcomes. Randomizing the method of preparing responders and the community at large before an exercise is conducted in order to determine the best way to conduct them. Outcomes to be measured would include professional participation rates, community participation, and major desired outcomes of the activity.

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 The committee also suggests several areas for further study: A systematic assessment of all lessons learned in the course of the smallpox vaccination program (which could be considered a national-level, multi-site, months-long preparedness meta-exercise); Determine what knowledge is available about public health preparedness and about conducting exercises (and drills) in public health agencies with experience in this area (history of working with EMS, preparing for nuclear accidents, etc.). Many lines of questioning could be followed in gathering information from public health agencies with a variety of linkages to emergency and disaster response. For example, it may be instructive to compare the engagement of public health in emergency and disaster response in states where EMS is part of public health to states where the two agencies are separate. Systematically assess the lessons learned by state and local public health agencies (perhaps organized by type according to characteristics such as size, urban or rural location, structure and governance) that have conducted exercises and drills. Such an assessment must also include an examination of any evidence that is available or is becoming available, in the literature or in the reporting of public health agencies, of the effectiveness of exercises and drills conducted by public health agencies (similar to AHRQ’s recent work [AHRQ, 2004]). A systematic and comprehensive research agenda for studying the response to public health emergencies and disasters be developed; and An evaluation of patient safety literature to consider the pre- and post- handling of sentinel events. A Framework for Performance Evaluation Using Exercises Major outcomes in public health typically involve decreasing mortality and disease rates and progress is measured periodically (e.g., Healthy People 2010 process). Performance measurement in public health is, however, a relatively new field. In the case of public health preparedness for bioterrorism and other events with significant public health impact, outcomes are occasioned by actual events themselves, and the infrequency and huge variation among these events (including the proxy events discussed in preceding pages of this report) make it difficult or nearly impossible to gauge, for example, a decrease in rate of disease from contaminated water, or other reductions in mortality and morbidity attributable to the disaster. Due to the nature of disaster-related public health problems, performance measurement in this area is by necessity more process-oriented. When CDC and its state and local partners identify exercise objectives that will be used in evaluating the exercise, these objectives will be most helpful if they are linked with the Evidence-Based Performance Goals for Public Health Disaster Preparedness developed by CDC. Exercises offer an alternate way to measure performance and fine-tune preparedness before a crisis occurs. Public health preparedness exercises take place at national, state, and local levels, and it is important that evaluation of exercises take place at all levels. The committee believes it is essential to design and conduct exercises that

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 stress and test CDC’s own performance. As noted in the preceding discussion of proxy events, CDC is a vital part of preparedness and response and it is itself a limiting factor in terms of the resources it provides (e.g., laboratory reagents, information, technical assistance) to state and local counterparts. In asking “what if” questions in a proxy event or in an exercise, the limits of availability of such resources must be probed. In addition, modeling could be used to estimate such things as the rate of producing and renewing the supply of needed laboratory reagents, or the speed with which needed field experts could be moved from place to place. In a more dramatic type of exercise, questions could be asked about the potential impact if CDC itself was the target of an attack and critical facilities destroyed. After action reports will play an important role in facilitating continuous quality improvement. They provide an overview of agency or interagency performance in an exercise and identify areas where there are gaps in planning, unforeseen circumstances which are poorly managed, or areas where communication or the flow of information break down, among other issues. Various types of methods for measuring performance will eventually be determined to be effective and even to have some predictive value (e.g., of future successful response). The link between research and practice requires strengthening, so that as research validates certain practices, such as types of exercises, and the most effective techniques to communicate to or evacuate the public, they may be rapidly translated into practice. The practices demonstrated to be most effective (e.g., specific types of exercises) need to then be institutionalized and adapted to local circumstances, with particular attention to maintaining and updating staff competence and sustaining readiness. Staff turnover itself, which requires regularly updating training and conducting exercises, could be used to create new cohorts for performance evaluation. CDC might wish to consider describing the range and breadth of exercises needed for public health. The HSEEP Building Block Approach illustrates one typology of training and capacity-building methods, including exercises. CDC could develop a similar representation with specific applications to public health. For example, in the area of exercises, some exercises may be external, conducted in coordination with other agencies at the federal, state, and local level (refer to the section on Coordination Issues), while others will be strictly internal exercises on such issues as how to move from normal to emergency operations, including decisions about closing or curtailing planned clinics, outreach, or investigation; decisions about and use of personal protective equipment under various circumstances; establishment of databases for unexpected investigations or unusual outbreaks. One of the challenges in developing and implementing exercises is to make the mock disaster approximate as closely as possible a real-life one, including as much complexity and unpredictability as possible, and basing scenarios on what is likely to happen according to the microbiological, immunological, epidemiological, and disaster literature, not on myths or on widely embraced assumptions.

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 Ensuring Compatibility between the DHS Exercise Doctrine and Public Health Preparedness Exercises The DHS Homeland Security Exercise and Evaluation Program describes a yearly cycle of planning and development, followed by training, exercises, and the development and implementation of an improvement plan. The committee has learned that CDC intends to implement a similar cyclical process (target goals → exercise → target goals, etc.) with its grantees (Sosin, 2004). The goals of public health preparedness are a distinct subset of overall preparedness, and public health, as noted elsewhere in this report, has its unique capabilities, responsibilities, and information needs. The use of exercises to measure performance and public health preparedness will differ from their use in other fields in the processes being evaluated, in the skills and knowledge being assessed, in the specific relationships and coordination being tested. However, there will be areas of overlap with other disciplines and programs, and there will be some commonalities in structure and operations (e.g., a type of emergency operations center and/or other mechanisms for interagency collaboration and coordination, communication activities, information infrastructure). It is important to ensure that planning, conduct, and evaluation of public health exercises at the federal, state, and local levels are compatible with those of DHS activities under the HSEEP. For example, HSEEP describes three levels of performance evaluation: task level performance (individual); agency/discipline/function-level performance; and mission-level performance (interagency, inter-organizational, and community) (DHS/ODP, 2003). The HSEEP Exercise Evaluation Guides for table top and operational exercises include public health personnel/agencies under the “response element” heading in addition to EMS, law enforcement, fire department, HazMat, hospitals, and others, and though the exercise methodology indicates that public health is one of the agencies/disciplines/functions to be evaluated in HSEEP, it understandably does not go into detail. If CDC intends to coordinate with or make its public health exercise evaluation compatible with the HSEEP model, the committee suggests that existing resources, such as the Public Health Competencies for Bioterrorism and Emergency Preparedness be utilized in customizing the individual-level evaluation and that the Local and State Public Health Preparedness and Response Capacity Inventories be included in customizing the agency-level evaluation. In preceding pages, the committee has explored the potential of proxy events and exercises as means to performance measurement. The committee recommends that CDC should use the Evidence-Based Performance Goals for Public Health Disaster Preparedness to develop standards against which CDC, states, and localities may regularly measure their performance in exercises and in response to proxy events. Public health agency performance in exercises and proxy events should be used to identify gaps in preparedness and to improve planning, communication, and coordination at the agency and interagency levels, as part of a process of continuous quality improvement in preparedness planning and response. Preparedness drills and exercises should not be evaluated individually, but their cumulative and long-

OCR for page 1
Review of the Centers for Disease Control and Prevention’s Smallpox Vaccination Program Implementation: Letter Report # 6 term impact on preparedness, such as generalizability to other potential hazards, must be considered in the evaluation process. CONCLUDING REMARKS In closing, the committee encourages CDC to learn from the experience and research available from other fields, including, but not limited to disaster research and emergency and disaster response, and to develop the evidence base specific to public health preparedness; strengthen and sustain active coordination and communication with all relevant entities and government agencies at the federal, state, and local levels; and focus on continuous improvement in planning and performance to further the process and the goal of preparedness. The committee wishes to thank you for the continuing opportunity to be of assistance to the Centers for Disease Control and Prevention and its partners as they work to protect the nation’s health. Brian L. Strom, Committee Chair Kristine M. Gebbie, Committee Vice Chair Robert B. Wallace, Committee Vice Chair Committee on Smallpox Vaccination Program Implementation