Appendix B
Syndromic Surveillance

Kelly J. Henning, M.D.

Department of Medicine,

University of Pennsylvania School of Medicine

BACKGROUND

Infectious disease threats, both naturally occurring and intentional, continue to challenge the medical and public health communities. Even before the tragic events of September 11, 2001, public health officials had begun a search for new and innovative methods to enhance the detection of emerging infections and illness due to bioterrorist agents. In response to a series of Institute of Medicine reports citing deficiencies in the ability of U.S. public health systems to deal with emerging infectious diseases (Institute of Medicine, 1987, 1988, 1992) the Centers for Disease Control and Prevention (CDC) prepared a plan entitled Preventing Emerging Infectious Diseases (CDC, 1998). Strengthening surveillance is one of the primary objectives of this plan. Similarly, developing programs that allow for the “early detection and investigation of outbreaks”(CDC, 1998) is cited in Goal One of the 1998 CDC guideline Preventing Emerging Infectious Diseases: A Strategy for the 21st Century. And CDC’s strategic plan for biological and chemical preparedness calls for early detection by integrating bioterrorism into existing systems and developing “new mechanisms for detecting, evaluating, and reporting suspicious events” (CDC, 2000).

At the local level, public health officials evaluated lessons learned during the 1999 introduction of West Nile virus into New York City, and emphasized the importance of preparing surveillance tools that would allow tracking of emerging infections and simultaneously be available for bioterrorism events (Fine and Layton, 1999). Likewise, a major obstacle identified in Operation Topoff, a simulated plague attack on metropolitan



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 281
Appendix B Syndromic Surveillance Kelly J. Henning, M.D. Department of Medicine, University of Pennsylvania School of Medicine BACKGROUND Infectious disease threats, both naturally occurring and intentional, continue to challenge the medical and public health communities. Even before the tragic events of September 11, 2001, public health officials had begun a search for new and innovative methods to enhance the detection of emerging infections and illness due to bioterrorist agents. In response to a series of Institute of Medicine reports citing deficiencies in the ability of U.S. public health systems to deal with emerging infectious diseases (Institute of Medicine, 1987, 1988, 1992) the Centers for Disease Control and Prevention (CDC) prepared a plan entitled Preventing Emerging Infectious Diseases (CDC, 1998). Strengthening surveillance is one of the primary objectives of this plan. Similarly, developing programs that allow for the “early detection and investigation of outbreaks”(CDC, 1998) is cited in Goal One of the 1998 CDC guideline Preventing Emerging Infectious Diseases: A Strategy for the 21st Century. And CDC’s strategic plan for biological and chemical preparedness calls for early detection by integrating bioterrorism into existing systems and developing “new mechanisms for detecting, evaluating, and reporting suspicious events” (CDC, 2000). At the local level, public health officials evaluated lessons learned during the 1999 introduction of West Nile virus into New York City, and emphasized the importance of preparing surveillance tools that would allow tracking of emerging infections and simultaneously be available for bioterrorism events (Fine and Layton, 1999). Likewise, a major obstacle identified in Operation Topoff, a simulated plague attack on metropolitan

OCR for page 281
Denver, was the lack of a surveillance system that could be sustained and available to continuously communicate information to the central command system (Hoffman and Norton, 2000). Several epidemics of the recent past have illustrated the need for enhanced, more timely reporting of infectious diseases. The 1976 Legionnaires’ Disease outbreak in Pennsylvania is an example of a point-source outbreak of an unknown agent with rapid transmission and high mortality associated with the dispersal of exposed persons (Fraser et al., 1977)—an outbreak that today would certainly require evaluation as a potential bioterrorist attack. Yet surveillance and outbreak data related to this investigation were so unwieldy that they had to be evaluated using mainframe computers (Martin and Bean, 1995). The 1993 hantavirus outbreak in the southwestern United States (CDC, 1993) and the West Nile virus encephalitis outbreak in New York City (CDC, 1999) illustrate the importance of prompt reporting by clinicians in triggered public health investigations. The availability of timely, flexible surveillance systems could have aided in characterizing and determining the scope of the outbreaks after their initial reporting. CDC notes several recent successes in strengthening surveillance efforts and in implementing new surveillance strategies, and has initiated the Epidemiology and Laboratory Capacity program to provide health departments with laboratory and technical capacity in dealing with emerging infections (CDC, 1998). Seven states have initiated emerging infections programs (EIPs) to conduct population-based surveillance and special research on emerging and re-emerging diseases. Creation of the Foodborne Diseases Active Surveillance Network (FoodNet) within EIPs has provided a model program for outbreak detection within EIPs. Provider-based networks have been established to collect information from nontraditional public health venues (CDC, 1998; Binder et al., 1999). Examples include infectious diseases surveillance in 11 academic emergency rooms (EMERGEncy ID NET) (Talan et al., 1998) a network of enhanced communication among 500 infectious disease practitioners via the Internet (the Infectious Diseases Society of America Emergency Infections Network [IDSA EIN]), and a group of 22 linked travel medicine clinics in the United States and abroad to monitor disease among returning travelers (GeoSentinel) (CDC, 1998). With the exception of the unexplained death and severe illness project within selected EIP sites (discussed below), all of these enhanced or innovative systems rely on the reporting of specific clinically and/or laboratory-confirmed diagnosed cases. None of these systems are based on the reporting of clinical syndromes or groups of clinical signs and symptoms. Although the need for innovative surveillance techniques had been identified prior to September 11, the U.S. outbreak of anthrax following the intentional delivery of B. anthracis spores through the mail in fall 2001,

OCR for page 281
(CDC, 2001) greatly accelerated the development and initiation of enhanced surveillance systems around the country. DEFINITIONS AND RATIONALE The covert aerosol release of a bioterrorist agent, such as anthrax, plague, or botulinum toxin, would require increased surveillance for illness by the public health and medical communities and rapid institution of illness prevention measures (Rotz et al., 2000). With these agents, as well as with numerous naturally occurring emerging infections, people would likely present initially with nonspecific mild illness. Exposed individuals might stay home from work or school, go to the pharmacy to buy over-the-counter remedies, and, as illness progressed, might call their physician’s offices to report symptoms. As their illness worsened, patients might seek appointments with primary care offices or go to emergency rooms for treatment. Even after presenting to a health care provider, many patients might be sent home with prescriptions for various antibiotics, while others would be ill enough to require hospital admission, some to intensive-care units. The rate with which new cases would occur might depend on infectious dose, location at time of agent release or exposure, environmental factors, and host factors. The geographic pattern of cases could be large-scale, widely dispersed, or focal. Surveillance for the above events, before definitive diagnosis, would require innovative, flexible, disease syndrome-based surveillance systems that do not currently exist in the United States. No published definition of syndromic surveillance has been identified by this author. For the purpose of this discussion, syndromic surveillance is defined as the surveillance of disease syndromes (groups of signs and symptoms), rather than specific, clinical, or laboratory-defined diseases. Syndromic surveillance is a relatively new concept in public health surveillance. Several different terms have been used to denote syndromic systems. Box B-1 lists selected examples. There is considerable overlap in structure and function among these systems, although the source of data collected by each may differ. The lack of an accepted definition for syndromic surveillance and the inconsistent nomenclature in the published literature add to confusion regarding the structure, usefulness, and applicability of this approach. ATTRIBUTES Public health surveillance can be described as the ongoing, systematic collection, analysis, interpretation, and dissemination of data regarding a health-related event for use in public health action to reduce morbidity and

OCR for page 281
BOX B-1 Syndromic Surveillance Systems: Nomenclature The various terms used to denote syndromic systems include the following: Syndromic surveillance Early warning systems Prodromic surveillance Outbreak detection systems Information system-based sentinel surveillance SOURCES: Adapted from the following: Brinsfield et al., 2001; Duchin et al., 2001; Harcourt et al., 2001; Lazarus et al., 2001; Lober et al., 2002; Mostashari and Karpati, 2002; Stern and Lightfoot, 1999; Wagner et al., 2001b; Treadwell, CDC, Personal Communication, 2002. mortality and to improve health (CDC, 2001). CDC has identified a list of surveillance system attributes that are useful for evaluation, including usefulness, simplicity, flexibility, data quality, acceptability, sensitivity, predictive value positive, representativeness, timeliness, and stability. Routinely evaluated surveillance system attributes are relevant to syndromic surveillance systems; however, timeliness and sensitivity may take on added importance (Bravata, 2001). CDC has identified early detection as an essential component for ensuring a prompt response to an intentional biological or chemical attack or the emergence of an unusual or unknown disease (CDC, 2000). Some authors have suggested that, given the level of importance associated with early detection of bioterrorist agents in initiating response, “extreme timeliness of detection” may become a new requirement of at least some public health surveillance systems (Wagner et al., 2001a). For syndromic surveillance, simplicity and acceptability of the system will likely require electronic data transfer that is transparent to providers. Syndromic systems will necessarily be flexible so they can capture a broad range of signs and symptoms that may emerge. Evaluation of the sensitivity of syndromic systems to detect new or emerging health diseases is an evolving science. Most investigators have used naturally occurring, cyclical influenza outbreaks to evaluate existing systems (Tsui et al., 2001; Espino and Wagner, 2001; Canas et al., 2000; CDC, 2002). Because bioterrorism-related events and emerging or reemerging diseases may be spread over very large geographic areas, the representativeness of any one syndromic surveillance system will likely depend on its ability to interact/ communicate with other systems in a given locale and with systems in neighboring states or regions.

OCR for page 281
TYPES OF SYNDROMIC SURVEILLANCE SYSTEMS Syndromic surveillance systems can be categorized in several ways. Syndromic systems can operate for short-term surveillance projects or they can be designed for ongoing, sustained activities. Some syndromic systems have been designed to “drop in” to a locality, usually to bolster local public health surveillance efforts in response to a defined event. Such drop-in systems have been used to enhance surveillance efforts surrounding large-scale events that are national in scope. Drop-in syndromic surveillance, supported by local health departments and CDC, was implemented in Seattle for the 1999 World Trade Organization Meetings (Duchin, Public Health—Seattle and King County, Personal Communication, 2002), in the Washington metropolitan area for the 2001 presidential inauguration (Blythe, Maryland Department of Health and Mental Hygiene, Personal Communication, 2002; Sockwell, Virginia Department of Health (Northern Region), Personal Communication, 2002), in Philadelphia for the July 31–August 4, 2000, Republican National Convention (Chernak, Philadelphia Department of Health, Personal Communication, 2001), and in Los Angeles County for the August 14–17, 2000, Democratic National Convention (Bancroft, County of Los Angeles, Department of Health Services, Personal Communication, 2002; Peterson, County of Los Angeles, Department of Health Services, Personal Communication, 2002). Drop-in syndromic systems used to date have literally “dropped in” to a local health department, operated during the event and for a few days after (an incubation period), and then “dropped out” of the locality. Drop-in syndromic surveillance systems can be used to lay groundwork for sustained syndromic surveillance by building relationships with hospitals, infection control practitioners, information specialists, and others in the health care environment. Since early recognition of new or emerging diseases or a bioterrorist release is expected to be an ongoing goal of innovative surveillance systems, sustained syndromic surveillance systems, ideally operating seven days a week throughout the year, are being actively investigated. Most of these systems are in the pilot or development phase. Syndromic systems differ primarily in the way that they capture data. Table B-1 lists several of the broad categories of syndromic systems that are being explored. Manual systems rely heavily on hospital personnel. A simple, manual system is currently operating in Santa Clara County, California, where a “tally sheet” is used by the emergency department triage nurses in 12 acute care hospitals (Bravata, 2001; Cody, Santa Clara County Department of Health, Personal Communication, 2002). The nurse ticks a mark on the sheet for every patient who has a chief complaint compatible with one of six syndromes: flu-like symptoms, fever with mental status changes, fever with skin rash, diarrhea with dehydration, visual or swallowing difficulties/

OCR for page 281
TABLE B-1 Syndromic Surveillance: Characteristics, Advantages, and Disadvantages   Selected Characteristics Advantage Disadvantage Event-based surveillance   Drop-In Defined time period Active Emergency departments (ED) Large clinics Develop relationships with ED staff, ICPs; transportable to various sites Labor intensive; not sustainable; not scalable Sustained surveillance   Manual Active/passive FAX-based reporting Usually ED triage logs/tally sheets Develop relationships with hospital staff; easy to initiate; detailed information obtainable Labor intensive; difficult to maintain 24/7; not sustainable Electronic Passive Auromated transfer of hospital (usually ED triage or diagnosis) or outpatient data; use of data collected for other purposes; data mining for large collections from multiple sources Can be scalable; minimum or no provider input programming required; data available continuously; data standardized Need expertise and health dept. informatics expertise; confidentiality Novel modes of collection Active Hand-held or touch screen devices Easy to use; rapid provider feedback; can post alerts/info Requires providers input; not sustainable Novel sources of data Active/passive Medical examiner data; unexplained death or severe illness data Clearly defined “syndrome”; may be supplemented with laboratory data Not an early warning; scalable   SOURCES: This table was adapted from the following: Wagner et al., 2001a; Duchin et al., 2001; Pavlin, 2001; Lazarus et al., 2001; Moser et al., 1999; Zelicoff et al., 2001; Stanford report, 2001; Kluger et al., 2001; Rainbow et al., 2000.

OCR for page 281
slurred speech or dry mouth, and acute respiratory distress syndrome. If the patient’s condition does not fit any syndrome, the nurse puts a hash mark in the column “none of the above.” The tally marks for each syndrome are totaled at the end of each nursing shift, and the sheet is faxed to the Santa Clara Health Department. No personal identifiers are transmitted. The information is entered into a computer program at the health department, and the totals are reviewed every 24 hours. As noted in Table B-1 and confirmed by the group in Santa Clara, this method is labor-intensive. Participation by hospitals has declined dramatically since the cessation of additional anthrax cases after December 2001, and Santa Clara County is now actively pursuing alternative systems for implementation. Despite the lack of baseline data for comparison and uncertainties regarding when and how to investigate “clusters” of particular syndromes, many local health departments across the country initiated similar efforts immediately following the terrorist attacks of September 11, 2001 (Blythe, Maryland Department of Health and Mental Hygiene, Personal Communication, 2002; Sockwell, Virginia Department of Health (Northern Region), Personal Communication, 2002; Chernak, Philadelphia Department of Health, Personal Communication, 2001; Paladini, Bergen County Department of Health Services, Personal Communication, 2002). In contrast, several investigators and collaborating health departments have been exploring electronic transfer of data from health facilities to public health departments (Wagner et al., 2001b; Duchin et al., 2001; Pavlin, ESSENCE, Personal Communication, 2001; Mostashari, New York City Department of Health, Personal Communication, 2001; Lazarus et al., 2001; Moser et al., 1999). The key feature of electronic syndromic surveillance is the ability to collect data in an ongoing way without the direct input of health care personnel, so that their operation is transparent to providers. Systems that do not place additional burdens on health care providers are essential for large-scale, sustained syndromic surveillance. Electronic systems have been implemented by the U.S. military (Pavlin, ESSENCE, Personal Communication 2001), regionally within states (Lazarus et al., 2001; RODS; Piposzar, Alleghany County Health Department, Personal Communication, 2002), and at the local level (Mostashari, New York City Department of Health, Personal Communication, 2001). All of these systems are in the pilot or early development stages. The network developed within the Department of Defense—Global Emerging Infections System (DoD-GEIS)—has initiated surveillance for early detection of infectious disease outbreaks by monitoring seven syndromes (respiratory, fever/malaise/sepsis, gastrointestinal, neurological, dermatological-infectious, dermatological-hemorrhagic, and coma/sudden death) in 313 military treatment facilities worldwide (Pavlin, ESSENCE, Personal Communication, 2001). This system, the Electronic Surveillance

OCR for page 281
System for the Early Notification of Community-Based Epidemics (ESSENCE), captures data daily from the standardized ambulatory data record and categorizes syndromes based on International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM) diagnoses assigned by providers. The data are routed to a central server in Denver and forwarded to a secure server at Walter Reed Army Institute of Research for analysis and generation of reports. The time delay from visit to data capture and analysis is 2–4 days. Obtaining the syndromic information places no additional requirements on providers or clinic administrators. Regional and local syndromic surveillance systems, such the Real-Time Outbreak and Disease Surveillance (RODS) system in western Pennsylvania (RODS; Wagner et al., 2001b) and the emergency department chief complaint-based system in New York City (Mostashari, New York City Department of Health, Personal Communication, 2001), collect data principally from emergency department visits. Novel modes of collecting electronic data include several devices that have been developed for direct data entry via touch screens, keypads, or web-based programs (Zelicoff et al., 2001; Stanford Report, 2001; Weiss, Stanford University, Personal Communication, 2001; Coiera, 2001; Zelicoff, Sandia National Laboratories, Personal Communication 2001). These systems simplify the collection of data, but generally require input from health care providers. Although the data transfer can be streamlined by downloading to health authorities via phone lines (Weiss, Stanford University, Personal Communication, 2001) or web-based interfaces (Zelicoff et al., 2001; Coiera, 2001), the systems are not transparent. The Rapid Syndrome Validation Program (RSVP), developed by Sandia National Laboratories, uses a touch-screen-based system to enable health care providers in the emergency department to enter clinical and demographic data on patients with a variety of infectious disease syndromes. The system has network-based reporting that is fast and relatively easy to use. The pilot phase has collected information on six syndromes (flu-like illness, fever with skin findings, fever with altered mental status, acute bloody diarrhea, hepatitis, and adult respiratory distress syndrome) as defined by “physician judgment.” The system gives the physician immediate feedback after a syndrome has been entered. These reports include a geographic plot of the syndrome that has been entered, a temporal graph of similar reports over the past several weeks, and alert screens with outbreak information, if indicated. The New Mexico Department of Health can be notified (via beeper, cell phone, or e-mail) of each syndrome report. This system is in the pilot phase, and reports have not yet been received or utilized by the New Mexico Department of Health (Baumbach, New Mexico Department of Health, Personal Communication, 2002).

OCR for page 281
In 1994, CDC provided funds through state EIPs to Connecticut, California, Minnesota, and Oregon for the conduct of population-based surveillance on unexplained life-threatening illnesses and deaths due to possibly infectious causes among previously healthy persons aged 1 to 49 years (Kluger et al., 2001; Rainbow et al., 2000; Hajjeh et al., 2002). This network was designed to detect emerging infections, and has an extensive laboratory component that includes advanced serological and polymerase chain reaction (PCR) testing to identify novel disease associations or new agents. Reported cases of severe illness or death are assigned clinical syndromes based on the predominant system involved (neurological, cardiac, respiratory, hepatic, or other) (Hajjeh et al., 2002). From 1995 to 1998, 137 cases were identified at the four sites (population 7.7 million), for an overall incidence rate of 0.5 per 100,000 per year. The projects are beginning to report new presentations of known infectious agents. The northern California project has identified a new virus–disease syndrome association, adenovirus Type 3 as an agent of adult toxic shock syndrome (Price et al., 2001), and a novel presentation of Sin Nombre virus (Passaro et al., 2001). The network is not designed for the timely reporting of death or severe illness. Clusters or outbreaks of unexplained death or severe illness have not yet been reported by the network (Hajjeh et al., 2002). COST-EFFECTIVENESS DATA There is no published literature on the cost-effectiveness of syndromic surveillance. However, models for estimating the economic impact of a bioterrorist attack due to anthrax, brucellosis, or tularemia have clearly demonstrated that rapid implementation of a post-attack prophylaxis program is the most important means of reducing cost (Kaufmann, 1997). Similarly, modeling of potential responses to the use of smallpox as a biological weapon has emphasized that delay in intervention would be very costly (Meltzer et al., 2001). For a smallpox scenario with 100 initially infected persons, holding the number infected per infectious person, the percent of the population removed by quarantine, and the percent vaccinated constant, a delay in initiation of control measures of 15 days would result in 15,705 excess cases at 1 year (Meltzer et al., 2001). Because outbreak detection must necessarily precede post-attack prophylaxis or other control measures, rapid outbreak detection (by whatever means available) is key. Some authors have used modeled data (Kaufmann et al., 1997) on anthrax to estimate the financial benefit of even 1 hour of earlier detection for an aerosol release of B. anthracis affecting 100,000 persons (Dato et al., 2001). Most of the achievable benefit occurs by day 4, and the monetary savings from even 1 hour of earlier detection during days 2 and 3 (the steepest part of the cumulative cost curve) could be as high as $200

OCR for page 281
million. These estimates assume that postexposure treatment is 90 percent effective, and that treatment is available and administered instantaneously. KEY STEPS IN DEVELOPMENT OF SYNDROMIC SURVEILLANCE SYSTEMS— QUESTIONS AND UNKNOWNS Several elements of evaluating a public health surveillance system apply to syndromic surveillance (Centers for Disease Control and Prevention, 2001). A number of features of syndromic surveillance, such as defining specific disease syndromes and ensuring timeliness of reporting, are unique. Most of these components have not been systematically evaluated to date. Box B-2 lists selected practical issues faced by investigators and public health officials as syndromic systems are being developed or contemplated. Public Health Authority Most local and state health departments interviewed for this report cited local public health laws that allow the collection of syndromic data (Blythe, Maryland Department of Health and Mental Hygiene, Personal Communication, 2002; Sockwell, Virginia Department of Health (Northern Region), Personal Communication, 2002; Chernak, Philadelphia Department of Health, Personal Communication, 2001; Cody, Santa Clara County Department of Health Services, Personal Communication, 2002; BOX B-2 Issues in Developing Syndromic Surveillance Systems The following issues must be addresses during the development of syndromic surveillance systems: Is there legal authority to support the system? What are the correct syndromes to monitor? How are these syndromes defined? What population should be under surveillance? Which sources of data are most sensitive, specific, and useful? How is timeliness ensured? Are security and confidentiality requirements met? What is the best method for detecting syndrome aberrations? How are aberrations (disease clusters) prioritized and investigated? Are there adequate personnel and laboratory resources available for investigations? How will surveillance results be disseminated to those who need to know?

OCR for page 281
Paladini, Bergen County Department of Health Services, Personal Communication, 2002; Mostashari, New York City Department of Health, Personal Communication, 2001; Baumbach, New Mexico Department of Health, Personal Communication, 2002; Barry, Boston Department of Health, Personal Communication, 2002; Klundt, Massachusetts Department of Health, Personal Communication, 2001). The specific areas of public health law viewed by epidemiologists as allowing such jurisdiction varied. Some states cited mandatory reporting of anthrax as sufficient to allow the collection of syndromic data; other localities referred to public health laws designed to allow data collection for clusters of unusual illness. Recent articles have highlighted the special challenges to public health law that would result from a widespread bioterrorist attack (Barbera et al., 2001; Fidler, 2001). However, the degree to which current public health law addresses any of the unique aspects of syndromic surveillance, such as the acquisition of large data sets to search systematically for particular disease syndromes, requires further evaluation and discussion. Definition of Syndromes Categorization of clinical symptoms into disease syndromes is the cornerstone of syndromic surveillance. Almost all syndrome categories currently in use are based on expected prodromal symptoms associated with the most likely biological weapon agents. Nevertheless, it is not clear which syndromes are most sensitive for identifying emerging infections or agents of bioterrorism. Many systems operating since September 11, 2001, have adopted the seven ICD-9-CM code-based syndromes used in the ESSENCE system (Pavlin, ESSENCE, Personal Communication, 2001). As noted earlier, ESSENCE includes seven syndromes: respiratory (common cold, sinus infection), fever/malaise/sepsis, gastrointestinal (vomiting, diarrhea, abdominal pain), neurological (headache, meningitis), dermatological-infectious (vesicular rash), dermatological-hemorrhagic (bruising, petechiae), and coma/ sudden death. The drop-in surveillance systems implemented for the 2000 Republican and Democratic National Conventions used somewhat different categories: respiratory infection with fever, diarrhea/gastroenteritis, rash with fever, sepsis or nontraumatic shock, meningitis/encephalitis, botulism-like syndrome, and unexplained death with history of fever (Chernak, Philadelphia Department of Health, Personal Communication, 2001; Bancroft, County of Los Angeles, Department of Health Services, Personal Communication, 2002). Systems that rely on novel collection devices, such as touch-screen or hand-held devices, often query the provider in an algorithm style (e.g., fever present; if yes, rash present; if yes, hemorrhagic, etc.) (Weiss, Stanford University, Personal Communication, 2001). The RSVP touch-

OCR for page 281
TABLE B-2 Selected Syndromic Surveillance Systems: Domestic Site Date Initiated Population Studied Data Source(s) Special Feature(s) Obstacles Future Plans Philadelphia Republican National Convention: CDC Drop-In 7/17/00–8/11/00 Philadelphia, adjoining PA, NJ, DE counties— census data 5 Phila. ED First-aid stations, ED syndrome diagnoses, hospital census Multifaceted approach, engaged surrounding counties Provider fatigue, not sustainable, lots of coding and data, entry errors Planning regional approach, enhanced communication with all providers Los Angeles Democratic National Convention: CDC Drop-In 8/7/00–8/22/00 11 LA County emergency departments 1 airport clinic ED/clinic syndrome diagnoses Manually grouped previous ED data into syndromes for baseline (36,000 visits) Required training of hosptials, lots of data (IT) problems, not sustainable Development of ongoing electronic data collection from ED Maryland Department of Health 9/11/01–present 2 counties adjacent to Wash., DC— 9 hospital ED ED triage logs— manually code and enter syndromes Close collaboration with VA and DC health depts. Very labor-intensive, inefficient Developing Web-based reporting and electronic data transfer Virginia Health Dept.-Northern Region (5 health districts) 9/11/01–present 7 hospital ED, counties adjacent to Wash., DC ED triage logs— manually code and enter syndromes, share daily with Maryland/D.C. Close collaboration with other states and between health districts Very labor-intensive, data have been used for other purposes Would like to collaborate with Maryland in web-based reporting

OCR for page 281
Santa Clara County Health Dept. 10/01/01–present 12 hospital ED in the county Tick mark by ED triage nurse for 6 syndromes Simple Data entry, analysis labor-intensive; provider fatigue Evaluating several systems, incl. Health Buddy Baltimore City Health Department 9/11/01–present All Baltimore ED, selected community clinics, 911 calls, dog/cat deaths, school absenteeism ED diagnosis-based syndromes, no. dog/cat deaths, total school absentees Comprehensive collaboration with academic centers, strong political will by the city Not yet real-time or entirely electronic, staff needed to follow-up flags from multiple sources Planning real-time, electronic ED syndromic system Alleghany County, western Pennsylvania 9/11/01–present 60–70% Alleghany County ED chief complaint, electronic Incorporating several health systems, fully automated HD only receives aggregate data Expanding to 13 counties, 54 hospitals Minnesota Department of Health September 2000–present St. Paul/ Minneapolis metro area ED closures/bed capacity (web-based); electronic ICD-9 Health Partner discharge dx Already available data source (Health Partners), flexible —can easily change codes to new syndrome Unclear how useful, sensitive; denominator and historical data pending Planning to add EMS data, couple Health Partners data with laboratory component New Mexico Department of Health—U of NM, statewide medical examiner System 1999–present Statewide Autopsy if antecedent syndromes, specific pathological syndromes reported to NMHD Uniform criteria for performing autopsies and reporting cases to HD; captures reportable conditions, not only BT Broad range of timeliness, required training of field staff Expect to export the system to other medical examiner systems

OCR for page 281
Site Date Initiated Population Studied Data Source(s) Special Feature(s) Obstacles Future Plans Boston City Department of Health 1999–present Citywide 11 ED-total volume data— electronic, Poison Control call volume, death certificate and EMS data Jan. 2002 increase ED volume due to falls on black ice, Oct—review of volume flag noted cases seeking swabs and Cipro 83 days in 2000 exceeded threshold— real-time follow-up may be labor-intensive and frequent Add additional sites, including adjacent county; collect additional data from some sites Seattle-King County Health Department 1999–present Selected sites in Seattle 3 ED and 1 large primary care clinic—electronic data transfer Have detected influenza seasonal patterns; collaborate with academic partner HIPAA issues with obtaining identifiers Expand to population-based system, increase number of data sources, collect identifiers, add GIS capacity Hawaii Department of Health 3/01–present Statewide Infectious disease subset of all claims from largest insurer (60–65% coverage) ICD-9 based, electronic, insurer with excellent data processing capability Long lagtime (18% claims available ≤ 7 days), poor coding accuracy Continue to work toward improved timeliness, add GIS component

OCR for page 281
Massachusetts Department of Health/Harvard Vanguard Medical Associates 9/00–present About 10% of greater Boston area (250,000 pop.) HMO electronic medical record, electronic calls to nurse and doctors Uses 4–5 years of historical data, good denominator data, real-time; academic partners Special population (insured) Developing outbreak reporting algorithms, plan to integrate with other systems Children’s Hospital Boston not available Children’s Hospital Boston and Beth Israel Hospital ED chief complaint, ED ICD-9 discharge diagnosis, web-based MD reporting Reviewed 500 ED charts, chief complaint “respiratory syndrome” detected ~60% Operating on a small pilot basis presently, no data to health dept. yet, no clusters investigated Refine detection algorithms, add 9 hospitals for web-based reporting   SOURCES: Blythe, Maryland Department of Health and Mental Hygiene, Personal Communication, 2002; Sockwell, Virginia Department of Health (Northern Region), Personal Communication, 2002; Chernak, Philadelphia Department of Health, Personal Communication, 2001; Bancroft, County of Los Angeles, Department of Health Services, Personal Communication, 2002; Peterson, County of Los Angeles, Department of Health Services, Personal Communication, 2002; Cody, Santa Clara County Department of Health, Personal Communication, 2002; RODS; Piposzar, Alleghany County Health Department, Personal Communication, 2002; Hirshon, University of Maryland and Baltimore City Department of Health, Personal Communication, 2002; ; Kassenborg, Minnesota Department of Health, Personal Communication, 2002; Barry, Boston Department of Health, Personal Communication, 2002; Nolte, University of New Mexico, Personal Communication, 2002; Kleinman, Harvard Pilgrim Health Care and Harvard Vanguard Medical Associates, Personal Communication, 2001; Lazarus, Channing Laboratory, Brigham and Women’s Hospital, Harvard medical School, Personal Communication, 2001; Chang, Hawaii Department of Health, Personal Communication, 2002; Mandl, Children’s Hospital Boston, Personal Communication, 2002; Klundt, Massachusetts Department of Health, Personal Communication, 2001.

OCR for page 281
FIGURE B-2 Gastroenteritis syndrome count, ESSENCE, San Diego, 2002. Reproduced with permission. J. Pavlin, ESSENCE program, Department of Defense. access now allow for much more rapid awareness of local disease outbreaks at distant points around the globe. Although there are few published reports of syndromic surveillance systems operating outside of the United States, efforts to enhance outbreak detection and link disease surveillance information have increased. The World Health Organization (WHO) established a new approach to global disease surveillance in 1997 termed “outbreak verification.” This system collects data from a broad range of sources, including national institutes of health, nongovernmental organizations, media, the World Wide Web, and electronic mail-based discussion groups. Follow-up is performed by outbreak verification teams in WHO regional offices. Information on outbreaks with potential international public health importance is circulated to subscribers on the Outbreak Verification List. This system is not strictly designed to detect clusters of disease syndromes, but the early nature of reports often includes disease syndromes prior to laboratory diagnosis. Between 1997 and 1999, 246 outbreaks were reported, the most common being cholera (78), acute hemorrhagic fever (24), and acute diarrheal disease (22). Internet outbreak reporting, although not specifically designed for syndrome detection, is timely and increasingly available. As more and more countries and international organizations post information on outbreaks or syndrome clusters on publicly accessible e-mail services, such as ProMED-mail (Woodall, 1997, 2001), the use of this modality for international syndromic surveillance may increase.

OCR for page 281
A network of 22 travel/tropical medicine clinics (14 in the United States and 8 in other countries), GeoSentinel, was initiated in 1996 to collect disease- or syndrome-specific diagnoses on returning travelers, immigrants, and foreign visitors (Freedman et al., 1999). GeoSentinel was designed as a sentinel system and does not have the elements of timeliness or representativeness demonstrated by most syndromic surveillance systems, although it may serve as an early warning system. ESSENCE collects data from DoD medical treatment facilities worldwide and is therefore international in scope. The U.S. Naval Medical Research Unit No. 2 participates in an Early Warning Outbreak Recognition System (EWORS) that collects real-time electronic syndromic data from selected hospital pediatric and internal medicine clinics and emergency departments in Indonesia (Corwin, 2000). A V. cholerae 0139 outbreak was identified in the 1999 pilot phase of the project. The Israeli Ministry of Health has reported on a system of enhanced infectious disease surveillance in Israel during the six-week Gulf War, specifically looking for evidence of biological warfare (Slater and Costin, 1992). Details of the enhanced system are not available, but it apparently included analysis of daily mortality data (rather than the standard weekly procedure) and measures of pneumonia or other severe respiratory symptoms suggestive of pulmonary anthrax. Other international reports stress early detection of outbreaks (Toubiana and Flahault, 1998; Reintjes et al., 2001; Hashimoto et al., 2000) and the importance of networks and collaborations for outbreak detection (Osaka et al., 1999; Pebody et al., 1999). However, the systems described do not collect data. SUMMARY AND CONCLUSIONS Syndromic surveillance is a method of obtaining information about cases exhibiting one or more disease symptoms before a definitive clinical or laboratory diagnosis is available. Outbreaks of disease due to a biological warfare agent may be difficult to diagnose. Delays in diagnosis would likely result in larger numbers of casualties and a more prolonged outbreak. Early detection, by monitoring increases in prodromal symptoms such as fever and cough, forms the basis for most current syndromic surveillance systems. More complex systems that include an advanced laboratory component, such as the CDC unexplained death and critical illness project, are being explored for the detection of emerging infections. The implementation of syndromic surveillance is under way. Many public health officials perceive the need to provide enhanced surveillance following the attacks of September 11, and syndromic surveillance is meeting that need in some localities. Similarly, academic and industry partners

OCR for page 281
are quickly embracing this surveillance technique. There is no nationally coordinated plan to guide the development of syndromic surveillance. More research is needed to guide future planning before specific recommendations can be made. There are a number of potential benefits from syndromic surveillance: New opportunities for collaboration among health departments, emergency medical service providers, hospitals, information system managers, and commercial vendors An opportunity to reinforce the importance of identifying standards-based vocabularies, messages, and case definitions to facilitate the use and transfer of data from clinical information systems to public health surveillance Improved training for public health personnel in the area of information systems and disease-detection techniques The potential to enhance notifiable disease and noninfectious disease reporting systems. Despite these potential benefits, however, caution is appropriate. Syndromic surveillance is a new area of public health surveillance. Studies have not yet been completed to demonstrate the value of this surveillance tool. There is scant information available regarding the best syndromes to monitor for bioterrorism or emerging infections, and among those syndromes being used, there are no generally accepted case definitions. Rather, syndrome definitions differ from site to site, and comparisons across jurisdictions may be difficult. Moreover, the best source of syndromic data is unknown. A combination of different sources may be needed to best capture an increase in early or new disease in a community. And syndromic surveillance systems, by definition, are not laboratory-based, yet supplementing syndromic data with laboratory results may greatly enhance the power and specificity of current systems. Minimally, syndromic surveillance systems should be electronic, should not rely on provider input (be transparent), should be monitored continuously, and should have a mechanism that allows for follow-up if critical increases are detected. Methods for the detection of clusters amid background syndrome “noise” require additional evaluation to identify optimal alarm thresholds. Public health epidemiologists should be involved in the planning of detection methods and response protocols. Sustainability, personnel training needs, and cost are other major considerations. Syndromic surveillance systems should also be viewed as but one of several methods for detection of bioterrorism and emerging infections; resources should not be diverted from proven, core public health functions to syndromic surveillance programs.

OCR for page 281
Studies are needed to evaluate the incremental contribution of syndromic surveillance to other surveillance and outbreak detection programs that are in place or planned. Syndromic surveillance appears to be useful for early outbreak detection and potentially for the identification of emerging infections, but numerous questions remain. REFERENCES Barbera J, Macintyrre A, Gostin L, Inglesby T, O’Toole T, DeAtley C, Tonat K, Layton M. Large-scale quarantine following biological terrorism in the United States: scientific examination, logistic and legal limits, and possible consequences. JAMA 2001;286:2711– 2717. Binder S, Levitt AM, Sacks JJ, Hughes JM. Emerging infectious disease: public health issues for the 21st century. Science 1999;284:1311–1313. Bravata D. Evidence Report/Technology Assessment. Bioterrorism: Use of Information Technologies and Decision Support Systems. Prepared for AHRQ. Draft. December 2001. Brinsfield KH, Gunn JE, Barry MA, McKenna V, Syer KS, Sulis C. Using volume-based surveillance for an outbreak early warning system. Acad Emerg Med 2001;8:492. Canas LC, Lohman K, Pavlin JA, Endy T, Singh DL, Pandey P, Shrestha MP, Scott RM, Russell KL, Watts D, Hajdamowicz M, Soriano I, Douce RW, Neville J, Gaydos JC. The department of defense laboratory-based global influenza surveillance system. Mil Med 2000;165(Suppl 2):52–56. CDC. Outbreak of acute illness—southwestern United States, 1993. MMWR 1993;42:421– 424. CDC. Addressing Emerging Infectious Disease Threats: A Prevention Strategy for the United States. Atlanta, Georgia. 1994. CDC. Outbreak of West Nile-like viral encephalitis—New York, 1999. MMWR 1999;48: 845–849. CDC. Biological and chemical terrorism: strategic plan for preparedness and response. Recommendations of the CDC strategic planning workgroup. MMWR 2000;49(RR04):1– 14. CDC. Preventing Emerging Infectious Diseases: A strategy for the 21st century. Atlanta. October, 1998. Accessed at www.cdc.gov on February 1, 2001. CDC. Update: investigation of anthrax associated with intentional exposure and interim public health guidelines, October 2001. MMWR 2001;50:889–893. CDC. Use of ambulance dispatch data as an early warning system for community-wide influenza-like illness, New York City. MMWR 2002 (in press). Centers for Disease Control and Prevention. Updated guidelines for evaluating public health surveillance systems: recommendations from the guidelines working group. MMWR 2001;50(No. RR-13):1–35. Coiera E. Use of web-based systems for clinician reporting of suspected bioterrorism events. 20 pages. A report prepared for AHRQ. June 2, 2001. Corwin A. Developing regional outbreak response capabilities early warning outbreak recognition system (EWORS). Navy Med 2000;Sept/Oct:1–5. Dato VM, Wagner MM, Allswede MP, Aryel R, Fapohunda A. The Nation’s Current Capacity for the Early Detection of Public Health Threats Including Bioterrorism. 78 pages. Report commissioned by AHRQ. June 8, 2001. Department of Health and Human Services. Standards for privacy of individually identifiable health information. Final rule. Fed Reg 2000;65(250):82462–82829.

OCR for page 281
Duchin JS, Karras BT, Trigg LJ, Bliss D, Vo D, Ciliberti J, Stewart L, Rietberg K, Lober WB. Syndromic surveillance for bioterrorism using computerized discharge diagnosis databases. Proc AMIA Symp 2001;897. Espino J, Wagner M. The accuracy of ICD-9 coded chief complaints for detection of acute respiratory illness. Proc AMIA Symp 2001:164–168 (in press). Fidler DP. The malevolent use of microbes and the rule of law: legal challenges presented by bioterrorism. Clin Infect Dis 2001;33:686–689. Fine A, Layton M. Lessons from the West Nile viral encephalitis outbreak in New York City, 1999: implications for bioterrorism preparedness. Clin Infect Dis 2001;32:277–282. Fraser DW, Tsai TR, Orenstein W, Parkin WE, Beecham HJ, Sharrar RG, Harris J, Mallison GF, Martin SM, McDade JE, Shepard CC, Brachman PS. Legionnaires’ disease: description of an epidemic of pneumonia. N Engl J Med 1977;297:1189–1197. Freedman DO, Kozarsky PE, Weld LH, Cetron MS. GeoSentinel: the global emerging infections sentinel network of the International Society of Travel Medicine. J Travel Med 1999;6:94–98. Goldenberg A, Shmueli G, Caruana RA, Fienberg SE. Early statistical detection of anthrax outbreaks by tracking over-the-counter medication sales. Proc Nat Acad Sci 2002 (under consideration for publication). Grein TW, Kamara K-BO, Rodier G, Plant AJ, Bovier P, Ryan MJ, Ohyama T, Heymann DL. Rumors of disease in the global village: outbreak verification. Emerg Infect Dis 2000:97– 102. Hajjeh RA, Relman D, Cieslak PR, Sofair AN, Passaro D, Flood J, Johnson J, Hacker JK, Shieh W-J, Hendry RM, Nikkari S, Ladd-Wilson S, Hadler J, Rainbow J, Tappero JW, Woods CW, Conn L, Reagan S, Zaki S, Perkins BA. Surveillance for unexplained deaths and critical illnesses due to possibly infectious causes, United States, 1995–1998. Emerg Infect Dis 2002;8:145–152. Harcourt SE, Smith GE, Hollyoak V, Joseph CA, Chaloner R, Rehman Y, Warburton F, Ejidokun OO, Watson JM, Griffiths RK. Can calls to NHS Direct be used for syndromic surveillance? Commun Dis Pub Health 2001;4:178–182. Hashimoto S, Murakami Y, Taniguchi K, Nagai M. Detection of epidemics in their early stage through infectious disease surveillance. Int J Epidemiol 2000;29:905–910. Hoffman RE, Norton JE. Lessons learned from a full-scale bioterrorism exercise. Emerg Infect Dis 2000;6:652–653. http://epi.health.state.nm/rsvpdesc/default.asp. Accessed December 12, 2001. http://www.army-technology.com/contractors/nbc/idaho/. Accessed February 6, 2002. http://www.ehealthinitiative.org/. eHealth Initiative 2001:Year in Review. Accessed February 8, 2002. Hutwagner LC, Maloney EK, Bean NH, Slutsker L, Martin SM. Using laboratory-based surveillance data for prevention: an algorithm for detecting Salmonella outbreaks. Emerg Infect Dis 1997;3:395–400. Institute of Medicine. 1987. The U.S. Capacity to Address Tropical Infectious Disease Problems. National Academy Press. Washington, D.C. Institute of Medicine. 1988. The Future of Public Health, National Academy Press. Washington, D.C. Institute of Medicine. 1992. Emerging Infections: Microbial Threats to Health in the United States. National Academy Press, Washington, D.C. Jormanainen V, Jousimaa J, Kunnamo I, Ruutu P. Physician database searches as a tool for early detection of epidemics. Emerg Infect Dis 2001;7:474–476. Jousimaa J, Kunnamo I, Makela M. Physicians’ patterns of using a computerized collection of guidelines for primary health care. Int J Technol Assess Health Care 1998;14:484–493.

OCR for page 281
Kaufmann A, Meltzer M, Schmid G. The economic impact of a bioterrorist attack: are prevention and post attack intervention programs justifiable? Emerg Infect Dis 1997;3:83– 94. Kluger MD, Sofair AN, Heye CJ, Meek JI, Sodhi RK, Hadler JL. Retrospective validation of a surveillance system for unexplained illness and death: New Haven County, Connecticut. Am J Pub Health 2001;91:1214–1219. Kortepeter MG, Pavlin JA, Gaydos JC, Rowe JR, Kelley PW, Ludwig G, McKee KT, Eitzen EM. Surveillance at US military installations for bioterrorist and emerging infectious disease threats. Mil Med 2000;165:ii–iii. Kulldorff M. Prospective time periodic geographical disease surveillance using a scan statistic. JR Stat. Soc 2001;164, Part 1:61–72. Lazarus R, Kleinman KP, Dashevsky I, DeMaria A, Platt R. Using automated medical records for rapid identification of illness syndromes (syndromic surveillance): the example of lower respiratory infection. BMC Public Health. 2001;1:9. Accessed at http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pubmed&pubmedid=11722798 on December 3, 2001. Lober WB, Karras BT, Wagner MM, Overhage JM, Fraser H, Mandl KD, Espino JU, Tsui FC. Roundtable on bioterrorism detection: information systems-based surveillance. JAMIA. 2002 (in press). Martin SM, Bean NH. Data management issues for emerging diseases and new tools for managing surveillance and laboratory data. Emerging Infect Dis 1995;1:124–128. Meltzer MI, Damon I, LeDuc JW, Millar JD. Modeling potential responses to smallpox as a bioterrorist weapon. Emerg Infect Dis 2001;7:959–969. Moser SA, Jones WT, Brossette SE. Application of data mining to intensive care unit microbiologic data. Emerg Infect Dis 1999;5:454–457. Mostashari F, Karpati A. Towards a theoretical (and practical) framework for prodromic surveillance. Abstract. International Conference on Emerging Infectious Diseases, Atlanta. March 24–27, 2002 (accepted). National Electronic Disease Surveillance System Working Group: National Electronic Disease Surveillance System (NEDSS): A standards based approach to connect public health and clinical medicine. J Pub Health Manage Prac 2001;7:43–50. Osaka K, Inouye S, Okabe N, Taniguchi K, Izumiya H, Watanabe H, Matsumoto Y, Yokota T, Hashimoto S, Sagara H. Electronic network for monitoring travellers’ diarrheoea and detection of an outbreak caused by Salmonella enteritidis among overseas travellers. Epidemiol Infect 1999;123:431–436. Passaro DJ, Shieh W-J, Hacker JK, Fritz CL, Hogan SR, Fischer M, Hendry RM, Vugia DJ. Predominant kidney involvement in a fatal case of hantavirus pulmonary syndrome caused by Sin Nombre virus. 2001;33:263–264. Pavlin J. Electronic surveillance system for the early notification of community-based epidemics (ESSENCE). Information Paper. Personal communication. December 17, 2001. Pebody (RG, Furtado C, Rojas A, McCarthy N, Nylen G, Ruutu P, et al. An international outbreak of Vero cytotoxin-producing Escherichia coli 0157 infection amongst tourists: a challenge for the European infectious disease surveillance network. Epidemiol Infect 1999;123:217–223. Price NO, Hacker JK, Silvers JH, Crawford-Miksza L, Hendry RM, Flood J, Hajjeh RA, Reingold AL, Passaro DJ. Adenovirus type 3 viremia in an adult with toxic shock-like syndrome. CID 2001;33:260–262. Rainbow J, Lynfield R, Johnson JR, Danila RN. Minnesota surveillance for unexplained deaths and critical illnesses of possible infectious cause. 2000;83:61–63.

OCR for page 281
Reintjes R, Baumeister H-G, Coulombier D. Infectious disease surveillance in North Rhine-Westphalia: first steps in the development of an early warning system. Int J Hyg Environ Health 2001;203:195–199. RODS. http://ultra.cbmi.upmc.edu/-ju/rods/. Rotz LD, Koo D, O’Carroll PW, Kellogg RB, Sage MJ, Lillibridge SR. Bioterrorism preparedness: planning for the future. J Pub Health Manage Prac 2000;6:45–49. Slater PE, Costin C. Infectious disease and mortality surveillance in Israel in peace and war. Pub Health Rev 1992;93:280–284. Stern L, Lightfoot D. Automated outbreak detection: a quantitative retrospective analysis. Epidemiol Infect 1999;122:103–110. Talan DA, Moran GJ, Mower WR, Newdow M, Ong S, Slutsker L, Jarvis WR, Conn LA, Pinner RW. EMERGEncy ID NET: an emergency department-based emerging infections sentinel network. Ann Emerg Med 1998;32:703–711. Stanford Report. New device helps track symptoms. October 31, 2001. Accessed at www.stanford.edu/dept/news/report/news/october31/healthbuddy.html on December 21, 2001. Toubiana L, Flahault A. A space-time criterion for early detection of epidemics of influenza-like illness. Eur J Epidemiol 1998:14:465–470. Tsui F-C, Wagner MM, Dato V, Chang C-CH. Value of ICD-9-coded chief complaints for detection of epidemics. Proc AMIA Symp 2001:711–715 (in press). Wagner MM, Aryel R, Dato VM, Krenzelok E, Fapohunda A, Sharma R. Availability and comparative value of data elements required for an effective bioterrorism detection system. 119 pages. Report commissioned by AHRQ. November 28, 2001a. Wagner MM, Tsui F-C, Espino JU, Dato VM, Sittig DF, Caruana RA, McGinnis LF, Deerfield DW , Druzdzen MJ, Fridsma DB. The emerging science of very early detection of disease outbreaks. J Pub Health Manage Prac 2001b;7:50–58. Woodall J. Official versus unofficial outbreak reporting through the Internet. Inter J Med Inform 1997;47:31–34. Woodall JP. Global surveillance of emerging diseases: the ProMED-mail perspective. Cad Saude Publica 2001;1(Supplement):147–154. Zelicoff A, Brillman J, Forsland DW, George JE, Zink S, Koenig S, Staab T, Simpson G, Umland E, Bersell K. The rapid syndrome validation project (RSVP). Proc AMIA Symp 2001;771–776.