National Academies Press: OpenBook

Frontiers of Engineering: Reports on Leading-Edge Engineering from the 2009 Symposium (2010)

Chapter: Medical Informatics for Detecting Adverse Events--Genevieve B. Melton

« Previous: Calibration in Computer Models for Medical Diagnosis and Prognostication--Lucila Ohno-Machado, Frederic Resnic, and Michael Matheny
Suggested Citation:"Medical Informatics for Detecting Adverse Events--Genevieve B. Melton." National Research Council. 2010. Frontiers of Engineering: Reports on Leading-Edge Engineering from the 2009 Symposium. Washington, DC: The National Academies Press. doi: 10.17226/12821.
×

Medical Informatics for Detecting Adverse Events

GENEVIEVE B. MELTON

University of Minnesota

Minneapolis, Minnesota


The detection and prevention of adverse events (AEs) in medicine represents a national priority. AEs, defined as injuries that occur in the course of medical management, have important consequences, including higher costs, higher morbidity, and higher mortality. Large-scale focus by the Institute of Medicine (Kohn, 2000) has emphasized most prominently the importance of detecting and preventing AEs to improve patient outcomes. By identifying the AE and analyzing, the context in which it occurred, AE detection can improve system factors and cognitive processes surrounding possible future events and direct resources into targeted efforts to prevent AEs in healthcare.

Unfortunately, with traditional voluntary reporting methods, many AEs are not reported. Manual reviews of charts, although effective, are too costly for routine use. Information technology and informatics tools that use data from electronic health records (EHRs) can potentially improve AE detection and have been identified as important tools for creating a “culture of safety.”

Several classes of automated AE detection systems have been described, most of which use numeric or coded data from EHRs, such as codes for diagnoses and procedures, records of medication administration, laboratory values, and vital signs. Substantial progress has been made in detecting and preventing adverse drug events (ADEs), particularly with the introduction of electronic prescriptions in both inpatient and outpatient settings. Natural language processing (NLP), a set of automated techniques for converting narrative text into a format appropri-

Suggested Citation:"Medical Informatics for Detecting Adverse Events--Genevieve B. Melton." National Research Council. 2010. Frontiers of Engineering: Reports on Leading-Edge Engineering from the 2009 Symposium. Washington, DC: The National Academies Press. doi: 10.17226/12821.
×

ate for computer-based analysis, can be used alone or in combination with other automated methods to improve AE detection.

CHALLENGES AND CONSIDERATIONS FOR AUTOMATED SYSTEMS

AE detection can be challenging because of the complexity and lack of standardization of health care EHR systems and associated electronic data, particularly the lack of standards for the quality and formatting of data, standard definitions of AEs, the variable performance of heuristic rule-based systems, and sparse data sets on low-incidence events. Although most automated AE systems provide feedback retrospectively, “active surveillance” systems that alert providers or administrators of events as they occur can potentially identify and investigate AEs much more quickly.

Many robust EHR systems code data for administrative and billing processes (International Disease Classification version 9 codes and Current Procedural Terminology codes), demographics, laboratory results, admission and discharge registrations, medication administration, and computerized physician order entry (CPOE). However, with the exception of coding for administrative processes, even these structured data are often formatted differently for different EHR and hospital systems.

Although some AEs can be found using coded data, a large number of them require supplementary methods and data sources. One reason for this is that administrative and billing data can be incomplete or inaccurate, and they often do not include AEs explicitly. In addition, more sophisticated data necessary for AE detection, such as clinical reasoning, signs and symptoms, clinical summaries, and physical findings are typically not included as structured data.

Standardized definitions of AEs are a fundamental prerequisite for accurate measurement and analysis. However, centralized nomenclatures, or taxonomies, have not been agreed upon for each health care setting. National initiatives will be necessary to reach agreement and bring about consensus. Several promising AE classification systems have been proposed according to setting or discipline, including the JCAHO Patient Safety Event Taxonomy and the Clavien-Dindo Classification of Surgical Complications.

Up to now, rule-based heuristic systems based on data from a variety of sources have been primarily used for AE detection. Although such systems perform well for certain tasks, they rely heavily on “triggers,” such as abnormal laboratory values or low blood pressure, to indicate a possible AE. Machine-learning techniques are a promising set of approaches that can help to detect events within datasets. Supervised and semi-supervised techniques are particularly helpful in cases where the AE is not obvious or intuitive, particularly when a large, robust, and well-defined dataset exists to allow for adequate system training and optimized performance.

Suggested Citation:"Medical Informatics for Detecting Adverse Events--Genevieve B. Melton." National Research Council. 2010. Frontiers of Engineering: Reports on Leading-Edge Engineering from the 2009 Symposium. Washington, DC: The National Academies Press. doi: 10.17226/12821.
×

However, classification systems using machine learning can also perform poorly, particularly for AEs that have a low incidence (< 1 percent), for which data sets may be sparse and unbalanced. Several techniques have been proposed for balancing such data sets, and some have been using sampling techniques with variable success.

Developers of AE detection systems must also be aware of the cost of false negatives and false positives. An important trade-off must be made between the clinical indication and relative cost of screening extra patients to find AEs and the cost of missing AEs. Most AE detection systems are designed to minimize false negatives so as to maximize the overall detection rate; adjunct manual screening is usually used to confirm identified AEs.

ADVERSE DRUG EVENTS: AN EXAMPLE OF IMPROVED DETECTION

Most ADEs occur when drugs are ordered (55 percent), administered (35 percent), transcribed (5 percent), or dispensed (5 percent). In hospitals that use CPOEs, orders for medications and other clinical care are entered directly into the EHR system. CPOE has been the most successful example of an information technology that helps to detect ADEs. Because many CPOE systems now include alerts and reminders about drug prescriptions, they can also prevent many ADEs.

Additional “triggers” in coded data about medication administration or abnormal laboratory values (e.g., supra-therapeutic or sub-therapeutic drug levels, low hemoglobin, or poor renal function) can improve detection or even prevent many ADEs. This has been demonstrated in several clinical trials in both inpatient and outpatient settings.

NATURAL LANGUAGE PROCESSING: A TOOL TO IMPROVE DETECTION

Clinical documents in EHRs are promising data sources for AE detection systems because they often include clinical reasoning, signs and symptoms, clinician’s summaries, and physical findings, all of which may be helpful for AE detection. Although such narratives are rich in content, they present significant challenges to automated systems in the medical domain. Several investigators have tried using “trigger words,” such as “perforation,” “iatrogenic,” or “error,” for AE detection. However, this technique has limited utility because it does not distinguish between a real, potential, or past event or condition.

Serious challenges to using NLP, or medical text-mining, will have to be overcome. Clinical documents are variably formatted with section headers, tabs or other spatial formatting, and transcription errors (i.e., misspellings and grammatical errors). Meaning in medical texts is not straightforward; there are often

Suggested Citation:"Medical Informatics for Detecting Adverse Events--Genevieve B. Melton." National Research Council. 2010. Frontiers of Engineering: Reports on Leading-Edge Engineering from the 2009 Symposium. Washington, DC: The National Academies Press. doi: 10.17226/12821.
×

uncertainties, negations, and questions about timing. In addition, medical terms include synonymy (related or synonymous terms), abbreviations (often redundant), and context-specific meanings.

Several automated text-mining tools have been developed, including opensource tools available through the National Library of Medicine. One widely used proprietary medical NLP application, MedLEE, uses a vocabulary and grammar to extract data from text. Although MedLEE was initially used to extract information from radiographic reports, it has been expanded for application to a wide range of medical texts. When applied to discharge summaries, it has demonstrated a significant improvement in AE detection compared to traditional reporting alone. Thus NLP techniques can potentially improve AE detection systems.

CONCLUSION

Automated AE detection systems with automated informatics techniques have shown promising results for improving the detection and ultimately the prevention of AEs. National initiatives for the adoption of universal EHR systems and advances in informatics techniques for AE detection are likely to increase the use of these systems, which are now widely used only for ADE systems in health care. Addressing technical challenges related to AE nomenclature, machine-learning methods, sampling techniques, and NLP will improve system performance and, ultimately, improve patient safety.

ADDITIONAL READING

Chang, A., P.M. Schyve, R.J. Croteau, D.S. O’Leary, and J.M. Loeb. 2005. The JCAHO patient safety event taxonomy: a standardized terminology and classification schema for near misses and adverse events. International Journal for Quality in Health Care 11(2): 95–105.

Clavien, P.A., J. Barkun, M.L. de Oliveira, J.N. Vauthey, D. Dindo, R.D. Schulick, E. de Santibañes, J. Pekolj, K. Slankamenac, C. Bassi, R. Graf, R. Vonlanthen, R. Padbury, J.L. Cameron, and M. Makuuchi. 2009. The Clavien-Dindo classification of surgical complications: five-year experience. Annals of Surgery 250(2): 187–196.

Handler, S.M., R.L. Altman, S. Perera, J.T. Hanlon, S.A. Studenski, J.E. Bost, M.I. Saul, and D.B. Fridsma. 2007. A systematic review of the performance characteristics of clinical event monitor signals used to detect adverse drug events in the hospital setting. Journal of the American Medical Informatics Association 14(4): 451–458.

Kilbridge, P.M., and D.C. Classen. 2008. The informatics opportunities at the intersection of patient safety and clinical informatics. Journal of the American Medical Informatics Association 15(4): 397–407.

Kohn, L.T., Corrigan, J.M., and Donaldson, M.S., editors. 2000. To Err is Human: Building a Safer Health System. Washington, D.C.: National Academy Press.

Melton, G.B., and G. Hripcsak. 2005. Automated detection of adverse events using natural language processing of discharge summaries. Journal of the American Medical Informatics Association 12(4): 448–457.

Suggested Citation:"Medical Informatics for Detecting Adverse Events--Genevieve B. Melton." National Research Council. 2010. Frontiers of Engineering: Reports on Leading-Edge Engineering from the 2009 Symposium. Washington, DC: The National Academies Press. doi: 10.17226/12821.
×
Page 99
Suggested Citation:"Medical Informatics for Detecting Adverse Events--Genevieve B. Melton." National Research Council. 2010. Frontiers of Engineering: Reports on Leading-Edge Engineering from the 2009 Symposium. Washington, DC: The National Academies Press. doi: 10.17226/12821.
×
Page 100
Suggested Citation:"Medical Informatics for Detecting Adverse Events--Genevieve B. Melton." National Research Council. 2010. Frontiers of Engineering: Reports on Leading-Edge Engineering from the 2009 Symposium. Washington, DC: The National Academies Press. doi: 10.17226/12821.
×
Page 101
Suggested Citation:"Medical Informatics for Detecting Adverse Events--Genevieve B. Melton." National Research Council. 2010. Frontiers of Engineering: Reports on Leading-Edge Engineering from the 2009 Symposium. Washington, DC: The National Academies Press. doi: 10.17226/12821.
×
Page 102
Next: Managing and Coordinating Health Care: Creating Collaborative, Proactive Systems--David A. Dorr »
Frontiers of Engineering: Reports on Leading-Edge Engineering from the 2009 Symposium Get This Book
×
Buy Paperback | $55.00 Buy Ebook | $44.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In 1995, the National Academy of Engineering initiated the Frontiers of Engineering Program, which brings together about 100 young engineering leaders at annual symposia to learn about cutting-edge research and technical work in a variety of engineering fields. The 2009 U.S. Frontiers of Engineering Symposium was held at The National Academies' Arnold O. and Mabel Beckman Center on September 10-12. Speakers were asked to prepare extended summaries of their presentations, which are reprinted in this volume. The intent of this book is to convey the excitement of this unique meeting and to highlight cutting-edge developments in engineering research and technical work.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!