Cover Image

PAPERBACK
$40.50



View/Hide Left Panel

Why Health Information Technology Doesn’t Work

ELMER V. BERNSTAM AND TODD R. JOHNSON

University of Texas Health Science Center

Houston, Texas

To improve the quality of our health care while lowering its cost, we will make the immediate investments necessary to ensure that within five years all of America’s medical records are computerized. This will cut waste, eliminate red tape, and reduce the need to repeat expensive medical tests. . . it will save lives by reducing the deadly but preventable medical errors that pervade our health care system.

—Barack Obama

George Mason University, January 8, 2009


Widespread dissatisfaction with health care in America and rapid advancements in information technology have focused attention on information technology, which has dramatically improved efficiency and safety in other industries, as an obvious part of the solution to our health care woes. However, there is increasing evidence that the adoption of health information technology (HIT) will not guarantee comparable benefits in health care.

In fact, unmitigated enthusiasm for HIT may even be dangerous. Similar enthusiasm has repeatedly threatened the field of artificial intelligence (AI), resulting in cycles of excitement and disappointment (referred to as “AI winters”). Motivated by a desire to avoid “HIT winters,” we will briefly review the effects of HIT and the “semantic gap,” that is, the difference between “health data” and “health information.” In addition, we identify significant social and administrative barriers to the adoption of HIT in the context of the technical issues; because



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 81
Why Health Information Technology Doesn’t Work ElmEr V. BErnstam and todd r. Johnson University of Texas Health Science Center Houston, Texas To improve the quality of our health care while lowering its cost, we will make the immediate investments necessary to ensure that within five years all of America’s medical records are computerized. This will cut waste, eliminate red tape, and reduce the need to repeat expensive medical tests. . . it will save lives by reducing the deadly but preventable medical errors that pervade our health care system. —Barack Obama George Mason University, January 8, 2009 Widespread dissatisfaction with health care in America and rapid advance- ments in information technology have focused attention on information technol - ogy, which has dramatically improved efficiency and safety in other industries, as an obvious part of the solution to our health care woes. However, there is increas- ing evidence that the adoption of health information technology (HIT) will not guarantee comparable benefits in health care. In fact, unmitigated enthusiasm for HIT may even be dangerous. Similar enthusiasm has repeatedly threatened the field of artificial intelligence (AI), resulting in cycles of excitement and disappointment (referred to as “AI winters”). Motivated by a desire to avoid “HIT winters,” we will briefly review the effects of HIT and the “semantic gap,” that is, the difference between “health data” and “health information.” In addition, we identify significant social and administra - tive barriers to the adoption of HIT in the context of the technical issues; because 1

OCR for page 81
2 FRONTIERS OF ENGINEERING HIT is embedded in a social context, these technical issues must be resolved in a socially and administratively acceptable way. We conclude with research chal - lenges that must be addressed before the full promise of HIT can be realized. EFFECTS OF HEALTH INFORMATION TECHNOLOGY HIT is an “easy sell” to an American public increasingly dissatisfied with the U.S. health care system. Indeed, based on some evidence that HIT can improve the quality of health care (Chaudhry et al., 2006), prevent medical errors (Bates et al., 2001), and increase efficiency (Chaudhry et al., 2006), there seem to be some good reasons for optimism. Unfortunately, many, perhaps most, HIT projects have failed (Littlejohns et al., 2003), and evidence shows that HIT can worsen health care quality in some ways by increasing errors (koppel et al., 2005; Levenson and Turner, 1993), decreasing efficiency, and perhaps even increasing mortality (Han et al., 2005). The term “e-iatrogenesis” has been coined to describe the unintended deleterious consequences of HIT (Weiner et al., 2007). Enough negative evidence has accumulated to prompt the Joint Commission (formerly the Joint Commission on Accreditation of Healthcare Organizations) to issue a “Sentinel Event Alert” (defined as “unexpected occurrence[s] involving death or serious physical or psychological injury, or the risk thereof”) cautioning health care organizations about potential hazards associated with the implementa- tion and use of HIT (Joint Commission, 2008). WE’vE BEEN THERE BEFORE: AI WINTERS During the 1950s, we were faced with a different problem—the cold war. At that time, the government considered IT a promising solution (at least a partial solution) to the problem of tracking Russian communications. It was thought that if researchers could develop automated translation, we would be able to monitor Russian communications and scientific reports in “real time.” There was a great deal of optimism about this, and there were “. . . many predictions of fully auto - matic systems operating within a few years” (Hutchins, 2006). Although many promising applications were found for the poor-quality auto- mated translations that resulted, the optimistic predictions were not realized. To this day, the fundamental problem of context and meaning remains unsolved, mak- ing disambiguation difficult and resulting in some amusing failures. Anecdotal examples include: “The spirit is willing, but the flesh is weak” was translated from English  Russian  English as “The vodka is good, but the meat is rotten,” and “out of sight, out of mind” came out as “blind idiot.” In 1966, the influential Automatic Language Processing Advisory Committee (ALPAC) concluded that “there is no immediate or predictable prospect of useful machine translation” (NRC, 1966). As a result, research funding was stopped, and

OCR for page 81
 WHY HEALTH INFORMATION TECHNOLOGY DOESN’T WORK little research was done on automated translation in the United States from 1967 to 1976, when it was revived and supported until 1989 (Hutchins, 2006). Interest - ingly, disappointment in automated translation in the 1960s was not an isolated event. Similar “AI winters” occurred with respect to connectionism (1970s), expert systems (1990s), and other AI topics. So, although there is tremendous interest in HIT, and even good evidence that it can be useful, some will certainly be disappointed with the results. A recent report by the National Research Council concluded that “. . . current efforts aimed at the nationwide deployment of health care IT will not be sufficient to achieve the vision of 21st century health care, and may even set back the cause if these efforts continue wholly without change from their present course” (NRC, 2009). Thus, there is also good reason for concern that HIT (and the field of biomedi - cal informatics, in general) may be headed for a bust. However, an “HIT winter” would be unfortunate, because there are real benefits to pursuing research and implementation of HIT. THE SEMANTIC GAP Loosely speaking, philosophers who study information draw a distinction between data (syntax) and information, defined as meaningful data (i.e., data + meaning or, alternatively syntax + semantics) (Floridi, 2005). The fundamental problem is that existing technology can store, manipulate, and transmit data but not information. Thus the utility of HIT is limited to the extent to which data approximates meaning, and, unfortunately, there is a large gap between health care data and health care information. Because the difference between data and information is meaning (semantics), we call this the “semantic gap.” Interestingly, Claude Shannon hinted at this issue in 1948 in a seminal paper, “A Mathematical Theory of Communication.” This “mathematical theory of communication” came to be known as information theory. Shannon wrote that “[f]requently, the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem.” Thus, Shannon’s “information theory” explicitly refers to data rather than infor- mation in the philosophical sense. Consider the differences between banking data and health care data, an account at a bank versus a patient’s record (Table 1). One difference is that concepts relevant to health are vague compared to banking concepts. The proper interpretation of the symbols relevant to health care requires significant back- ground knowledge. For example, a patient can be “sick” in many ways, including derangements in vital signs (e.g., extremely high or low blood pressure), prognosis associated with a diagnosis (e.g., any patient with myocardial infarction [heart attack] is sick), and other factors. Two clinicians who are asked to describe the same “sick” individual may legitimately focus on different facts or data.

OCR for page 81
 FRONTIERS OF ENGINEERING TABLE 1 Comparison of Health “Data” and Banking Data Banking Data Health Data Concepts and Precise General, subjective descriptions Example Example Account 123 balance = $15.98 Sick patient Actions Usually (not always) reversible Often not easily reversible Example Example Move money A  B Give a medication Perform a procedure Context Precise, constant, or irrelevant to the task Vague, variable Example Example US $ Normal lab values differ by lab. No two cells, organs, tumors, or patients are identical. User autonomy Well-defined and constrained Variable and dependent on circumstance Example What I can do with my checking account = Example what you can do with yours Clinical privileges depend on training, changes over time, and circumstances Users Clerical staff, account holder Varied, including highly trained professionals Workflow Well-defined, documented, and explicit Highly variable, implicit with many undocumented tasks and exceptions In contrast, the balance in a bank account (e.g., $1,058.93) is relatively objec- tive and is captured by the symbols. If we assume that all transactions (credits and debits) to the account are in the same units (dollars and not pounds or Euros), we need only the numbers and the mathematical operations of addition and subtrac - tion to compute and report the balance. Even though these symbols abstract away the rich semantic complexity of the balance, such as its current purchasing power or that the money can be used to purchase goods and services, this is of no con- sequence to the successful automation of bank accounts. Thus data-manipulating machines (IT) are much better suited to manipulating bank accounts than they are to manipulating clinical descriptors. Twenty-five years ago, S. Marsden Blois (1984) argued that the difficulty of using computers in medicine was due to the nature of medical concepts and medi - cal descriptions. Most medical concepts do not have explicit definitions in terms

OCR for page 81
 WHY HEALTH INFORMATION TECHNOLOGY DOESN’T WORK of necessary and sufficient conditions. Thus they are difficult to describe in the formal languages required by computer systems. For instance, a 2000 definition of a myocardial infarction (heart attack) is nine pages long and contains many imprecise terms, such as: “prolonged,” “usually,” and “experienced observer.” Other medical concepts, such as “sharp pain,” may be even more difficult to map to formal representations. SOCIAL AND ADMINISTRATIvE BARRIERS TO THE ADOPTION OF HEALTH INFORMATION TECHNOLOGY Manipulating data instead of information has many consequences for HIT. The problem for American clinics and hospitals is not usually a shortage of com - puters. Most hospitals and even small private practices use computers to manage financial and administrative data, and many hospitals have functioning e-mail systems and maintain a Web presence. In addition, many clinicians use personal digital assistants (McLeod et al., 2003), and some communicate with patients via e-mail. In contrast, however, most clinical records are kept on paper. There are many barriers to the adoption of HIT. Hospitals that have not implemented electronic medical records most frequently cite financial concerns, including the lack of adequate capital for purchasing equipment (74 percent), maintenance costs (44 percent), and unclear return on investment (32 percent) (Jha et al., 2009). Additional barriers include a mismatch between costs and benefits, cultural resistance to change, lack of an appropriately trained workforce to imple - ment HIT, and many others (Hersh, 2004). To some, clinicians’ resistance to computerization appears to be irrational. However, given the mixed evidence regarding the benefits of HIT, caution seems increasingly reasonable. Thus many clinical enterprises are not computerized because of rational skepticism about the costs and benefits of current HIT, not because of an irrational resistance to technological progress. RESEARCH CHALLENGES Significant research will be necessary to address serious problems before HIT becomes more attractive to clinicians. Many of these problems are outlined in a recent National Research Council report (NRC, 2009). First, there is a mismatch between what HIT can represent (data) and concepts relevant to health care (data + meaning). This very difficult, fundamental challenge subsumes multiple AI prob - lems (e.g., context or common sense) that have proven very difficult to solve. HIT can manipulate form, but not meaning—hence the term “formal methods.” Until we have true information technology, rather than data technology, the benefits of HIT will be limited to applications in which formal methods (i.e., methods that manipulate form) suffice. A second research challenge is to define appropriate applications for HIT,

OCR for page 81
 FRONTIERS OF ENGINEERING as well as policies, procedures, and methods of implementation. Clearly, HIT can be helpful in many ways. For example, computerized alerts and reminders can improve compliance with preventive-care guidelines (Shea et al., 1996) and may be cost effective when used in this way (Bernstam et al., 2000). Similarly, examples of reductions in the number of medication errors and other benefits have been published (e.g., Bates et al., 2001). Therefore, in spite of its limitations, cur- rent HIT can be useful when applied to suitable problems. A third research challenge is to evaluate HIT as a clinical intervention. An instructive example is that a commercial electronic health record was associated with increased mortality at one institution (Han et al., 2005), but no such associa - tion was found at another institution that implemented the same system in a similar care setting (Del Beccaro et al., 2006). Thus outcomes depend on the interplay among HIT, its implementation at a particular institution, and the nature of the institution (e.g., workflow, patient mix, policies, availability of specialists, etc.). A computer system cannot be considered in isolation. Its effects must be evaluated in the context of a specific organization. Any system that can affect clinical decisions has the potential to worsen as well as to improve outcomes. Therefore, these systems should be evaluated as clinical interventions, just as drugs, medical devices, and procedures are evaluated. Fourth, HIT must augment human cognition and abilities. This has been elegantly expressed as the “fundamental theorem of informatics”: human + computer > human (Friedman, 2009). In other words, there must be a clear and demonstrable benefit from HIT. Clearly, it can be beneficial in some situations, and in some ways human cognition and computer technology are complementary. Computers excel at precise, efficient manipulation of data, whereas we excel at discovering, storing, and processing meaning. Thus there are tremendous oppor- tunities for effective human-machine collaboration. For example, monitoring (e.g., waveforms) is much easier for computers than for humans. In contrast, reasoning by analogy across domains is natural for humans but difficult for computers. Defining scenarios, with all relevant parameters, in which HIT is beneficial and demonstrating that using HIT is reliably beneficial in these scenarios remains a research challenge. In its present form, HIT will not transform health care the same way that IT has transformed other industries, partly because of the large semantic gap between health data and health information (concepts). In addition, it is worth noting that many problems with health care will only be solved by changes in health care policy, financing, and so forth. To address the research challenges described above will require unprec- edented collaboration among disciplines that have traditionally worked indepen- dently and have fundamentally different methods, values, and domains of study. Nevertheless, many promising interdisciplinary approaches have been developed. For example, seemingly simple safety devices, such as checklists, which were pioneered in aviation, have been applied to health care with dramatic results (Pronovost et al., 2006). Statistical process control, simulation, and other engi -

OCR for page 81
 WHY HEALTH INFORMATION TECHNOLOGY DOESN’T WORK neering methods have also been successfully applied to certain aspects of health care (NAE and IOM, 2005). HEALTH INFORMATION TECHNOLOGY AND U.S. COMPETITIvENESS True HIT (i.e., health information technology, not health data technology) is critical for U.S. competitiveness in biomedicine—both for biomedical research and for clinical care. Clinical trials are increasingly being conducted in countries with large populations (i.e., large subject pools) and lower regulatory barriers (compared to those in the United States), such as India (Glickman et al., 2009). Barring substantial changes in our values, privacy concerns, and expectations, we simply cannot compete. For example, because of privacy concerns, the United States has no universal patient identifier. As a result, it is very difficult to identify subjects across clinical trials or patients who move between hospitals and other care settings. In contrast, unique patient identifiers in other countries have greatly facilitated clinical research. Similarly, the high cost of health care in the United States encourages “medi - cal tourism.” Many Americans travel abroad for care that is too expensive for them to obtain in the United States (Wapner, 2008). Some foreign hospitals actually specialize in providing care to Americans who come for a high-cost procedure, such as coronary artery bypass surgery. True HIT can help address both of these problems. If we can collect clinical information (meaningful data) as a byproduct of routine care, we can then learn from experience, rather than relying solely on clinical trials. In parallel, we can leverage this information to improve care processes. Thus we would fulfill the promise of HIT described by President Obama. CONCLUSIONS Clearly we must improve health care in fundamental ways, and HIT will be important in transforming the health care system. However, disappointment seems inevitable, because the promises made on behalf of HIT are not likely to be fully realized in the near future. Historical precedents for such cycles of enthusiasm and disappointment with technology include AI, for which boom and bust cycles appear to be the rule rather than the exception. Realizing the promise of HIT to improve health care will require an unprec- edented level of collaboration among communities that have traditionally had little in common, speak different languages, and have very different world views. Thus we are faced with both challenges and opportunities to find fresh perspectives on fundamental problems in the health care domain. In the process, we may also solve some fundamental information (i.e., computer science) problems related to context and meaning.

OCR for page 81
 FRONTIERS OF ENGINEERING REFERENCES Bates, D.W., M. Cohen, L.L. Leape, J.M. Overhage, M.M. Shabot, and T. Sheridan. 2001. Reducing the frequency of errors in medicine using information technology. Journal of the American Medical Informatics Association 8(4): 299 –308. Bernstam, E.V., H.R. Strasberg, and D.L. Rubin. 2000. Cost-Benefit Analysis of Computer-based Patient Records with Regard to their Use in Colon-Cancer Screening. Presented at the Asia Pacific Medical Informatics Conference, Hong kong, China. Available online at http://bmir. stanford.edu/file_asset/index.php/1/BMIR-2000-0.pdf. Blois, M.S. 1984. Information and Medicine: The Nature of Medical Descriptions. Berkeley, Calif.: University of California Press. Chaudhry, B., J. Wang, S. Wu, M. Maglione, W. Mojica, E. Roth, S.C. Morton, and P.G. Shekelle. 2006. Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Annals of Internal Medicine 144(10): 742–752. Del Beccaro, M.A., H.E. Jeffries, M.A. Eisenberg, and E.D. Harry. 2006. Computerized provider order entry implementation: no association with increased mortality rates in an intensive care unit. Pediatrics 118(1): 290 –295. Floridi, L. 2005. Semantic conceptions of information. In Stanford Encyclopedia of Philosophy. Available online at http://plato.stanford.edu/entries/information-semantic/. Friedman, C.P. 2009. A “fundamental theorem” of biomedical informatics. Journal of the American Medical Informatics Association 16(2): 169 –170. Glickman, S.W., J.G. McHutchison, E.D. Peterson, C.B. Cairns, R.A. Harrington, R.M. Califf, and k.A. Schulman. 2009. Ethical and scientific implications of the globalization of clinical research. New England Journal of Medicine 360(8): 816 – 823. Han, Y.Y., J.A. Carcillo, S.T. Venkataraman, R.S. Clark, R.S. Watson, T.C. Nguyen, H. Bayir, and R.A. Orr. 2005. Unexpected increased mortality after implementation of a commercially sold computerized physician order entry system. Pediatrics 116(6): 1506 –1512. Hersh, W. 2004. Health care information technology: progress and barriers. Journal of the American Medical Association 292(18): 2273 –2274. Hutchins, J. 2006. Machine Translation: History. Pp. 375–383 in Encyclopedia of Language and Linguistics, second edition, edited by k. Brown. Burlington, Mass.: Elsevier. Jha, A.k., C.M. DesRoches, E.G. Campbell, k. Donelan, S.R. Rao, T.G. Ferris, A. Shields, S. Rosen- baum, and D. Blumenthal. 2009. Use of electronic health records in U.S. hospitals. New England Journal of Medicine 360(16): 1628 –1638. Joint Commission. 2008. Safely Implementing Health Information and Converging Technologies. Avail- able online at http://www.jointcommission.org/SentinelEvents/SentinelEventAlert/sea_2.htm. koppel, R., J.P. Metlay, A. Cohen, B. Abaluck, A.R. Localio, S.E. kimmel, and B.L. Strom. 2005. Role of computerized physician order entry systems in facilitating medication errors. Journal of the American Medical Association 293(10): 1197–1203. Levenson, N.G., and C.S. Turner. 1993. An investigation of the Therac-25 accidents. IEEE Computer 26(7): 18 – 41. Littlejohns, P., J.C. Wyatt, and L. Garvican. 2003. Evaluating computerised health information sys - tems: hard lessons still to be learnt. British Medical Journal 326(7394): 860 – 863. McLeod, T.G., J.O. Ebbert, and J.F. Lymp. 2003. Survey assessment of personal digital assistant use among trainees and attending physicians. Journal of the American Medical Informatics Association 10(6): 605 – 607. NAE and IOM (National Academy of Engineering and Institute of Medicine). 2005. Building a Better Delivery System: A New Engineering/Health Care Partnership, edited by P.P. Reid, W.D. Comp - ton, J.H. Grossman, and G. Fanjiang.. Washington, D.C.: The National Academies Press. NRC (National Research Council). 1966. ALPAC, Language and Machines: Computers in Translation and Linguistics. Washington, D.C.: National Academy Press.

OCR for page 81
 WHY HEALTH INFORMATION TECHNOLOGY DOESN’T WORK NRC. 2009. Computational Technology for Effective Health Care: Immediate Steps and Strategic Directions, edited by W.W. Stead and H.S. Lin. Washington, D.C.: National Academies Press. Pronovost, P., D. Needham, S. Berenholtz, D. Sinopoli, H. Chu, S. Cosgrove, B. Sexton, R. Hyzy, R. Welsh, G. Roth, J. Bander, J. kepros, and C. Goeschel. 2006. An intervention to decrease catheter-related bloodstream infections in the ICU. New England Journal of Medicine 355(26): 2725 –2732. Shannon, C.E. 1948. A mathematical theory of communication. Bell System Technical Journal 27(2): 379 – 423, 623 – 656. Shea, S., W. DuMouchel, and L. Bahamonde. 1996. A meta-analysis of 16 randomized controlled trials to evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting. Journal of the American Medical Informatics Association 3(6): 399– 409. Wapner, J. 2008. American Medical Association provides guidance on medical tourism. British Medi - cal Journal 337: a575. Weiner, J.P., T. kfuri, k. Chan, and J.B. Fowles. 2007. “e-Iatrogenesis”: the most critical unintended consequence of CPOE and other HIT. Journal of the American Medical Informatics Association 14(3): 387–388; discussion 389.

OCR for page 81