Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
An Attempt to Manage Variation in Obstetrical Practice Stephen C. Schoenbaum The work presented here is not research; rather, it is clinical program evaluation and management. Compared to the conduct of a controlled clinical trial, this approach is awkward. It is, however, a practical attempt to mea- sure variation in clinical practice and to manage the apparent variation. The scenario below is a specific example from which it is possible to draw more general lessons about outcomes measurement and management. THE HARVARD COMMUNITY HEALTH PLAN EXPERIENCE Several years ago, Donald Berwick, Vice-President for Quality of Care Measurement at Harvard Community Health Plan (HCHP), decided that as part of his quality measurement activities it would be important to develop data bases on common clinical activities. The data bases would contain information on outcomes of interest. They would also contain some additional variables describing the population and processes of care. Dr. Berwick's hopes were that these data bases could be used to analyze outcomes in relation to process and that the data could be adjusted for population differ- ences so that data from different sites within HCHP and external data might be compared. THE CHOICE OF OBSTETRICAL CARE Health maintenance organizations (HMOs) have large numbers of young members of childbearing age, and it was logical that the first HCHP data base should be one on obstetrical care. The Health Centers Division of HCHP is currently a 275,000-member staff model HMO with care delivered in 10 centers around the Boston area. In 1986-1987, the first year of the 190
APPLICATION TO CLINICAL PRACTICE TABLE 1 HCHP Deliveries July 1, 1986 - June 30, 1987 191 Health Center/Hospital Used Deliveries 1/A 2/B 3/A 4/B 5/A 6/C 7/A 8/A Number 785 627 489 5 10 497 1 80 376 118 Total per hospital A 2265 B 1 137 C 180 obstetrical data base, there were eight health centers, seven of which used one of two Harvard teaching hospitals with large obstetrical services (Table 19. To obtain data for the period July 1, 1986 through June 30, 1987, HCHP staff reviewed and abstracted the hospital and ambulatory records of all HCHP members who had a delivery. Table 1 shows the number of deliveries among members of each health center and the hospital (A, B. or C) used by each center. Only center 6 used a nonteaching hospital distant from Boston; because it had a relatively small number of deliveries, I will not consider it further. The aggregate number of HCHP deliveries in hospitals A and B is substantial, even though in both instances, HCHP deliveries were less than one-third of the total deliveries in each hospital. VARIATIONS IN PRACTICE BETWEEN TWO HOSPITALS Hospitals A and B have fully developed academic departments of obstet- rics and gynecology headed by full professors at Harvard Medical School. They are believed to have comparable outcomes in terms of neonatal mortality for comparable populations, although hospital A serves as a regional perinatal center, takes care of more referred high-risk obstetrical patients, and has a much larger and more sophisticated neonatal intensive-care unit (ICU). Neither HCHP, the hospitals themselves, nor the state's department of vital statistics can provide appropriate, comparable neonatal mortality figures. Only in rare instances of extremely high-risk mothers does HCHP dis- tribute patients to hospital A because it can provide more intensive care. Almost always, the hospital of delivery is determined by the health center in which the member receives her prenatal care. In 1986-1987, each HCHP health center had its own department of obstetrics and gynecology; that is, each center had its own chief and staff, and each arranged its own coverage schedule. To the extent that cross-center coordination of schedules and
192 EFFECTIVENESS AND OUTCOMES IN HEALTH CARE TABLE 2 HCHP Deliveries July 1, 1986 - June 30, 1987, by Type of Delivery and Health Center and Hospital Used Deliveries (%) per Health Center/Hospital Used Type of Delivery 1/A 2/B 3/A 4/B S/A 7/A 8/A Spontaneous vaginal 57 76 59 78 57 55 59 Cesarear~ section 27 20 22 19 26 28 21 Forceps 15 1 18 1 15 16 18 Vaginal birth after cesarean section 1 3 1 2 2 1 2 TABLE 3 HCHP Deliveries July 1, 1986 - June 30, 1987, by Type of Delivery Deliveries (~o) per Hospital Type of Delivery A B Spontaneous vaginal 57 77 Cesarean section 26 19 Forceps 16 1 Vaginal birth after cesarean section 1 3 combined educational sessions occurred at all, they occurred along the lines of hospital use. Table 2 shows one of the initial analyses from the obstetrical data base. Type of delivery was the first object of attention, and it is the focus of subsequent attempts at intervention. Substantial variation is noted in the percentage of women in the various centers who had a spontaneous (nonoperative) vaginal delivery. This is due to higher rates of cesarean section and forceps deliveries in some centers than others. The lowest rates of operative deliveries are for health centers using hos- pital B (see Table 3~. The occurrence of forceps deliveries is much lower in the hospital with the lower cesarean section rate; this is an unexpected finding, since one might predict that in order to avoid a cesarean section for problems such as cephalopelvic disproportion, the physician would have to extract the baby with forceps. In addition, vaginal birth after cesarean section was somewhat more commonly performed in hospital B. The difference in rates between the two hospitals is even more dramatic if one considers only those women who are primiparas that is, those having their first
APPLICATION TO CLINICAL PRACTICE 193 delivery. In hospital A, 29 percent of primiparas had a cesarean section vs. 22 percent in hospital B. In hospital A, an additional 24 percent of primiparas had a forceps delivery vs. only 2 percent in hospital B. All of these findings suggested to us that practice style might differ significantly between the two hospitals. There are no "right" rates of operative delivery, but we believe that the differences should not be so great. POSSIBLE CAUSES OF THE VARIATIONS A preliminary version of these data was shown to the center-based chiefs of obstetrics. They reasoned that there are differences between health centers and that, since they and their staffs were all equally competent, the differ- ences in rates of operative deliveries must be due to differences in membership. Table 4 highlights these differences and shows the distribution by health center of obstetrical patients who are very young, relatively older, nonwhite, or not married. Although substantial center-to-center differences exist, no single charac- teristic correlates with a high cesarean section or forceps delivery rate. A multivariate analysis with a large number of potential confounding variables was unable to demonstrate any important contributor to the observed varia- tion in type of delivery by health center other than the hospital at which the delivery occurred. Another consideration is whether the variation in delivery rates was re- lated to characteristics of the offspring. When the data were adjusted for low birthweight (which is the only adverse neonatal characteristic that occurs TABLE 4 Patient Characteristics and Types of Delivery of HCHP Members, by Health Center and Hospital Used Members (%) per Health Center/Hospital Used Measure 1/A2/B3/A4/B 5/A7/A 8/A Patient characteristic Less than 18 3.00.71.11.9 0.53.6 years old Over 35 11.213.518.46.1 4.75.8 6.5 years old Nonwhite 48.624.22.67.1 9.551.9 29.0 Not married 24.()9.43.68.5 4.837.2 23.3 Type of delivery Cesarean section 27202219 2628 21 Forceps 151181 1516 18
194 EFFECTIVENESS AND OUTCOMES IN HEALTH CARE with enough frequency in this population to permit adequate analysis), the variation in type of delivery by hospital persisted. Cesarean section is associated with significant maternal morbidity, including infections, increased length of stay, and higher hospitalization costs. In recent years, despite continued increases in cesarean section rates nationwide, it has not been possible to show a continued concomitant improvement in neonatal outcomes. Accordingly, although little information on neonatal outcomes in HCHP patients in hospitals A and B existed, it was reasonable to consider operative deliveries a relatively independent outcome of obstet- rical care. There can be several determinants of outcome, including host (that is, the patient), environment in which care is delivered, and the process of care itself, the latter being within local control. We hypothesized that variation in operative delivery rates ought to have some relationship to the process of care, and we hypothesized that the following components of process of care might affect type of delivery: · Prenatal education, · Obstetrical care (prehospital, in-hospital), · Nursing care, · Anesthesia care, and · Other In a series of interviews we tried to determine from our chiefs of obstetrics and from others in hospitals A and B what the differences in the process of care might be for patients in these institutions. They identified several areas: Location of labor and delivery suite, · Ratio of nurse to patient in labor, · Obstetrical policies (labor curves, forceps), · Epidural anesthesia in labor, and · Relationship of HCHP obstetricians to hospital obstetrics department. We did find differences in these areas. The labor and delivery suite in hospital A is below ground, windowless, relatively noisy, and unattractive compared to that in hospital B. There was a nurse-to-patient ratio of 1:2 or greater for the labor suite in hospital A, compared to a 1:1 ratio at hospital B. In both hospitals the chairman of the department of obstetrics gave strong guidance to clinical policy but in hospital B a graphics tool was used to follow the progress of labor (with a strict definition of failure to progress) and the use of forceps was frowned upon and required specific justification. Consequently, the staff of hospital B did not get much instruction or experience in the use of forceps and may have been less comfortable than the average staff in using them. In contrast, a type of forceps instrument was developed
APPLICATION TO CLINICAL PRACTICE 195 at hospital A many years ago, and there was no injunction against using them. Anesthesia practices also differed. In hospital A, the practice was to use epidural analgesia for most patients in labor (about 70 percent) and not to reduce use of it prior to delivery. In hospital B. only about half the patients received epidural analgesia in labor, and the practice was to keep use of it light, especially as labor progressed. The obstetrical staff also differed. The HCHP obstetricians in hospital B had, for the most part, trained there and were considered "insiders," whereas the HCHP obstetricians in hospital A tended to have trained elsewhere and were more likely to be considered "outsiders." HCHP obstetricians in hos- pital A had poorer morale than HCHP obstetricians in hospital B. The chairman of obstetrics in hospital A had expressed concern about the coor- dination of care for HCHP patients by HCHP obstetricians. Finally, residents interacted closely and directly with HCHP obstetricians for virtually all HCHP patients in hospital B. but only for the highest risk or most complicated patients in hospital A. One thing that did not differ between the two hospitals was patient satisfaction (except for the rating of the ambiance of the labor and delivery suite in hospital A). THE ATTEMPT TO CHANGE PROCESS OF CARE To attempt to alter the process of care for HCHP patients in hospital A, HCHP felt that it needed a multifaceted programmatic intervention. Such interventions do not necessarily take the form of those that might be incor- porated into controlled clinical trials. The first intervention was to appoint a single Plan-wide chief of obstetrics and to arrange for him to have an office in hospital A. The person who was appointed happened to be HCHP's most senior and experienced center-based chief. He had been trained at hospital B. but he had also practiced in hospital A a decade earlier and was respected by the chairman in hospital A. He had moved back to hospital B and had developed the departments of obstetrics in the two health centers using hospital B. Over the next year, the workload of the new chief was very heavy. It included an enormous effort to recruit new obstetricians for a growing HMO and to improve HCHP's central infertility services in response to marked increases in demand. (In that year, Massachusetts mandated infertility benefits of all health insurers, including in vitro fertilization services.) The new central chief also began to work on changing the process of care in hospital A. Several things occurred almost simultaneously. The chief, in order to recruit successfully, convinced HCHP to increase obstetrical salaries. This, and his successful personal interactions with subchiefs and staff, seemed to
196 EFFECTf VENESS AND OUTCOMES IN HEALTH CARE improve morale among existing obstetricians. Three older obstetricians using hospital A ceased obstetrical practice and devoted themselves to gynecology. Thus, a high percentage of the persons doing obstetrics for HCHP at hospital A became direct recruits of the chief and could be thought of as "his people." He also recruited a senior perinatologist from the academic staff of hospital B to work for HCHP, based primarily at hospital A. This had at least two effects: it led the HCHP obstetricians at hospital A to realize that their performance was being monitored more closely (which might have an effect on issues such as continuity of care), and it provided them with an experienced and friendly consultant who could support them in a tough decision to wait it out with a patient rather than moving quickly to a cesarean. The chief also began to work directly with the anesthesia staff of hospital A. In April 1988 an agreement was reached, in the form of a memorandum from the clinical chief of anesthesia to the entire obstetrical staff (HCHP and others), that obstetricians could have a say in the degree of analgesia provided to their patients in labor. The chief also worked to make nurses in hospital A aware that HCHP obstetricians might want longer and more forceful pushing by their patients in labor than had been the usual practice in hospital A in the past. PRELIMINARY RESULTS Table 5 shows preliminary results. The burdens of obtaining data for the obstetrical data base were sufficiently great that no additional data were collected for deliveries between July 1987 and September 1988. Data collection resumed in the fall of 1988, but only for a 50 percent sample of deliveries; the data shown in Table 5, therefore, are for only a small number of patients. They may not prove stable, and we will not have additional data, on January TABLE 5 HCHP Deliveries in 1986-1987 and in October-December 1988, by Type of Delivery Deliveries (%) per Hospital A B 1986-1987 1988 1986-1987 1988 Type of Delivery (N = 2265) (N = 319) (N = 1137) (N = 143) Spontaneous vaginal 57 68 77 76 Cesarean section 26 21 19 19 Forceps 16 9 1 1 Vaginal birth after cesarean section 1 2 3 4
APPLICATION TO CLINICS PRACTICE 197 TABLE 6 HCHP Deliveries For Primiparas in 1986-1987 and in October- December 1988, by Type of Delivery Deliveries (%) per Hospital A B 1986-1987 1988 1986-1987 1988 Type of Delivery (N = 1213) (N = 168) (N = 586) (N = 67) Spontaneous vaginal 47 59 76 77 Cesarear~ section 29 25 22 22 Forceps 24 16 2 1 TABLE 7 Type of Delivery Among Candidates for Repeat Cesarean Section at HCHP Type of Delivery Deliveries (%) per Year 1986-1987 1988 (N = 268) (N = 37) Cesarean section 87 63 Vaginal birth 13 37 through March 1989, until approximately November 1989, because of a continued backlog of work. Nevertheless, as we look at the available data, we see an encouraging trend toward fewer operative deliveries in hospital A; anecdotal evidence suggests that the trend is continuing. Table 6 shows a decrease in the rate of operative deliveries in primipa- ras, and Table 7 shows that the practice of vaginal birth after cesarean section is also increasing at HCHP. There has been a significant contribution from hospital A, and this, too, will lead to decreased cesarean section rates. LESSONS As I stated at the outset, this is not research, but rather a description of work in progress to manage variations in practice by altering process of care. What lessons might we derive from it? First, the data collection process is difficult and expensive. Even though HCHP has automated medical records for all but two centers, and thus easy access to prenatal records, the information presented here does not come from routinely collected data. It is hard to collect and analyze the data for enough potential confounders
198 EFFECTIVENESS AND OUTCOMES IN HEALTH CARE to satisfy doubters that demonstrated variation is not attributable just to some unanalyzed confounding variable. Process interventions in the real world are complex. They tend to be different from situation to situation, institution to institution. They often do not come directly from controlled trials, and they may take the form of appointing a new chief, or firing an old one, or changing a reimbursement scheme, or threatening to move one's business to another vendor or hospital. The complex intervention leads to multiple changes rather than to the single, sometimes unrealistically simple, changes of controlled trials. Another lesson is that the unit of data collection and organization for process improvement is relatively small by statistical or epidemiological standards. This makes it hard to sort random from significant variation it is not easy to convince caregivers or managers that there is a problem worth working on. It also makes it harder to analyze interventions results will not necessarily be statistically convincing, as they are in a controlled trial, and the results will be confounded by time-trend differences in care that would have occurred anyway. Despite these problems, it may be important to act in the face of apparent variation, as we did, even without the most solid data. Such action needs to be accompanied by a commitment to watch, nonjudgmentally, what happens over time. Some observers will undoubtedly feel we acted too fast; others may be uncomfortable in concluding at this time that we are making a difference. Another instructive point from this example is that we took a very differ- ent tack in assessing variation in cesarean section rates than we might have taken if we had followed the appropriateness approach (as The RAND Cor- poration did in its work on variation in surgical practices). Table 8 shows the reasons for cesarean sections in hospitals A and B. as extracted from the records. Although the distributions differ somewhat, over 50 percent of the procedures are attributed either to cephalopelvic disproportion or failure to progress, 20 percent to breech presentation, and 15 percent to fetal distress. I believe that there is enough softness in the definitions of cephalopelvic disproportion, failure to progress, and fetal distress that an independent group of experts assessing only the records of patients having a cesarean section in the two hospitals would have concluded that a similar percentage of the cesarean sections in the hospitals was "appropriate." The depart- ments of obstetrics in these hospitals have regular reviews of their own cesarean sections and rarely conclude that one is inappropriate. Accordingly, had we followed the appropriateness approach to assessing variation in surgical practices, we might have concluded that the overall difference in cesarean rates between these two hospitals was most likely due to some occult underlying difference in the populations rather than to a difference in the process of care. Although it is important to eliminate truly
APPLICATION TO CLINICAL PRACTICE TABLE 8 Reason For Cesarean Section (% of Total Per Hospital) Hospital Reason A B Breech1723 Failure to progress3623 Cephalopelvic disproportion1929 Fetal distress1614 Multiple pregnancy10 Placenta previa12 Abruptio placentae21 Maternal indications21 Herpes33 Other34 TOTAL 100100 199 inappropriate procedures, some, perhaps many, procedures that in retrospect are judged appropriate may actually be unnecessary. A CAUTION I would like to end on a word of caution for those who think that out- comes measurement and management are "the way to go." It is clearly important to assess what we are doing in medical care and to try to determine how, in real life and real time, we can do it better. These efforts are, however, going to be slow, difficult, and costly. There is a whole science of program evaluation that needs to be developed and learned by providers of health care. Experienced epidemiologists will need to be recruited to these efforts. Journals will need to begin to report program evaluations so that we can learn from them. Leaders in health care will have to learn what is realistic and what is not. There are important priorities to be set and the Effectiveness Initiative is a step in this direction. Experts have to put their heads together to consider what information might be obtained from routinely collectable data and what information might be collected at a low marginal cost. Most important, health care experts and providers will have to learn how information, once obtained, can be used to generate process improvements. The trick is in getting from Health Care Financing Administration mortality data or HCHP cesarean rates to some intervention. I believe that regulatory approaches are not very conducive to ongoing, creative, process improvement,
200 EFFECTIVENESS AND OUTCOMES IN HEALTH CARE although regulation or accreditation may play an important role in getting the process going. In general, process improvements require skilled assessment and skilled management. Until and unless the efforts I have just mentioned occur, "outcomes" is just a buzzword. It will wear thin and disappear from our vocabulary. That would be very unfortunate, for we will have lost a major opportunity to examine, evaluate, and improve the way we give medical care. ACKNOWLEDGMENTS I wish to thank the following persons on the HCHP obstetrical data base staff who developed and maintained this data base and who kindly made available the data in this chapter: Kay Larholtz, statistical specialist; Debra Cookson, project coordinator; Diana Parks Forbes, obstetrical database consultant; and Donald M. Berwick, Vice-President for Quality of Care Measurement.
