|
Items in cart [1] |
Questions? Call 888-624-8373 |
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 201
Using Outcome Measures to
Improve Care Delivered by
Physicians and Hospitals
Eugene C. Nelson
The question "What works in the practice of medicine?" is very impor-
tant. It is largely methodological and focuses on measurement. Yet an even
more critical question is this: What works to improve the practice of medicine?
It is one thing to use measurement to find out what works, but it is quite
another thing to know what to do to improve that work.
"If you always do what you always did, you will always get what you
always got." This simple saying, spoken by a factory worker to W. Edwards
Deming, the father of continuous improvement, makes that point (1~. Improvement
in outcomes requires change upstream in the process. Measurement is part
of a process of change it can help the process get started in the right
direction and monitor the effect of efforts, but measurement alone will not
create improvements.
If effectiveness is to be increased, process improvement thinking must be
included while constructing outcomes measurement systems. The challenge
is not to create outcomes measurement systems, but to construct outcomes
measurement/improvement (MI) systems for use by clinicians, hospitals,
and other health care organizations. In this chapter, I will cover four points
briefly. First, I describe two MI systems for medical practices. Second, I
introduce two MI systems for hospitals. Third, I highlight the hallmarks of
these systems, and fourth, I offer guidelines for using outcomes measures to
make improvements.
Before moving to point number one, I wish to illustrate the concept of an
outcomes measurement/improvement system. Figure 1 illustrates an MI
system for individual patients. The cycle begins with a patient visiting the
physician or entering the hospital. The patient's baseline health outcomes
are measured (disease-specific measures, general health status indicators,
and patient expectations for care) and assessed by the clinician; the patient's
201
OCR for page 202
202
EFFECTIVENESS AND OUTCOMES IN HEALTH CARE
Patient Visit/Stay
1
Measure
Outcomes
Implement and
Foilow-Up
Plan
Regimen
Assess Status
FIGURE 1 A Measurement/lmprovement System for Individual Patients
regimen is planned; care is implemented and follow-up is instituted; and
outcomes measures are periodically gathered. The cycle continues making
adjustments as the patient's status changes.
OUTCOMES MEASUREMENT/IMPROVEMENT
SYSTEMS FOR MEDICAL PRACTICES
THE COOP CHARTS
The first two systems I describe might be thought of as early attempts to
develop MI systems for doctors' offices. One of these is the Dartmouth
COOP Chart system. The following are the vital facts about the system:
What? ~ ~ ~ - ~ ~ ~
How?
Illustrated posters of health status
Patient rates health
Patient scores self
Resource guide for clinician to prompt a regimen
Benefits? Better communication
Discovery of important problems
Ease of use
The COOP Charts (Figure 2) are similar to the Snellen charts that physi-
cians have used in their offices for decades to test vision quickly. In fact,
the Snellen charts were the inspiration for the COOP Charts (2~. The idea
was to construct simple charts that could be used to measure some 10 key
dimensions of overall health rapidly-physical function, mental health, so-
cial function, pain, quality of life, and so on. Recently, we have begun to
link COOP measurements with a functionally oriented resource guide. The
OCR for page 203
203
oo
oo
o
C)
z
O cn
a)
I
~:
O a
_ c
- CJ)
O
C]
z
o
-
>
a)
o
~) ~ E
~ a)
<1 ~n ~
~ Q (O
_ ~
:~_ ~ C
._ 2
I ~ ~
a' ~ a)
0, ·_ D
(L) Ct
4- ~ ·~
D `,, .'
m-~)
C ~
~n
~n
~n a)
O ~) Q
>`E a)
_ ~
,~, D 0,,
Q . - o
=_ X
CO ~
C Co
O
~ O Q
O ~ Q
I a, C
C
o
o
o
C'
o
.~
> C~-
~n
~,
~ .c
,~
U)
~CM
Q u'
o a)
~ (4 1~=3(,j,~':
I
C)
·_,
o
o
o
-
au
-
o
o
o
c~
ca
r
~ -
pH
o
u
. .
o
u)
~;~1 ~
W~ o
I E1_~ O
~ - ~ O O _, ' . ~ ~. 0 ~ ~ ~ ~ v) ~ a
>~C(O [ 7L~ O ~0 ~ ~0 > ~ ~ ~
C)
OCR for page 204
204
EFFECTIVENESS AND OUTCOMES IN HEALTH CARE
objective is to link measurement of the patient's functioning with sugges-
tions for improvements that the physician can use to plan the regimen,
thereby building improvement into the process of patient care delivery.
Studies of the COOP Charts show that they are very easy to use in busy
medical practices, that they are reliable and valid, and that use of them has
several benefits (3,4~. Both patients and physicians believe that the charts
improve communication and frequently lead to the discovery of important
problems that would otherwise be missed. In a study conducted in about a
dozen medical practices, physicians said that when the COOP Charts are
used for case-finding, new, important information is produced for approximately
25 percent of patients; physicians also said that this leads to new treatment
in two of five of these patients, providing a better fit between the patient's
problems and the physician's plan of treatment. In addition to case-finding,
COOP Charts can be used to monitor the overall functioning of patients
with serious chronic diseases. Research suggests that the charts are able to
show what impact discrete medical events, such as falls and adverse drug
reactions, have on the patient's basic physical and mental function. Thus,
use of the charts may help the doctor to understand better the effect of
disease on the "whole" patient and thus to deliver more comprehensive
care.
The COOP Chart system for measuring and improving health outcomes
holds great promise. The Henry J. Kaiser Family Foundation is sponsoring
a large randomized trial at the Harvard Community Health Plan to document
the system's case-finding utility in clinical practice, and the charts are being
field tested in 20 countries to determine their value in other parts of the
world.
THE RUBENSTEIN FUNCTIONAL HEALTH STATUS APPROACH
The second MI system for use in medical practice was developed by Lisa
Rubenstein and her colleagues at the University of California, Los Angeles.
The following are vital facts about the system:
What? Questionnaire on health status
How? Patient rates health
Computer scores and profiles patient
Resource guide for clinician
Benefits? Better mental health
Better social function
The development of the Rubenstein functional health approach is an in-
teresting and important story for anyone interested in improving outcomes.
Several years ago, Dr. Rubenstein and colleagues at UCLA, BIAC (Beth
Israel Ambulatory Care Center), RAND, and Harvard collaborated on a
OCR for page 205
APPLICATION TO CLINICAL PRACTICE
205
randomized trial. Their goal was to show that functional assessment of
elderly patients visiting the offices of general internists would improve
outcomes of care (5~. The measurement strategy was based largely on the
short-form general health status tools developed by John Ware and his col-
leagues at RAND. Two rather large randomized trials were conducted, one
in Los Angeles and the other in Boston (6,7~. The results were disappoint-
ing. Measurement of functioning of the internists' patients did nothing to
improve outcomes a year later.
In analyzing the reasons for these negative results, the investigators dis-
covered that the MI cycle had been broken. Patients' baseline functioning
had been measured and the results placed in the medical records; however,
there was very little evidence that physicians had used this new information
to add to their assessment or to plan treatment. As noted earlier, measure-
ment of outcomes alone may produce no gains: "If you always do what you
always did, you will always get what you always got."
Dr. Rubenstein conducted a second randomized trial using the same
measurement tool, but this time adding a function-oriented resource guide
to the system. The resource guide was designed to link the patient's prob-
lem with specific treatments that would be appropriate and effective. It
provided site-specific "tips" on what the physician might do for an elderly
patient with a physical disability such as poor balance or a mental health
condition such as depression. The results from this second randomized
trial, which included more than 76 physicians and 571 patients, were positive.
Patients in the test group had significantly better mental health and social
activity scores than patients in the control groups who received customary
care after one year (8~. This time, the entire measurement/improvement
cycle had been completed, and patients' outcomes had improved.
MEASUREMENT/IMPROVEMENT SYSTEMS FOR HOSPITALS
The first MI system for hospitals that I will discuss is being used in my
organization, Hospital Corporation of America (HCA), and other hospitals
around the country.
PICA PATIENT JUDGMENT SYSTEM
Here are the vital facts on the Hospital Quality Trends (HQT) Patient
Judgment System:
What? Random sample of patients rating hospital quality and health
status on questionnaire
How?
Patient rates quality and health status
Computer scores and profiles hospitals
Results show improvement opportunities
OCR for page 206
206
EFFECTIVENESS AND OUTCOMES IN HEALTH CARE
Benefits? Trends in quality over time
Benchmarks across hospitals
Linked to process improvement method (FOCUS-PDCA)
The HQT Patient Judgment System was developed by a multidisciplinary
design team that included practicing physicians, hospital administrators,
nurses, and quality research leaders such as Paul Batalden, Donald Berwick,
and John Ware from HCA, Harvard, and RAND, respectively. The system
was tested in eight hospitals in 1987 and is now in use in approximately 100
hospitals. About 65 of these hospitals are owned by HCA; the others are a
mix of large and small voluntary hospitals. An article describing the sys-
tem was published in the June 1989 issue of Quality Review Bulletin and a
monograph summarizing the development work is in press at Medical Care
(9,10~.
The aim of the system is to provide hospitals with valid, reliable, and
useful trends in hospital quality, based on the voice of the patient. A
random sample of discharged patients judges 10 dimensions of hospital
quality (for example, admissions, nursing, physicians, information, daily
care, and discharge) that are measured with a 68-item questionnaire. Patients
also evaluate their health benefit from the stay and complete selected COOP
Charts showing postdischarge functioning. Each hospital receives reports
twice a year. The reports use graphic techniques to reveal longitudinal
trends in quality. Hospitals use the reports to monitor trends and to identify
(or focus on) high priority areas for improvement. These areas can then be
addressed by Quality Improvement Teams using a structured improvement
method, FOCUS-PDCA, that takes advantage of the scientific method in
planning and managing process improvement (11~.
An example of how this MI system is used can be found at West Paces
Ferry Hospital in Atlanta, Georgia. The senior leadership team there iden-
tified those aspects of quality that were most important to patients-clinical
outcome, nurse response time and caring, nurse skill, the admitting process,
and the discharge process-and that were candidates for improvement. The
leadership then "chartered" several Quality Improvement Teams (composed
of members from different departments involved in the process). It challenged
every department in the hospital to identify which of its processes influence
these key areas of quality and to begin FOCUS-PDCA on one or more of
them. West Paces will use the HQT Patient Judgments System to monitor
the overall success of its quality improvement efforts.
In October 1989, 50 hospitals began using the HQT system for adult
psychiatric patients. In addition to everything in the system described above,
it includes the clinician's assessment of mental health at admission and
discharge plus the patient's rating of his or her own physical function,
mental health, and quality of life at admission, discharge, and one month
after discharge.
OCR for page 207
APPLICATION TO CLINICAL PRACTICE
SOUTH SHORE HOSPITAL GERIATRIC ASSESSMENT AND
PLANNING PROGRAM
207
The final example of a measurement/improvement system is the Geriatric
Assessment and Planning (GAP) program. A thumbnail sketch follows:
What?
How?
Functionally oriented hospital record system for managing
and following elderly patients
Nurse rates functioning at admission, midstay and discharge
Functional ratings are linked to treatment plan
Benefits? Replaces nursing notes
Basis for comprehensive discharge planning
Better match between patient function and treatment plan
Frail patient follow-up after discharge
The GAP system was developed by leaders at South Shore Hospital,
Carolee DeVito and William Zubkoff, with the assistance of external consultants
in functional assessment such as Paul Densen and Charlotte Hamill (12~.
The purpose of the GAP program is to provide a standard method of comprehensive
patient assessment that will enable the hospital to improve the match of its
services to the changing needs of elderly patients (13~.
Starting in about 1983, South Shore began modifying the processes for
admitting, nursing, and discharge planning to include full assessment of the
patient's clinical and functional status at admission, midstay, and discharge.
The GAP program involves all patients age 65 and older admitted to the
hospital. Assessment includes standard data on such aspects of health as
clinical parameters; Activities of Daily Living; Instrumental Activities of
Daily Living; social, emotional, and cognitive function; and continuing care
needs after discharge. The entire caregiving team physicians, nurses, discharge
planners, and home health professionals-builds and uses the assessment/
management form to update the patient's status and to match services to
patient needs. Patients with continuing care needs who are discharged to
their homes are checked to see if the ordered services are being delivered, if
their needs have changed, and if they need to be "relinked" with services.
The GAP system is being extended and applied to new areas. For ex-
ample, it serves as the backbone of a major demonstration program sponsored
by the Centers for Disease Control to prevent falls leading to hip fractures
in frail elderly patients.
HALLMARKS OF MEASUREMENT/IMPROVEMENT SYSTEMS
It is probably fair to say that none of the MI systems discussed above
possess all of the desired features needed to be as good as it could possibly
be. It is a fact that all systems can and should-be improved continuously
OCR for page 208
208
EFFECTIVENESS AND OUTCOMES IN HEALTlI CARE
(14~. Nevertheless, these systems share certain characteristics that medical
practices, hospitals, health maintenance organizations, and other providers
could use to both measure and improve outcomes. Chief among them are
the following:
commitment by leaders in the provider organization
to foster improvement,
valid and reliable measures of outcomes.
systematic, repeated assessment of outcomes,
easy to fit into day-to-day pattern of care delivery,
ease of administration, scoring, and interpretation of measures,
directly linked between outcomes measures and improvement efforts,
direct benefit to individuals and groups of patients,
high value placed on system's utility by patients and clinicians
ability to pass information "up-line" and to aggregate it for multisite
efficacy studies and appropriate comparisons, and
to use measurement
.
ability to compare outcomes against those of other providers.
These features, when combined into a working system that is part and
parcel of the caregiving routine, can be very powerful. Such a system
creates a new way of processing and using measures to manage and improve
outcomes. In the right environment one that promotes cooperation on
quality improvement clinicians can work together to improve the system.
USE OF OUTCOMES MEASURES TO
BENEFIT A PATIENT POPULATION
The use of outcomes measures in the aggregate to benefit an entire pa-
tient population, as opposed to benefiting an individual patient, produces
special challenges. The improvement cycle for a patient population is illustrated
in Figure 3. The cycle begins with a population of patients with a selected
health problem or condition. Measurements of structure, process, and outcomes
are taken and then the relationships among them are analyzed to attempt to
determine the "best" upstream settings (elements of structure) and the "best"
upstream actions (processes) that appear to yield the "best" downstream
results (outcomes). A field trial of the "best" upstream conditions is conducted
to determine if they will produce the desired results in multiple settings.
Finally, if the results are positive, this new information is disseminated to
providers.
Even a casual comparison of this cycle with the simpler one for indi-
vidual patients (Figure 1) shows that it is far easier to make improvements
for an individual patient than for a population of patients. It is still harder
to construct an outcome MI program that can help improve an entire system
of care composed of autonomous health care providers.
OCR for page 209
APPLICATION TO CLINICAL PRACTICE
Disseminate to Field
Conduct Trial
of "Best"
Vet
Select Problem
1
Measure, Structure,
Process and Outcomes
Identif y "Best"
Process and Settings
Analyze
Relationships
FIGURE 3 A Measurement/Improvement System for a Population of Patients
209
Recognizing that the challenge that is, how best to use outcomes mea-
sures for improvement- is very great, one might be wise to look outside the
health care industry for guidance. There one would find a new way of
thinking about what quality is and how best to improve it that stresses
continuous improvement of processes (15~. One tool that is being used
widely in quality improvement circles is an activity called "benchmarking."
A recent book by R.C. Camp, an executive at Xerox, describes what benchmarking
is and how to practice it (16~. Camp defines benchmarking this way:
"Benchmarking is the search for industry's best practices that lead to supe-
rior performance." The term "best practices" is equivalent to the term "best
processes" and the term "superior performance" is analogous to "superior
outcomes." Hence the purpose of benchmarking is to search upstream for
the best processes that lead to superior outcomes. Note that the aim is not
to find out who is best able to achieve superior ends. Rather, the goal is to
spot superior outcomes as a way of flagging providers who employ outstanding
processes that might be adapted for use in one's own organization.
Benchmarking, in my opinion, could be a powerful vehicle for improve-
ment in health care if it is a voluntary, provider-based, "from-the-bottom-
up" activity. Benchmarking could succeed if it is undertaken with zeal by
physicians, hospitals, and other providers as a search for the conditions and
processes that are most likely to produce the best outcomes.
Benchmarking, however, is unlikely to be helpful in health care if it is
imposed from the top down. In fact, such a strategy for benchmarking
might be counterproductive. Why? There are many reasons: (1) top-down
benchmarking does not begin with the genuine need felt by most providers
OCR for page 210
210
EFFECTIVENESS AND OUTCOMES IN HEALTH CARE
to find a "better way"; (2) it is likely to produce fear, a desire to protect
one's own position and to discredit the information and its source; (3) the
focus will be on the ends-the outcomes as opposed to the process and the
means for achieving the end; and (4) top-down benchmarking is likely to
foster blind competition among providers rather than useful cooperation.
With these thoughts about the potential power of benchmarking and some
sense of the pitfalls if it is launched in the wrong way, I would like to offer
a few guidelines on how to use outcomes measures for improvement.
· When measuring outcomes over time, one must measure related upstream
conditions in order to understand the outcomes measures.
· It is essential to separate the technical results (often termed clinical end
points or parameters) from the benefits desired or achieved by patients.
· Strive to understand all relevant upstream conditions (settings, processes,
practices, and events) when interpreting outcomes measures.
Identify the key features of the upstream conditions most likely to yield
superior outcomes and conduct a trial to determine if the new way is more
efficacious than the old.
CONCLUSION
Measuring outcomes is important. Improving outcomes is even more
important. Outcomes can be improved by developing dual-purpose measurement/
improvement systems that are useful for individual patients, physicians, and
other providers of care. These systems should link measurement of health
outcomes directly with the care-giving process. They can best be assembled
using a bottom-up, rather than a top-down, approach. This will be more
likely to stimulate the curiosity of providers to make constructive clinical
comparisons and thereby discover better ways for continuously improving
patient care.
REFERENCES
1. Deming, W.E. Out of the Crisis. Cambridge, MA: MIT Center for Advanced
Engineering Study, 1988.
2. Nelson, E.C., Conger, B., Douglass, R., et al. Functional Health Status Levels
of Primary Care Patients. Journal of the American Medical Association 249:3331-3338,
1983.
3. Nelson, E.C., Wasson, J.H., and Kirk, J.W. Assessment of Function in
Routine Clinical Practice: Description of the COOP Chart Method and Preliminary
Findings. Journal of Chronic Diseases 40(Supplement 1~:55S-63S, 1987.
4. Nelson, E.C., Landgraf, J.M., Hays, R.D., et al. The COOP Function Charts:
A System to Assess Functional Health Status in Physicians' Offices. Final report to
OCR for page 211
APPLICATION TO CLINICAL PRACTICE
211
the Henry J. Kaiser Family Foundation. Hanover, NH: Dartmouth Medical School,
1987.
5. Jette, A., Davis, A., Cleary, P., et al. The Functional Status Questionnaire:
Reliability and Validity When Used in Primary Care. Journal of General Internal Medicine
1:143-149, 1986.
6. Rubenstein, L.V., Calkins, D.R., Young, R.T., et al. Improving Patient Func-
tional Status: Can Questionnaires Help? Clinical Research 34:835a, 1986.
7. Calkins, D.R., Rubenstein, L.V., Cleary, P.D., et al. The Functional Status
Questionnaire: Initial Results of a Controlled Trial. Clinical Research 34:359a, 1986.
8. Rubenstein, L.V., McCoy, J.M., Cope, D.W., et al. Improving Patient Func-
tional Status: A Randomized Trial of Computer-Generated Resource and Manage-
ment Suggestions. Paper presented the annual meeting of the American Federation
of Clinical Research, Washington, D.C., May 1989.
9. Nelson, E.C., Hays, R.D., Larson, C., et al. lithe Patient Judgment System:
Reliability and Validity. Quality Review Bulletin 15: 1 85 - 1 9 1, 1 989.
10. Meterko, M., Nelson, E.C., and Ruben, H.R. Patient Judgments of Hospital
Quality: Report of a Pilot Study. Medical Care, in press.
11. Batalden, P.B. and Buchanan, E.D. Industrial Models of Quality Improvement.
Pp. 133-159 in Providing Quality Care: The Challenge to Clinicians. Goldfield, N.
and Nash, D.B., eds. Philadelphia: American College of Physicians, 1989.
12. W.K. Kellogg Foundation. Patient Assessment for Continuing Care: Execu-
tive Summary. Westchester Patient Assessment Program. Battle Creek, MI: W.K. Kellogg
Foundation, 1987.
13. DeVito, C.A. and Zubkoff, W. Discharging the Frail Elderly: One Hospital's
Model Program. Continuing Care 42:26-31, 1989.
14. Berwick D.M. Continuous Improvement as an Ideal in Health Care. New England
Journal of Medicine 320:53-6, 1989.
15. Walton, M. The Deming Management Method. New York: Dodd, Mead &
Company, 1986.
16. Camp, R.C. Benchmarking: The Search for Industry's Best Practices that
Lead to Superior Performance. Milwaukee, WI: ASQC Quality Press and White
Plains, NY: Quality Resources, 1989.
OCR for page 212
Representative terms from entire chapter:
health status
