|
Items in cart [1] |
Questions? Call 888-624-8373 |
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 120
5
CMS Oversight of the Operations and
Management of the QIO Program
CHAPTER SUMMARY
This chapter focuses on the Centers for Medicare and Medicaid
Services (CMS) and its oversight of the operations and manage-
ment of the Quality Improvement Organization (QIO) program,
including communications among the various program partici-
pants, data processing, strategic planning, and program evaluation
and funding. CMS's current management is examined from two
perspectives: the first considers how CMS could improve the opera-
tions of the QIO program in the short term; the second is a longer-
range perspective that considers how CMS and the QIO program
might fit into the operations of a national performance measure-
ment and reporting system once it is fully implemented.
The Centers for Medicare and Medicaid Services (CMS)--specifically
the Quality Improvement Group, which manages the Quality Improvement
Organization (QIO) program--has undertaken considerable discussions re-
garding tools and methods the Medicare program can use at the national
level to enhance the quality of care received by both Medicare beneficiaries
and other patients (Rollow, 2005; personal communication, W. Rollow,
CMS, July 7, 2005). Likewise, long-range issues related to better quality
measurement and reporting, such as the expansion of public reporting and
implementation of pay-for-performance methods, are being discussed at
various levels within CMS (CMS, 2005; McClellan, 2005). The Quality
Improvement Group is planning, under the guidance of a consultant, well
beyond the 8th scope of work (SOW) to consider the role of the QIOs 12
years into the future, during the 12th SOW (W. C. Rollow, unpublished
data, 2005; Rollow, 2005). With a horizon of 2017, it can be tempting to
assume that major changes, such as the universal adoption of electronic
health records (EHRs), will reduce or eliminate certain current problems.
120
OCR for page 121
CMS OVERSIGHT AND MANAGEMENT 121
The committee encourages both short- and long-range planning. It also cau-
tions, however, that managerial and organizational problems of current
concern should be tackled now instead of being deferred until potential
intervening changes occur. At the same time, near-term actions should be
undertaken with the aim of moving the health care delivery system closer to
the long-range goal of producing the right care every time for every person.
The recommendations presented below reflect three basic themes voiced
to the committee by many sources throughout this study: (1) the need to
improve the processing and availability of quality-related information at all
levels throughout the operations of the QIO program; (2) the need to man-
age the program better to increase the likelihood of quality improvements;
and (3) the need for evaluations to determine what quality improvement
methods do and do not work and under what circumstances, to assess the
overall performance of individual QIOs, and to assess the impact of the
QIO program overall on the quality of health care. Fundamental to each of
these themes is the need for adequate funding, which is addressed in the
committee's final recommendation.
DATA PROCESSING
Recommendation 5: The secretary of DHHS and CMS should re-
vise the QIO program's data-handling practices so that data will be
available to providers and the QIOs in a timely manner for use in
improving services and measuring performance.
· CMS should initiate a comprehensive review of its data-sharing
systems, processes, and regulations to identify and correct prac-
tices and procedures, including abstraction of medical chart data,
that restrict the sharing of data by the QIOs for quality improve-
ment purposes or that inhibit prompt feedback to the QIOs and
providers on provider performance.
· The QIO program should support the processes of national re-
porting of performance measures, data aggregation, data analy-
sis, and feedback.
· The secretary of DHHS should allow and encourage the sharing
of medical claims data when the sharing of such data is not pre-
cluded by the privacy protections of the Health Insurance Port-
ability and Accountability Act, as well as the sharing of more
detailed complaint-resolution data with complainants.
· CMS should work toward the ultimate goal of integrating more
care data from all providers and public and private payers to
create both records of patient care over time and population-
level data.
OCR for page 122
122 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
· Independently of the core QIO contract, CMS should be respon-
sible for ensuring and auditing the accuracy of the data submit-
ted by providers that participate in the Medicare program. Pro-
viders should be accountable for the validity and accuracy of the
quality measurement data they submit. The QIOs should supply
providers with technical assistance to improve the validity and
accuracy of the data collected.
Data Processing for Quality Improvement Interventions
The QIO program needs to receive and provide timely data on quality
indicators for use by the QIOs and providers during quality improvement
interventions. Because the program relies on pre- and postintervention mea-
surements both to provide feedback to the participating providers and to
enable CMS to evaluate the QIOs' performance during the 8th SOW, time-
liness is critical. The current 3-year contract cycle is relatively short, given
the time needed to start up new interventions and the remeasurement dead-
line for judging contract performance. For example, data presented by CMS
on accomplishments under the 7th SOW show that the periods between
pre- and postintervention measurements may be as short as 12 months for
hospitals. It will be difficult to demonstrate rapid transformational changes
during the 8th SOW, particularly because the QIOs need time to adjust to
the SOW's new tasks and late release. The feedback loop to all participants
in an intervention becomes too slow to provide guidance on the techniques
that work or that need to be adjusted to achieve better results within the 3-
year period of a SOW (see Chapter 13). Also, some providers value real-
time data feedback to improve care for specific beneficiaries.
Now that CMS publicly reports data on hospitals, nursing homes, and
home health care agencies on at least a quarterly basis, relatively current
data are available for a very limited set of measures at a frequency greater
than that in the past. A rapid expansion of the measures included in the
publicly reported measure sets, as envisioned in the committee's first report,
Performance Measurement: Accelerating Improvement (IOM, 2006), would
increase the amount of timely data that could be used for technical assis-
tance, as well as for QIO performance assessments. Much of the publicly
reported data represents all of a provider's patients, regardless of payer,
and that approach should eventually be expanded to all provider settings.
Until the number of measures collected for public reporting is increased,
CMS should give priority to speeding up the process of data feedback and
should ensure prompt data processing for future publicly reported measures.
Methods for collecting data from paper records in physician offices
need to be improved. In physician practices and facilities lacking EHRs,
QIOs could help with the testing and use of paper-based templates or flow
OCR for page 123
CMS OVERSIGHT AND MANAGEMENT 123
sheets that could be used to collect encounter and performance data in real
time during provider contact with the patient. Properly designed, such docu-
ments could be cost-effective for entering data into electronic databases,
could be less time-consuming than retrospective chart abstraction, and could
serve as a guide for clinicians at the time of service. Retrospective chart
review can also be a valuable tool for research and should not be ignored.
At the same time, it is important to consider how the growing use of
EHRs and the electronic transmission of the data from those records could
enhance the ability of QIOs to offer technical assistance to physicians. The
centralized development of tools that physician offices could use indepen-
dently would aid in the acquisition and use of health information technol-
ogy and would perhaps help the QIO program reach a larger audience.
Some physicians who are already using EHRs could also use specially de-
signed tools on their own to produce aggregated data and analyses to guide
their quality improvements efforts, as well as to facilitate the redesign of
their practices to take advantage of the EHRs. This would in turn allow the
QIOs to concentrate on providers that have less capacity for adjusting to
the new technology and a greater need for guidance, and would benefit
from the experiences of other providers. With the expectation that many
small physician offices will require hands-on assistance with the adoption
of health information technology and that many thousands of practices
may request assistance with office redesign, new methods to fill the need
must be developed.
Confidentiality of QIO Data
There are currently three potential key audiences for QIO data on pro-
vider performance:
· Providers--both those participating in QIO projects who need to
measure their own progress from the baseline and all providers who need
benchmarking data to see how they compare to their peers. In general,
providers tend to show an interest in publicly reported data, and many are
motivated by the reporting of these data to improve their performance
(Hibbard et al., 2005).
· Consumers and beneficiaries--those who may want to compare the
performance of providers on more measures than the handful now publicly
reported and, if QIOs continue to have a role in case review activities, com-
plainants who want to understand the details of their case.
· CMS--which needs data for assessment of the performance of the
QIOs and providers and for case review, complaints, and appeals. In addi-
tion to highlighting specific providers that have quality problems or that
provide high-quality care deserving of recognition by a payment incentive,
OCR for page 124
124 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
the performance data could highlight clinical or geographic areas in need of
quality improvement.
Beneficiary Confidentiality
Even before a national performance measurement and reporting system
is implemented, more data will begin to flow through the health care system
and will receive increased scrutiny as reporting of measures expands.
Greater openness needs to be balanced with the maintenance of protections
for patients, particularly with the growing use of data from patient records.
It will be essential to include strong safeguards in communications systems
to protect the confidentiality of those records. Results of a national con-
sumer survey show that two-thirds of the American public and nearly three-
quarters of members of racial and ethnic minorities are concerned about the
privacy of their personal health information (CHCF, 2005). The current
QIONet Exchange, designed for the transmission of clinical quality data in
the QIO program, is also used for the hospital public reporting program
because it is secure and is compliant with the mandates of the Health In-
surance Portability and Accountability Act. To maintain the confidence of
beneficiaries, it is essential that any new communications systems or expan-
sions of existing systems used for clinical reporting include similar safe-
guards. Precautions concerning access to data at geographic or provider
levels will also need to be taken when the sample or cell size is small enough
to permit identification of individual patients. A national performance mea-
surement and reporting system will need to ensure patient confidentiality
for all data on measures and providers at the same time that it promotes the
transparency of the data.
Greater flexibility in the use of case data is also important for tracking
the quality of services received by a patient during an episode of care or
over a longer period. Longitudinal data that can be tracked across various
providers or settings and linked to a patient identifier are essential to pro-
moting improved care.
Provider Confidentiality
The current confidentiality requirements of the U.S. Department of
Health and Human Services (DHHS) prohibit QIOs from sharing their
data--such as those collected within a quality improvement collaborative,
used in case reviews, or derived from a complaint or appeal--with provid-
ers other than the source of the data unless the source agrees to their use.
These confidentiality requirements also constrain the QIOs from sharing all
but a minimum of information with complainants on how their cases were
resolved. Complainants may receive notice only of whether their care met
OCR for page 125
CMS OVERSIGHT AND MANAGEMENT 125
professionally recognized standards; they may not receive information on
what if any actions were taken as a result. If the physician withholds con-
sent, the QIO cannot release the physician's name or other details to the
complainant (Gaul, 2005). Consumers do not have access to data identify-
ing sources and providers by name because of the same issues of confidenti-
ality; CMS also has limited access to QIO data. (The publicly reported data
from hospitals, nursing homes, and home health agencies are considered
CMS data, not QIO data, and thus are not subject to the same confidential-
ity restrictions.)
The current QIO confidentiality restrictions are not necessary or sup-
portable in the current era of public reporting and are incompatible with
the aim of a national performance measurement system and the goal of
CMS to serve multiple audiences with a transparent system. The committee
recognizes the need to balance concerns about malpractice with the impor-
tance of limiting the discoverability of QIO data. Because the QIO legisla-
tion gives the secretary of DHHS the authority to set confidentiality stan-
dards by regulation, new legislation for this purpose is not necessary
(personal communication, T. Jost, January 7, 2005). Thus the secretary
may establish new regulations that increase the transparency of QIO data
and the ability to share those data. (See Chapter 7 for more detail on confi-
dentiality issues.)
Ensuring the Accuracy of Reported Data
Many current public reporting efforts rely on the QIO program, its
Clinical Data Abstraction Centers, and its national Data Warehouse at the
Iowa QIO to edit, validate, aggregate, and store the reported data (personal
communication, W. Matos, CMS, July 7, 2005; personal communication,
J. Kelly, M. Krushat, and W. Matos, CMS, October 25, 2004; personal
communication, M. B. McClellan, June 24, 2005). The Data Warehouse,
originally designed for QIO data, has grown to accommodate the public
data from hospitals for all payers and will need to grow further as the
reporting of performance measures expands. (See Chapter 13 for more de-
tail on data issues.)
Although public reporting reduces the need for separate abstraction of
data from patient records within the QIO program for some settings, CMS
has an increased responsibility to audit the data that are reported to ensure
their accuracy. Use of the data on quality measures for payment as well as
public reporting purposes increases the incentives to manipulate the system
and the need for accuracy. Although the data-auditing function will be cru-
cial, there is no need for it to be a responsibility of every QIO. In fact, it
would be better for the QIOs not to have this added regulatory function, as
conflicts of interest could arise. Exercising this function could jeopardize
OCR for page 126
126 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
the QIOs' collegial relationships with providers in their states and their
ability to conduct quality improvement interventions with providers, who
participate as voluntary partners. Moreover, in the interest of maintaining
good relations with providers, QIOs might be less than zealous in carrying
out their regulatory auditing function. In addition, there does not appear to
be a strong reason for conducting audits comparing reported data with data
in case records at the state level. A national or regional contractor could
conduct such reviews. Private organizations and QIOs with strong auditing
capacities could compete for such contracts, with the QIOs precluded from
conducting audits in states where they offer providers technical assistance
for quality improvement.
On the other hand, measure validation (consultation with providers on
their initial submissions of data on new measures to ensure that the data
accurately represent what is intended) is an important function the QIOs
could appropriately handle. Because the QIOs have staff available in each
state, they could meet with providers, particularly those in small practices
or those that serve as safety net providers, to help them interpret their data
before the data are publicly posted and to gather comments concerning any
possible problems with the data.
Decisions about the level at which the data repository should be estab-
lished under a national performance measurement and reporting system--
national or subnational--will have to be made as part of the strategy for
that system. In any case, the system will have to be able either to accumu-
late data at the patient level and aggregate them to the local, state, and
national levels or to do the reverse: accumulate the data in a national re-
pository, with the ability to break them down to lower levels of aggregation
for analyses, such as tracking patients across states and time.
QIO PROGRAM MANAGEMENT
Recommendation 6: CMS should establish clear goals and strategic
priorities for the QIO program. Congress, the secretary of DHHS,
and CMS should improve their management of the QIO program
as necessary to support those goals, especially by enhancing con-
tracting processes for the QIO core contract and QIO Support Cen-
ter (QIOSC) contracts; integrating the program's core, support, and
special study contracts; and improving coordination and communi-
cation within the program.
· CMS should provide the QIOs with a coherent and feasible scope
of work that sets forth clear priorities for quality improvement
and performance measurement.
OCR for page 127
CMS OVERSIGHT AND MANAGEMENT 127
CMS's priorities and planning efforts should focus on
integrating QIO collaboration with various types of provid-
ers to improve the coordination of patient care across mul-
tiple settings.
To prepare for the 9th scope of work, CMS should consider
conducting a national survey of the main provider settings
(nursing homes, home health agencies, hospitals, outpatient
physician practices, end-stage renal disease facilities, prescrip-
tion drug plans, and pharmacies) to determine specific unmet
needs for technical assistance. Such information might be
complemented by information from focus groups conducted
with a mix of representatives from the various settings.
The QIO core contracts and the QIOSC contracts should in-
clude incentives aimed at promoting a broader transfer of
knowledge concerning successful quality improvement inter-
ventions and more rapid improvement.
The QIOs should have the resources they need to conduct at
least one locally initiated quality improvement project on the
basis of demonstrated need and the design and evaluation cri-
teria established by CMS.
· Congress and CMS should change the contract structure for core
QIO services for the 9th scope of work:
Strong incentives and penalties that reward high performance
and penalize poor performance should be included. CMS
should encourage sufficient competition for the core contracts
to permit the selection of a QIO contractor on the basis of
contractor-proposed interim and final performance measures
and goals. During the contract period, there should be less
process management of internal QIO operations by CMS.
Congress should permit extension of the core contract from 3
to 5 years to allow for the measurement, refinement, and
evaluation of technical assistance efforts and the achievement
of transformational goals.
There should be greater competition for each new contract.
CMS should consider previous experience and performance
as a QIO among the selection criteria; demonstrated capacity
to support quality improvement on the part of any eligible
organization should predominate.
Performance periods should be consistent. All QIOs should
begin and end the contract cycle on the same date so the plan-
ning, implementation, and evaluation of each scope of work
can be applied nationally.
OCR for page 128
128 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
A timetable should be established for goal setting, program
planning, and funding processes for the core QIO contracts.
The schedule should ensure that new scopes of work are is-
sued in a timely fashion, and that contracts and funding lev-
els are developed and finalized so as to allow sufficient time
for QIOs and competing organizations to prepare in advance
for the new contract without major program and staff
disruptions.
· CMS should award QIOSC contracts several months in advance
of a new QIO contract cycle to allow for the preparation of
tools and materials for QIO use, definition of the required tasks
and deliverables that will serve the QIOs and the Government
Task Leaders, and inclusion of explicit methods for assessment
of the contractor's performance. Congress and CMS should al-
low entities other than QIOs with expertise in quality improve-
ment to bid on QIOSC contracts; familiarity with QIO work,
the capability to carry out the work, and experience in perform-
ing the required functions should be appropriately weighted
when the bids are assessed.
· The QIO core contract and contracts for special studies, support
services, and QIOSCs should all reflect the explicit goals and
priorities of the program.
· CMS and the Agency for Healthcare Research and Quality
should establish ongoing mechanisms for sharing quality im-
provement knowledge and research results, especially through
QIOSCs.
· CMS should take steps to improve coordination and communi-
cations within the QIO program and with QIOs. In particular,
the roles and responsibilities of and communications among
Project Officers, Contract Officers, Government Task Leaders,
Scientific Officers, and QIO executives and their staff should be
clarified.
CMS should build self-assessment, transparency, clearer com-
munications, and continuous quality improvement into the
daily workings of the team overseeing the QIO program, just
as the QIOs expect providers to do.
The contracting function should be subordinate to and sup-
port the program management and business functions.
Ongoing program evaluations (see Recommendation 7)
should provide guidance for the continuous improvement of
program management, coordination, and communications.
OCR for page 129
CMS OVERSIGHT AND MANAGEMENT 129
Priorities
Whereas the 7th and the 8th SOWs included a multiplicity of different
tasks, subtasks, and performance measures and suffered from a lack of pri-
orities, the 9th SOW should be clearly focused on quality improvement and
related activities to support a national system for performance measure-
ment and reporting. At the same time, it will be important to indicate pri-
orities within that narrower focus. Future QIO contracts will need to be
more coherent and should include clearly stated priorities for QIO activities
and precise goals for the SOW overall.
Current quality improvement tasks are subdivided and evaluated by
provider setting; there needs to be a new priority on improving care across
provider settings. As the single largest purchaser of health care in the United
States, the Medicare program spent more than $295 billion in benefit pay-
ments in 2004 to care for 41.7 million beneficiaries. Among the population
over age 65, 84 percent have at least one chronic condition, and 62 percent
have two or more such conditions. Transitions in care from one provider
setting to another are particularly important for individuals who are chroni-
cally ill. The growing need for chronic care among the Medicare population
and the implementation of the Medicare Part D prescription drug benefit
make a crosscutting perspective particularly important. For example, the
continuity of drug therapies as a patient transitions from the hospital to his
or her own home or a nursing home could be tracked as new data became
available from the Part D benefit. Such tracking of the data would provide
an opportunity for the QIOs to integrate the clinical aspects of care, as well
as encourage multidisciplinary collaboration (Schulke, 2004).
Cross-site coordination should build on the initial efforts made under
the 8th SOW and should feature more prominently in future contracts. The
need for quality improvement interventions that follow the patterns of pa-
tient care delivered by a variety of providers in the community will increase
as a national performance measurement and reporting system implements
measures that cut across settings of care, monitor patients over time, and
include composites from more than one provider. The QIOs will be chal-
lenged to broaden their thinking about service delivery and to convene pro-
viders from hospitals, nursing homes, home health agencies, and outpatient
physician practices to work together on filling the gaps in care that occur
when a patient is between settings, a gap through which too many patients
now fall. The program should also help the health care system meet the
challenge of transforming relationships between patients and clinicians, as
well as within health care organizations. Many safety and quality problems,
often resulting from poor documentation and communication, occur dur-
ing the handoff of patients from one setting to another--a point illustrated
OCR for page 130
130 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
by beneficiary complaints, each case of which generally requires the review
of more than one set of provider records (CMS, 2004).
Provider Surveys
It is difficult to plan nationally for the technical assistance projects that
are needed and will be well received by providers without knowing basic
information about providers' internal capacities for quality improvement
and measurement work: what they have done in the past and are currently
doing internally, what assistance they are purchasing externally, and what
gaps remain with which the QIOs could be helpful. Gathering this informa-
tion could be viewed as basic market research to identify potential clients
and their needs. Provider surveys addressing the above questions might serve
multiple purposes beyond a basic needs assessment for QIO assistance. For
example, they could become part of an effort to track quality improvement
trends and serve as a mechanism for identifying and describing best prac-
tices. Although it is unlikely that such a survey could directly contribute to
measurement of the impacts of the QIOs' efforts, it could provide a picture
of the context within which the QIOs operate. In addition to surveys of
each provider setting, it would be useful to conduct some focus groups that
included a mix of providers to help identify needs for crosscutting quality
improvement interventions.
Knowledge Transfer and Local Creativity
Given the strong need for evaluation of quality improvement interven-
tions (as will be discussed further in Recommendation 7), it should be pos-
sible to identify various improvement methods, successful projects, and fail-
ures. QIOs should be encouraged to share their unsuccessful efforts and
their best practices, and to transfer that knowledge broadly both within the
QIO community and to other organizations, such as commercial health
plans and provider organizations seeking to improve quality at the local
and national levels. One type of incentive for knowledge transfer that might
be offered to the QIOs is giving them permission to develop their own
locally designed projects. In addition, similar incentives should be provided
to encourage the publication of articles on quality improvement. For ex-
ample, the evaluation formula in the contract for the 8th SOW provides a
bonus point for QIOs that publish reports on their payment review projects
(see Chapter 10). Alternatively, all QIOs performing at a high level or meet-
ing interim goals could be rewarded for their performance on a local project.
Group award fees are built into the contract for the 8th SOW (see
Chapter 13) in an attempt to encourage QIOs to share and cooperate to-
ward a mutual goal of improvement. The 8th SOW also allows four QIOs
OCR for page 132
132 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
ity to compare results across states, increased emphasis on formal evalua-
tions of individual interventions (see Recommendation 7) would allow bet-
ter comparisons, especially for determining the effectiveness of various qual-
ity improvement techniques. Therefore, QIOs would be able to focus on
achieving the desired result. This approach would also align the QIO pro-
gram with the focus on defined quality improvement goals in the national
system for performance measurement and reporting.
Extension of the QIO contract period from 3 to 5 years would allow
more time for providers to implement quality improvements and measure
the results, as well as to receive feedback and adjust their quality improve-
ment efforts accordingly. With a longer contract period, CMS should per-
form more interim monitoring of the progress of the QIOs. CMS should
also arrange the contract cycle so that all QIOs receive contracts for the
same time period. QIO boards should be held accountable for the opera-
tions of their organizations and should be involved in monitoring perfor-
mance. Shifting all QIOs to the same time frame for contracting rather than
continuing with the current situation of three rounds of contracting stag-
gered over 6 months should streamline QIO evaluations and some other
management functions, as well as improve CMS's ability to compare per-
formance results among QIOs. Effective management planning would be
necessary to process 53 contracts concurrently and to prevent other poten-
tial work-flow bottlenecks. It would be necessary to plan monitoring func-
tions carefully to ensure that they could be completed fairly with limited
staff from the Regional Office. (See Chapter 13 for additional discussion of
contracts.)
Planning for the 9th SOW should move expeditiously to avoid the de-
lays and uncertainties of the process for the 8th SOW. As discussed in Chap-
ter 2, the release of the 8th SOW, expected in the late summer of 2004, was
delayed until the spring of 2005, and significant changes were made as late
as November 2005. Much of the delay appeared to be related to internal
departmental debates and budgetary discussions. Because these discussions
would need to occur only every 3 or 5 years if the QIO contracts were
extended as proposed by the committee, it would be helpful for the pro-
gram office to establish an ongoing dialogue with the key parties to keep
them informed of current program progress, as well as CMS's thoughts
about the next SOW. Explicit planning and drafting of each SOW should
begin early enough to permit its timely release by CMS so that QIOs will
have adequate time to prepare for the SOW, including program planning
and staffing. Mindful of the time needed to prepare adequately for a new
SOW and the fact that this report will be released after the start of the 8th
SOW, the committee formulated most of its recommendations with a view
toward the 9th SOW rather than recommending many hasty, major revi-
sions to the 8th SOW.
OCR for page 133
CMS OVERSIGHT AND MANAGEMENT 133
QIO Support Centers and Other Contracts
The intent of the QIO Support Centers (QIOSCs) is to serve QIOs as
national technical resources, and QIOSCs are often expected to offer train-
ing programs for QIO staff and prepare materials all QIOs can use to carry
out their tasks. Therefore, the QIOSCs need a head start on each new SOW.
In particular, it is important that they have materials ready for the QIOs
when the new contract period begins. (See Chapter 7 for detail on QIOSC
structure and Chapter 13 for discussion of QIOSC contracts.) This is a par-
ticular challenge for QIOSCs that have not conducted special studies or
served in this capacity in the past, and it could be a problem for experienced
QIOSCs as well if the substance of the QIO tasks should change signifi-
cantly. Having the next SOW well defined before the current contract is
completed would allow CMS to contract for QIOSCs a few months before
the commencement of the SOW. Doing so would become feasible if the
QIOSC contract were opened to other entities with the requisite expertise.
Currently, only QIOs may act as QIOSCs; therefore, QIOSC contracts can-
not be awarded until the bidders know they have received a QIO core
contract.
The QIOSCs sometimes face competing demands to serve the QIOs
that need assistance, as well as the Government Task Leader. The QIOSC
contract should define in more detail, to the extent feasible, the specific
tasks expected of the QIOSC for each audience and the measures to be used
for evaluating its performance.
CMS should consider including in the new SOW for the QIOs and
QIOSCs the development of national resource teams. These teams would
enable QIOs to work collaboratively with multistate providers to promote
quality improvement by reducing the barriers encountered by corporate
providers operating in more than one state. This task should be based on a
thorough assessment of the Corporate Nursing Home Collaboratives, oper-
ated through the Nursing Home QIOSC and the Colorado Foundation for
Medical Care during the 7th SOW. This task coincides with the above-
discussed need for CMS and the QIO program to consider more of the
crosscutting issues of health care delivery.
The Quality Improvement Group in CMS administers not only the QIO
core contracts and QIOSC contracts, but also other contracts with QIOs
and other organizations that are intended to support the QIO program and
quality improvement in general, as defined more broadly within the Medi-
care program. These other contracts would have greater synergistic value if
better tracking and management systems were in place to oversee them and
ensure their coordination with program priorities. (See Chapter 7 for more
detail on contracts.) For example, if the QIOs had a list of all the special
studies and the QIOs holding the contracts for those studies, they might use
OCR for page 134
134 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
this list as both an indicator of expertise on a topic and a guide to the topics
under study. CMS should coordinate with other entities, such as the Agency
for Healthcare Research and Quality, to facilitate ongoing discussions of
research results. QIOSCs are one mechanism through which this knowl-
edge transfer could readily occur.
Management Relations
The development of a new and substantially different SOW presents an
opportunity for CMS to assess the different roles, responsibilities, and com-
munications systems that will be needed for all QIOs to accomplish the new
tasks. To ensure a smooth transition to the new SOW, all participants will
need to understand CMS's priorities and policies, as well as those of a na-
tional performance measurement and reporting system as it develops. Per-
haps some of the lessons QIOs learn in working with providers to improve
their communications across various health care settings could be applied
to better link the many offices within CMS that are involved with process-
ing QIO contracts and working with the contractors. Given the number of
personnel involved in running the QIO program, CMS should seek to clarify
the roles of all these individuals in light of the defined priorities of the
SOW. (See Chapter 13 for further detail on CMS management.)
QIO PROGRAM EVALUATIONS
Recommendation 7: CMS should develop four types of evaluation
to assess the QIO program. CMS should conduct three of these
four types of evaluation internally to assess QIO performance
against predetermined goals and priorities at the following levels:
(1) the program as a whole, (2) individual QIOs with respect to the
core contract, and (3) selected quality improvement interventions
implemented by QIOs. DHHS should periodically commission the
fourth type of evaluation--independent, external evaluations of the
QIO program's overall contributions.
· The QIOs should be learning organizations, continually improv-
ing the assistance they offer to health care providers. CMS should
develop explicit benchmarks for use in ongoing measurement of
progress on the effectiveness and costs of the program.
· CMS should form a technical expert panel to offer ongoing guid-
ance on the design of the three types of internal CMS evalua-
tions, including options for identifying optimally performing
QIOs, as well as methodologies for attributing quality improve-
ments to the QIO program's interventions.
OCR for page 135
CMS OVERSIGHT AND MANAGEMENT 135
· CMS should ensure that evaluations of the effectiveness of qual-
ity improvement interventions are conducted. The committee
suggests that CMS should use the most rigorous evaluation de-
signs practicable, including randomized controlled trials. This
approach should also contribute to CMS's overall program
evaluation.
Evaluations should include concurrent, qualitative descrip-
tions and assessments of the nuanced nature of the QIOs' role
in quality improvement interventions and the roles of other
players.
As appropriate, evaluations should be stratified among pro-
vider settings and across states and regions.
CMS should assess the cost-effectiveness of each type of inter-
vention to assist with the allocation of resources.
· The secretary of DHHS should allocate adequate funds from the
QIO apportionment to carry out, on an ongoing basis, both in-
ternal and external evaluations.
Evaluation of Progress Toward Goals
As discussed in Recommendation 6, CMS must set explicit overall goals
and priorities for the QIO program, in part because goals are needed to
create a robust evaluation plan for determining achievement. These goals
should be considered in conjunction with those already established that
also focus on enhancing health, such as those set for the public's health by
Healthy People 2010 (HHS, 2000) and the National Healthcare Quality
Report (AHRQ, 2004), when applicable. Evaluation of the program's suc-
cess in achieving those goals, as well as its overall impact, has been largely
ignored, and the focus has remained on evaluation of QIO contract perfor-
mance alone. Although contract evaluation is needed, CMS must initiate
plans to evaluate the QIO program as a whole and the effectiveness and
impacts of individual intervention approaches.
Levels of Evaluation
Four levels of evaluation are important. The first is evaluation of the
program as a whole to help prove its value and efficiency in the arena of
health care quality improvement. As noted above, explicit goals against
which the program can be evaluated must be set by CMS. This level of
evaluation should be focused not only on determining the success of the
program in improving quality overall, but also on such factors as CMS's
oversight of the program's management and operations and the effective-
OCR for page 136
136 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
ness and value of the support services provided within the program. With-
out such program-level evaluation, it is difficult to justify a specific appor-
tionment and budget.
Second, it is important for CMS to assess the performance of individual
QIOs because under a performance-based contract, the QIOs are compen-
sated for achieving specific results. It is necessary to understand which QIOs
are excelling across all their quality improvement tasks, not just in a par-
ticular provider setting. Thus evaluations must be used to determine the
success of individual QIOs in achieving the results delineated in their con-
tracts. The results of such assessments can help CMS judge the bids for
future QIO contracts. These evaluations can also contribute to setting the
bar for success. For example, if some QIOs are able to achieve much higher
levels of improvement, it may be that other QIOs could achieve the same
levels of improvement if encouraged to do so through knowledge transfer,
goal setting, and other methods. To the extent appropriate, QIO boards
should be involved in the monitoring and evaluation of their respective
organizations.
Third, evaluations of the QIOs are linked to the need for CMS to incor-
porate requirements for rigorous intervention study designs into the QIO
contract. For example, approval of local projects could be contingent on
plans to compare the results with those of other QIO projects. With a vari-
ety of QIOs carrying out different quality improvement interventions, CMS
is uniquely positioned to determine which intervention methods lead to
higher levels of quality improvement. The QIO program should be viewed
as a learning enterprise that fosters constant improvement, innovation, and
transfer of knowledge.
Finally, periodic evaluations of the QIO program as a whole should be
performed by an external entity to provide an independent point of view
and valuable information for CMS on the systems design and operations
management aspects of the program that need improvement. Such external
evaluations should also address the overall impact of the program.
CMS may choose to contract with others for assistance in all or a por-
tion of its internal evaluations. When appropriate, CMS should collaborate
with universities or others skilled in study design and evaluation to develop
the strongest evaluative designs possible. CMS should also appoint and use
an ongoing technical expert panel to provide guidance on the design of its
internal evaluations and of future SOWs to enhance collection of the neces-
sary data. This is not the first time the IOM has recommended an overall
evaluation of the QIO program. In its 1990 report Quality Assurance in
Medicare, the IOM mentioned the need to document the impact of the or-
ganization in each state on the quality of care (IOM, 1990). In 1994, the
IOM was asked to review the evaluation plan of the Health Care Financing
OCR for page 137
CMS OVERSIGHT AND MANAGEMENT 137
Administration (the predecessor of CMS) for the Peer Review Organization
program (the predecessor of the QIO program). At that time, the IOM
committee recommended both formative evaluations, to learn how to im-
prove ongoing projects, and a summative evaluation of the whole program's
achievements relative to its goals and objectives (IOM, 1994). The 1994
program evaluation was not conducted, although later almost two dozen
quality-of-care measures were tracked for each state over time and were
summed to obtain national measures (Jencks et al., 2003). The authors
could not attribute the improvements in health care quality to the QIO
program. The current IOM committee reiterates the ongoing need for evalu-
ations at both the national program level and the state and project levels.
While recognizing the challenges involved in conducting sound evalua-
tions of quality improvements and their causes (see Chapter 9), the commit-
tee is convinced that evaluations of the QIO program must be given high
priority. Not only do the technical assistance methods used by QIOs with
identified participants need to be examined, but the techniques they employ
to improve quality statewide also need to be reviewed. Because the QIOs
spent twice as much money on statewide efforts as on identified partici-
pants during the 7th SOW, it is important to learn whether these funds
were well spent and what was gained. Although studies showing how much
more identified participants improved relative to other providers in the state
are important in identifying certain impacts of the QIOs, they do not indi-
cate whether statewide investments were productive. As noted above, QIOs
should be charged as a part of their contract with proposing explicit plans
for rigorous evaluations based on CMS guidance and methodological op-
tions. These evaluations should be part of the QIOs' interventions and
should allow for formal and complete assessment of the effectiveness of
those interventions.
To date, the QIOs have published a paucity of studies showing the
effects of interventions on quality (see Chapter 9). Collecting data from
well-designed studies of the quality improvement interventions initiated by
each QIO can yield a compendium of information on the effectiveness of
specific interventions in multiple states that would contribute greatly to the
database on which interventions produce the best results. In addition, CMS
and the QIOs should assess how different types of providers selected by
various methods to participate in quality improvement projects affect the
improvement achieved. In these ways, the program can help build an evi-
dence base for determining the most cost-effective ways to improve popula-
tion health. The dissemination of this knowledge should contribute to
greater improvements in the future.
OCR for page 138
138 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
Types of Study Designs
Several types of studies may be considered or combined when evalua-
tion plans are designed, and CMS should choose the strongest designs pos-
sible. CMS should take advantage of the opportunity presented by the ex-
istence of 53 QIOs, namely, a naturally occurring experiment with the
potential to become a large observational study. The choice of study de-
signs is not easy, and many complex decisions must be made. As stated
above, CMS should consider the use of an ongoing technical expert panel to
provide guidance on the design of evaluation plans for individual quality
improvement interventions, as well as on the design of evaluations of the
QIOs themselves. The methodologies that should be considered include the
following:
· Randomized controlled trials
· Time series, crossover analyses
· Studies with nonequivalent control groups
· Case-control studies
· Qualitative analyses
Each of these types of design has advantages and disadvantages, which
must be carefully considered before an evaluation is undertaken. One major
challenge in implementing any study design concerns taking into account
the voluntary nature of the program. The challenge is to separate the effects
of an intervention from those due to differences in the providers seeking to
participate in studies, which lead to selection bias (Campbell and Stanley,
1963; Wholey et al., 2004). Randomized controlled trials are particularly
problematic for public programs such as the QIOs; nonetheless, their use
should be considered as they may be suitable under specific limited circum-
stances. (See Appendix C for descriptions of how each of the above meth-
odologies might be applied to aspects of the QIO program when formal
evaluation plans are designed.)
QIO PROGRAM FUNDING
Recommendation 8: Congress and the secretary of DHHS should
focus all QIO resources on supporting health care providers' per-
formance measurement and quality improvement efforts. The sec-
retary should remove from QIO core contracts funds sufficient to
support case reviews, appeals, and beneficiary complaints when
those functions are devolved to other organizations. The secretary
should increase the remaining funds to allow for inflation, the in-
corporation of evaluations into all QIO work, the increased num-
OCR for page 139
CMS OVERSIGHT AND MANAGEMENT 139
bers of providers and beneficiaries being served, and the labor-
intensive nature of technical assistance and quality improvement
activities.
· The multiple evaluations undertaken during the 8th and 9th
SOWs should guide future funding decisions, with budget in-
creases or decreases being provided according to the evaluation
findings. If the evaluations demonstrate that no positive impact
is attributable to the QIO program's efforts, CMS will need to
rethink its quality improvement approach and the possible ben-
efit of transitioning funds to an alternative structure and strat-
egy for Medicare.
· Once a national performance measurement and reporting sys-
tem has been established, its priorities should help guide the
funding levels and policy direction of the QIO program, recog-
nizing that adequate funding is necessary to reach the goals set
for the QIO program.
· The secretary of DHHS should ease the conflict-of-interest re-
striction with regard to supplementing the QIO quality improve-
ment budgets with external funds. Given the limits of federal
funding, the QIOs should be allowed to seek funds for quality
improvement activities from providers and other organizations
as appropriate.
The tasks confronting the QIOs during the 8th SOW are of great mag-
nitude, and as discussed in Chapter 4, the committee expects that provider
demand for technical assistance with quality improvement will increase.
The committee recognizes the huge gap between what is known about qual-
ity care and the care that is delivered to most patients. It also understands
that many health care providers and practitioners need help in making the
changes necessary for the consistent delivery of high-quality care. The com-
mittee recognizes as well that although the QIOs should no longer be re-
sponsible for case reviews, complaints, and appeals, CMS will need to con-
tract with other organizations to perform at least some of those functions.
Thus, a portion of the current funding should be shifted out of the QIOs'
budgets after the 8th SOW. During the 9th SOW, additional funds will be
necessary to cover the increased demand for technical assistance from pro-
viders and extensive program evaluations. (See Chapter 7 for further dis-
cussion of funding.)
Before increased funding can be justified past the 9th SOW, however, it
is necessary to assess the impacts the QIOs are having on providers at
present. It would be unrealistic to expect CMS and the QIO program to
perform evaluations with levels of rigor rivaling those of double-blind, ran-
OCR for page 140
140 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
domized controlled drug trials, but certainly more could be learned about
the impact of the program if evaluation were a priority. Decisions on how
future funds are to be spent should be based on the results of all evaluations
of the overall impact of the QIO program, the technical assistance provided
for performance measurement, the specific quality improvement methods
and techniques used, and which QIOs are most successful. If those evalua-
tions indicate that the QIO program is having a significant positive impact
and if there is continuing demand for technical assistance for quality im-
provement or new tasks are added, future funding should be increased.
Information resulting from the evaluation is essential as a guide to the wise
expenditure of additional funds. If the evaluations show that the program
has an insufficient impact, CMS must consider other strategies for improv-
ing the quality of health care and should shift funding accordingly.
At the same time, however, it is important to recognize that adequate
funding in proportion to contract requirements is necessary to accomplish
the goals established for the program. In 1990, the IOM determined that
investment in the QIO program was inadequate to achieve set goals (IOM,
1990). Since then, the program has added more task areas, while funding
has become a smaller percentage of the overall Medicare budget (see Chap-
ter 7). Additionally, while most of the QIOs' current expenditures for Tasks
1a1d (quality improvement activities in nursing homes, home health set-
tings, hospitals, and physician offices) go toward overall statewide activi-
ties, work with identified participants often incurs a higher cost per pro-
vider (see Chapter 7). For example, under the 7th SOW, QIOs worked with
approximately 7.5 percent of physician offices. If demand for help increases
in this provider setting, budget increases will be necessary because of the
greater cost of such activities per provider in this setting (as compared with
overall work with providers). At present, however, it is infeasible to esti-
mate the budget requirements for these activities in the 9th and 10th SOWs
because of a variety of unknown factors, including the scope of the in-
creased demand, the effects of pay-for-performance and public reporting
initiatives, and the possible development of more cost-effective tools for use
by QIOs in reaching out to a larger number of providers.
Additional funds will also be needed to enhance the QIO program's
role in supporting the implementation of a national performance measure-
ment and reporting system and the capacity of providers to participate in
the reporting of quality measures and pay-for-performance programs. Once
a national performance measurement and reporting system is functioning,
has established goals, and has identified additional measures for reporting,
its priorities should guide the policy direction of the QIO program. None-
theless, CMS and the QIO program also have responsibilities to the Medi-
care program. To the extent possible, efforts should be made to bring those
OCR for page 141
CMS OVERSIGHT AND MANAGEMENT 141
responsibilities into alignment with the operation of the national perfor-
mance measurement and reporting system.
Conflict-of-interest rules currently restrict QIOs from soliciting or re-
ceiving external funding from providers or grants for the provision of the
same services rendered under the QIO contract (see Chapter 7). Many QIOs
have already exhibited an ability to contract with non-Medicare sources for
quality improvement and case review functions. These contracts are unre-
lated to their core contract and state providers. However, some local pro-
viders might want to contract with their local QIO for services similar to
those provided by the QIO as part of the Medicare program. For example,
a hospital might want to pay the QIO for extra or more intensive assistance
in its internal quality improvement efforts to improve upon measures ex-
plicitly delineated in the QIO contract. In the past, the restriction on such
arrangements prevented a conflict for QIOs, which would be accepting
money from the same providers they might also review for any number of
regulatory activities. If the QIOs' efforts were focused on quality improve-
ment and performance measurement (see Recommendations 3 and 4), that
inherent conflict would no longer exist. Additionally, increased interaction
with the private sector could boost the QIOs' reputations, add to their skill
sets, and enhance publicprivate collaboration. Nonetheless, CMS should
ensure that the proportion of funds from providers and other private sources
is not so large relative to the Medicare core contract that it gives the impres-
sion of a conflict of interest or impropriety. Also, QIOs need to prevent
perceptions of bias toward paying contractors that might arise if resources
for free assistance became scarce. Finally, by removing this restriction, more
organizations with multiple lines of business might be enticed to compete
for QIO contracts.
REFERENCES
AHRQ (Agency for Healthcare Research and Quality). 2004. National Healthcare Quality
Report. Rockville, Maryland: Agency for Healthcare Research and Quality.
Campbell DT, Stanley JC. 1963. Experimental and Quasi-Experimental Designs for Research.
Chicago: Rand McNally & Co.
CHCF (California Health Care Foundation). 2005. Executive Summary. In National Con-
sumer Health Privacy Survey 2005. Bishop L, Holmes BJ, Kelly CM, eds. Oakland, CA:
California Health Care Foundation.
CMS (Centers for Medicare and Medicaid Services). 2004. The Quality Improvement Organi-
zation Program: CMS Briefing for IOM Staff. [Online]. Available: http://www.
medqic.org/dcs/ContentServer?cid=1105558772835&pagename=Medqic%2FMQ
GeneralPage%2FGeneralPageTemplate&c=MQGeneralPage [accessed December 26,
2005].
CMS. 2005. Quality Improvement Roadmap. [Online]. Available: http://www.medical
devices.org/public/issues/documents/CMSMedicareroadmap.pdf [accessed December 26,
2005].
OCR for page 142
142 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
Gaul GM. 2005, July 26. Once health regulators, now partners. Washington Post. p. A1.
HHS (United States Department of Health and Human Services). 2000. Healthy People 2010:
Understanding and Improving Health. Washington, D.C.: U.S. Government Printing
Office.
Hibbard JH, Stockard J, Tusler M. 2005. Hospital performance reports: Impact on quality,
market share, and reputation. Health Affairs 24(4):11501160.
IOM (Institute of Medicine). 1990. Quality Assurance in Medicare. Washington, DC: Na-
tional Academy Press.
IOM. 1994. An Assessment of the HCFA Evaluation Plan for the Medicare Peer Review Orga-
nization. Washington, DC: National Academy Press.
IOM. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The
National Academies Press.
Jencks SF, Huff ED, Cuerdon T. 2003. Change in the quality of care delivered to Medicare
beneficiaries, 19981999 to 20002001. Journal of the American Medical Association
289(3):305312.
McClellan MB. 2005, July 21. Statement of Mark McClellan, M.D., Ph.D., Administrator,
Centers for Medicare and Medicaid Services, U.S. Department of Health and Human
Services. Testimony before the Subcommittee on Health of the House Committee on
Ways and Means. Washington, DC: U.S. House of Representatives.
Rollow WC. 2005. QIO Program: Update and Policy Considerations. PowerPoint Presenta-
tion to the Committee on Redesigning Health Insurance, June 13, Washington, DC.
Schulke D. 2004. Opportunities for Quality Improvement Created by the Medicare Drug
Benefit. PowerPoint Presentation to the Committee on Redesigning Health Insurance,
December 13, Washington, DC.
Wholey JS, Hatry HP, Newcomer KE. 2004. Handbook of Practical Program Evaluation. San
Francisco: John Wiley & Sons, Inc.
Representative terms from entire chapter:
uncorrected proofs
