|
Items in cart [1] |
Questions? Call 888-624-8373 |
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 325
13
CMS Oversight
CHAPTER SUMMARY
This chapter examines how the Centers for Medicare and Medicaid
Services (CMS) oversees and manages the Quality Improvement
Organization (QIO) program as a whole. First, the integration of
the program into CMS's organizational structure is discussed, in-
cluding the use of personnel who help with oversight. Next, com-
munications, information technology, and data services are dis-
cussed both in the context of how they are used in the operations
of the program and how they are used as a resource for manage-
ment. Then, contract issues are presented, including how contracts
are competed, awarded, implemented, and monitored. Finally,
there is an examination of how CMS provides overall guidance to
the Quality Improvement Organization program through strategic
planning, policy decision making, coordination, and overall pro-
gram evaluation.
ORGANIZATIONAL STRUCTURE OF QIO PROGRAM IN CMS
Oversight of the Quality Improvement Organization (QIO) program
involves coordination of the efforts of multiple personnel in several offices
within the Centers for Medicare and Medicaid Services (CMS), each of
which has distinct roles. The administrative office of CMS, located in Balti-
more, Maryland, is commonly referred to as the "Central Office." Two
offices within CMS's Central Office share the responsibility for manage-
ment of the QIO program: the Office of Clinical Standards and Quality, the
"Program Office," and the Office of Acquisition and Grants Management,
the "Contracts Office." Other groups have indirect roles in the manage-
ment of the QIO program. The QIO and End-Stage Renal Disease Steering
Committee manages the daily operations of the QIO program. The mem-
bership on the QIO and End-Stage Renal Disease Steering Committee com-
prises the Associate Regional Administrator for each of the four Regional
325
OCR for page 326
326 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
Offices affiliated with the QIO program and both the director and the
deputy director of each of three groups within the Office of Clinical Stan-
dards and Quality: the Quality Improvement Group, the Information Sys-
tems Group, and the Quality Measurement and Health Assessment Group.
The committee, currently chaired by the director of the Quality Improve-
ment Group, meets weekly and primarily discusses operational issues (per-
sonal communication, J. V. Kelly, June 28, 2005).
Program Office
Overall responsibility for the QIO program lies in CMS's Office of
Clinical Standards and Quality, with direct oversight provided by the Qual-
ity Improvement Group (Jost, 1991; CMS, 2004c) and with support pro-
vided by other groups within that office. The Program Office monitors the
QIO program, coordinates with the Office of Internal Customer Support
on financial matters, and creates and interprets policy related to the QIO
program's operations. The office is divided into six groups, each of which
may have one or more of the following divisions:
· Quality Improvement Group,
· Quality Measurement and Health Assessment Group,
· Information Systems Group,
· Quality Coordination Team,
· Coverage and Analysis Group, and
· Clinical Standards Group.
In the Institute of Medicine (IOM) committee's web-based data collection
tool, 52 QIOs rated the Program Office on several functions. Overall, the
office received higher scores on "clarity" than on "timeliness" (Table 13.1).
Concerns over clarity and timeliness also arose during the IOM com-
mittee and staff site visits. Four QIOs mentioned that the information that
they receive is often ambiguous, and eight related frustration with the time-
liness of access to information or data related to their tasks (referred to here
as data lags). Data lags, however, may also be attributable to the measure-
ment process, based on claims (this is discussed more later in this chapter).
Contracts Office
Many groups contribute to the development of a QIO contract, includ-
ing the Office of Clinical Standards and Quality, the Office of Acquisition
and Grants Management, and Regional Office Divisions of Quality Im-
provement (CMS, 2004b). However, responsibility for the QIO contract
ultimately rests with the Acquisition and Grants Group of the Office of
OCR for page 327
CMS OVERSIGHT 327
TABLE 13.1 QIO Ratings of CMS Program Office
Clarity of
Overall Overall Clarity of Timeliness of
Program Support of Information Information on
Ratings Direction QIO Work on Core Tasks Core Tasks
Excellent 2 3 1 0
Good 33 33 26 8
Fair 15 9 22 30
Poor 2 7 3 14
NOTE: The data in the table represent the number of QIOs responding as indicated. Data
are for a total of 52 QIOs.
SOURCE: IOM committee web-based data collection tool.
Acquisitions and Grants Management. The Contracting Officer, a repre-
sentative of the Acquisition and Grants Group, is the only person with the
authority to release the contract or make modifications to the contract. The
Contracting Officer oversees all contracts for the QIO program, and several
contract specialists are each assigned to specific QIOs. As of June 2005,
nine contract specialists were each assigned to work directly with between
five and seven QIOs (personal communication, J. V. Kelly, June 30, 2005).
The QIOs expressed frustration with their interactions with the Con-
tracts Office. During the site visits, two QIOs raised issues about conflicting
messages between the Program and Contracts Offices. Additionally, at
CMS's annual technical conference for the QIO program (QualityNet
2004), many QIO staff related difficulties with being asked to perform du-
ties not specified within their contracts. They were asked to perform these
duties by different sources, such as their Program Officer or Government
Task Leader, or through a Transmittal of Policy System (TOPS) document
(all of these are described later in this chapter). Although CMS presenters
clarified that the Contracting Officer has the final say on required duties,
the QIOs expressed frustration with conflicting messages from different
individuals and groups at CMS (Hughes, 2004).
The QIOs rated the contracts office on many functions. Thirty-five of
52 QIOs stated that they had interaction with the Contracts Office only on
an as-needed basis. The majority of QIOs rated the Contracts Office as
"good" or "fair" on all questions (Table 13.2).
Regional Offices
CMS has 10 Regional Offices around the country. In four of these Re-
gional Offices (Boston, Dallas, Kansas City, and Seattle), CMS established
OCR for page 328
328 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
TABLE 13.2 QIO Ratings of CMS Contracts Office
Overall Overall Timeliness Expertise/
Clarity of Timeliness of of Contract Understanding
Ratings Communications Communications Modifications of QIO Tasks
Excellent 6 6 3 4
Good 26 25 27 19
Fair 14 17 15 19
Poor 6 4 7 10
NOTE: The data in the table represent the number of QIOs responding as indicated. Data
are for a total of 52 QIOs.
SOURCE: IOM committee web-based data collection tool.
Divisions of Quality Improvement that act as liaisons between the QIOs
and CMS's Central Office (Jost, 1991; CMS, 2004b). The remaining six
CMS Regional Offices do not have any direct responsibility for the QIO
program. The four Regional Offices with Divisions of Quality Improve-
ment (referred to in the QIO program as "Regional Offices") assist QIOs
with technical issues on a daily basis by interpreting CMS policy, monitor-
ing finances, and providing feedback.
The staff of the Divisions of Quality Improvement include an Associate
Regional Administrator, Project Officers, and Scientific Officers. The Asso-
ciate Regional Administrator oversees daily operations, including develop-
ment and the implementation of goals, participation in consortium meet-
ings, maintenance of stakeholder relationships, and management of funds
(CMS, 2004b). Before the 7th SOW, Divisions of Quality Improvement
existed in all 10 CMS Regional Offices and were generally staffed only by
Project Officers. As the program focus shifted toward quality improvement,
oversight was condensed into the four Regional Offices mentioned above,
as new skills were needed to parallel the skills needed at the QIO level. New
staff included epidemiologists, clinicians, biostatisticians, data managers,
and communications specialists (CMS, 2004b). Today, staffing at each Re-
gional Office varies in terms of both the numbers of personnel and the skill
sets of those personnel (CMS, 2004b).
CMS also divided the country into four consortiums that correlated
with the four Regional Offices with Divisions of Quality Improvement.
These consortiums (Northeast, Midwest, Southern, and Western) include
the one Regional Office's with QIO oversight in that area and any other
Regional Offices in that area that are not directly involved in the QIO pro-
gram. The consortiums act to improve communications and share resources
among the 10 Regional Offices and enhance consistency in the QIO pro-
gram as a whole (CMS, 2004b).
OCR for page 329
CMS OVERSIGHT 329
Project Officers
Project Officers monitor technical aspects of the QIO core contract
(CMS, 2004b). All Project Officers participate in a week-long basic training
session, with some officers completing optional advanced Project Officer
training or performance-based contracting training. Each QIO is assigned
one Project Officer, but a single Project Officer works with multiple QIOs.
The QIOs reported that they have frequent contacts with their Project Of-
ficers: half (26 of 52) reported weekly contact, and 92 percent (48 of 52)
reported at least monthly contact (Figure 13.1).
The Project Officer provides direct technical assistance to each QIO,
serves as the advocate for the QIO within CMS, and is an expert resource
for the QIOs in terms of contract content and CMS policy. The Project
Officers manage QIO contracts by monitoring the progress of the QIOs,
acting as a direct liaison to the Contracting Officer at CMS, and participat-
ing in strategic planning. Monitoring activities include scheduled calls with
individual QIOs and review of the data on the Dashboard section of CMS's
intranet site (see below). Official monitoring visits are discussed in greater
detail later in this chapter. The Project Officers also have communications
and coordination responsibilities at both the local and the national levels.
Table 13.3 shows the number of full-time Project Officers at each Regional
Other, 1
As needed, 3
Monthly, 14
Weekly, 26
Semi-monthly, 8
FIGURE 13.1 Frequency of Project Officer contact with QIOs reported by 52 QIOs.
The numbers in the figure represent the number of QIOs responding as indicated.
SOURCE: IOM committee web-based data collection tool.
OCR for page 330
330 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
TABLE 13.3 Numbers of Project Officers and Contracts for Each CMS
Regional Office
Regional Number of Number of Average Number of
Office Project Officers QIO Contracts Contracts per Project Officer
Boston 5 16 3.2
Dallas 4.6 11 2.4
Kansas City 4 13 3.25
Seattle 4 13 3.25
SOURCE: CMS (2004b).
TABLE 13.4 QIO Ratings of Project Officers
Expertise/ Expertise/
Clarity of Timeliness Understanding Understanding
Rating Responsesa of Responsesa of Review Tasks of HCQIP Tasks
Excellent 34 36 25 21
Good 13 13 19 24
Fair 4 3 7 6
Poor 1 0 1 1
NOTE: The data in the table represent the number of QIOs responding as indicated. Data
are for a total of 52 QIOs. HCQIP = Health Care Quality Improvement Program.
aResponses to questions raised or issues posed by the QIO.
SOURCE: IOM committee web-based data collection tool.
Office, as well as the total number of QIO contracts monitored in that
region as of June 2004 (CMS, 2004b).
The QIOs rated the Project Officers on various functions. Overall, the
Project Officers received high ratings in all areas, with the majority of
QIOs rating their Project Officers as "excellent" or "good" in each area
(Table 13.4).
Scientific Officers
The Scientific Officers support the Project Officers by providing scien-
tific and clinical expertise (CMS, 2004b). Scientific Officers are not as-
signed to specific QIOs but, instead, assist all QIOs in the region covered
by the Regional Office with specific technical needs. They also assist the
OCR for page 331
CMS OVERSIGHT 331
Semi-Weekly, 6
Only as needed, 17
Quarterly, 3 Monthly, 22
FIGURE 13.2 Frequency of Scientific Officer contact with QIOs reported by 48
QIOs. The numbers in the figure represent the number of QIOs responding as
indicated.
SOURCE: IOM committee web-based data collection tool.
QIOs in other regions, if they are requested to do so. As of June 2004, the
Boston Regional Office had five Scientific Officers on staff, and the three
other Regional Offices each had four Scientific Officers (CMS, 2004b).
The QIOs reported extremely variable interactions with the Scientific Of-
ficers (Figure 13.2).
Scientific Officers evaluate measurement methodologies and surveys,
analyze QIO data, review manuscripts, provide clinical expertise, and man-
age special studies (CMS, 2004b). Scientific Officers possess specific skills
in areas such as statistics, epidemiology, clinical science (Medical Officer),
and data management. Scientific Officers may complete any of the training
sessions described for Project Officers, but they are not required to do so.
Scientific Officers also participate in official monitoring visits, described
later in this chapter. In addition to their basic duties, Scientific Officers
often serve as Government Task Leaders (see below).
Table 13.5 shows the QIO ratings of Scientific Officers on a variety of
functions. In general, QIOs rated Scientific Officers highly in all areas,
with most QIOs providing "excellent" or "good" ratings for their Scien-
tific Officers.
Government Task Leaders
Each task of the QIO contract and each special study are assigned a
single Government Task Leader to provide direct oversight. The Govern-
OCR for page 332
332 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
TABLE 13.5 QIO Ratings of Scientific Officers
Clarity of Timeliness Timeliness of
Rating Responsesa of Responsesb Manuscript Reviewsc
Excellent 17 17 18
Good 24 29 14
Fair 6 2 1
Poor 0 0 2
aData are for a total of 47 QIOs.
bData are for a total of 48 QIOs.
cData are for a total of 35 QIOs.
SOURCE: IOM committee web-based data collection tool.
ment Task Leader may be located in either the Regional or Central Office.
In the IOM committee telephone interviews, 11 of 20 QIO chief executive
officers (CEOs) expressed problems with Government Task Leaders. Three
CEOs, including two with QIO support center (QIOSC) contracts, specifi-
cally mentioned, unprompted, that many Government Task Leaders lack
substantive expertise in their topic areas. Some of their comments were as
follows:
· "What QIOSCs need to do the best job are exceptional CMS Gov-
ernment Task Leaders. They blend a knowledge of breaking research with
pragmatism and good political instincts."
· "There should be better coordination among the Government Task
Leaders at CMS. They tend to get siloed in their specialties and do not
understand the scope of what QIOs are doing."
· "You can usually attribute the difference [in timeliness] to the rela-
tionship with the CMS Government Task Leader; if it is positive, you get
things approved in a timely manner."
Difficult relationships with Government Task Leaders were echoed in inter-
views with staff from five organizations representing seven QIOSCs. All of
them believed that the relationship often depended on the Government Task
Leader's experience in the topic area. One staff member stated that the
Government Task Leader used the QIOSC as an extension of his or her
personal staff. Two staff members indicated that the rate of turnover of
their Government Task Leaders was high and that their skills and experi-
ence with their assigned topic areas varied.
OCR for page 333
CMS OVERSIGHT 333
TABLE 13.6 Full-Time CMS Employees for the QIO Program
FTEa Percentage
Area of CMS Count of Total
Regional Offices Total count 42.45 32
Dallas 9.2
Boston 11.25
Seattle 11.0
Kansas City 11.0
Quality Improvement Total count 36.5 28
Group (Office of Division of Contract Operations and 14.5
Clinical Standards and Support
Quality) Division of Quality Improvement Policy 14.0
for Acute Care
Division of Quality Improvement Policy 4.5
for Chronic and Ambulatory Care
Front office staff 3.5
Information Systems Group 24.0 18
(Office of Clinical Standards
and Quality)
Quality Measurement and 20.5 16
Health Assessment Group
(Office of Clinical Standards
and Quality)
Office of Acquisition and
Grants Management 8.5 6
aFTE = full-time equivalent.
SOURCE: Personal communication, J. V. Kelly, September 8, 2005.
Full-Time Employees
As of September 2005, the full-time employee count for the QIO pro-
gram was 131.95 (personal communication, J. V. Kelly, September 8, 2005).
This includes all CMS employees who work on the core contract, special
studies, or developmental work. Most employees (62 percent) work in one
of the groups of the Office of Clinical Standards and Quality. The break-
down is presented in Table 13.6.
COMMUNICATIONS AND
INFORMATION TECHNOLOGY SERVICES
Communications
QIO Manual and Contract
Many conduits of communication exist within the QIO program (CMS,
2004b). A primary source of program information is the QIO manual,
OCR for page 334
334 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
which lays out basic program policy on the basis of legal and agency re-
quirements and which is unlikely to change during the course of a contract.
The QIO contract itself is another source of information for QIOs. The
contract includes a statement of work, a document that delineates detailed
work requirements, a list of deliverables, evaluation criteria, and a budget.
The scope of work (SOW) is a section of the statement of work that pro-
vides an overall nontechnical description of the required activities during
the contract cycle. According to the J-1 attachment of the QIO contract (the
glossary), the abbreviation "SOW" can be used to refer to either the scope
of work or statement of work but declares that the terms themselves are not
interchangeable (CMS, 2002).
Memos and Letters
CMS uses TOPS documents to inform the QIOs quickly about antici-
pated changes in policy, including draft statements (Jost, 1991; CMS,
2004b). Although TOPS documents deal with policy changes, Standard
Data Processing System (SDPS) memos inform QIOs about operational con-
cerns. Examples include one-time requests for information, emergency
alerts, and administrative announcements (CMS, 2004b). SDPS memos may
come from different sources, but all memos must be cleared by the Informa-
tion Systems Group of the Office of Clinical Standards and Quality.
CMS uses contractor clarification letters to inform QIOs of alterations
or additions to their contracts. The letters may also clarify requirements or
respond to specific questions. Two types of clarification letters are used.
The first type is an unofficial letter that explains an issue or question but
does not result in a contract modification (personal communication, J. V.
Kelly, May 31, 2005). The second type is one that is a precursor to a con-
tract modification, informs QIOs of forthcoming contract changes, and ul-
timately, results in a contract modification. No matter the source, all letters
must be cleared by the Contracting Officer in the Acquisition and Grants
Group. For day-to-day work and specific questions, CMS may use e-mail or
conference calls to communicate with the QIOs. These formal letters and
memos are all sent by e-mail to each QIO and are also posted in appropri-
ate sections of QIONet, CMS's internal intranet website (described later in
this chapter).
Figure 13.3 shows QIO satisfaction with the clarity and the timeliness
of TOPS memos. Overall, most QIOs believe that clarity was "good" but
that timeliness was "fair."
Regional Office Communications
The Regional Offices coordinate much of the communication between
CMS and the QIOs (CMS, 2004b). Informal interactions often occur daily
OCR for page 335
CMS OVERSIGHT 335
A B
Excellent , 1 Excellent, 1
Poor, 3 Poor, 4
Good, 11
Fair, 12
Fair, 36
Good, 36
FIGURE 13.3 Clarity (A) and timeliness (B) of TOPS memos reported by 52 QIOs.
The numbers in the figure represent the number of QIOs responding as indicated.
SOURCE: IOM committee web-based data collection tool.
via e-mail and telephone. Formal interactions occur at the 9- and 18-month
evaluations (discussed later in this chapter). Some Project Officers expressed
frustration that limited travel budgets do not permit more than two on-site
evaluation visits. Finally, the Regional Offices interact with each other as
well as with CMS Regional Offices that do not oversee the QIOs. The
Project Officers of the four Regional Offices that oversee QIOs participate
in a monthly community-of-practice call; this is a regularly scheduled tele-
conference that allows officers to exchange ideas and information. Interac-
tion with CMS Regional Offices not associated with the QIO program is
less formalized but still occurs, especially when national programs (like
Nursing Home Compare) are launched.
Medicare Quality Improvement Community
The Medicare Quality Improvement Community (MedQIC) (formerly
known as the Medicare Quality Improvement Clearinghouse) is a public
website available to anyone via the Internet at http://www.medqic.org
(CMS, 2004b). MedQIC currently features support for seven areas: struc-
tural and systems change, physicians' offices, hospitals, home health agen-
cies, nursing homes, underserved populations, and managed care organiza-
tions. These areas are subject to change with the evolution of the SOWs and
refinement of the website. The site serves as a resource for quality improve-
ment efforts and includes bibliographies, tool kits, flowcharts, and sugges-
OCR for page 350
350 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
must move ahead on its own. The QIOs in the first round were particularly
affected and had to start the new tasks with no QIOSC materials.
Award Fees
In the QIO contract for the 8th SOW, the Award Fee Plan involves a
combination of cost plus fixed fee and cost plus award fee mechanisms.
CMS also built in several types of incentives in addition to the base fee. The
base fee is 1 percent of contract costs, excluding pass-through costs (reim-
bursable expenses) and special studies (CMS, 2005a). The Full Pass Perfor-
mance Award Fee is an award of 1 percent of contract costs per applicable
subtask (excluding pass-through and special studies costs) for QIOs that
meet all full pass performance expectations. The Excellent Pass Performance
Award Fee is an additional award of 1 percent of contract costs per appli-
cable subtask (excluding pass-through and special studies costs) for QIOs
that meet the excellent pass criteria. Finally, a Group Award Fee of 2 per-
cent of contract costs per subtask is awarded to QIOs that meet the follow-
ing three criteria:
· the QIO receives a full pass on evaluation standards for that subtask,
· no more than five QIOs have failed to achieve at least a conditional
pass on that subtask, and
· the composite scores for all QIOs meet or exceed specific achieve-
ment standards delineated for each subtask in the J-2 attachment of the
QIO contract for the eighth.
The Group Award Fee is designed to encourage sharing and collaboration
among QIOs, for it is in the best interest of each QIO to ensure that all
QIOs pass so that all will receive the additional fee. The contract also speci-
fies that the QIOs will be paid a fixed fee for information systems, contrac-
tual requirements, and special studies costs.
In the 8th SOW, QIOs may also qualify for an interim award fee based
on performance as of January 2007 (CMS, 2005c). This includes up to
50 percent of the Full Pass Performance Award Fee and up to 50 percent of
the Group Award Fee. QIOs that do not qualify for Interim Award Fee
payments may receive full award amounts in November 2007. QIOs quali-
fying for an Interim Award Fee may receive the remainder of the fee at that
time as well. In the QIO contract for the 8th SOW, CMS presents detailed
information on the measures and calculations used to assess performance
for these payments.
OCR for page 351
CMS OVERSIGHT 351
OVERALL PROGRAM GUIDANCE
For a public program as diverse and multifaceted as the QIO program,
some of the most important functions at the federal level include strategic
planning, broad policy guidance and priority setting, coordination with
other programs of the U.S. Department of Health and Human Services
(DHHS), and evaluation of the whole program. In a federal program, even
one that does not require an annual appropriation, these guidance func-
tions take place in the context of the federal budget. That context inevitably
creates some uncertainties. Also, as the QIO program becomes more inte-
grated with other CMS activities, the independence of the program's plan-
ning and operations will likely be affected.
Strategic Planning
In the months preceding the start of the 8th SOW, CMS began an am-
bitious long-range planning process for the QIO program with the help of a
consultant and the Process Improvement QIOSC (Qualis Health, under its
contract as the QIO for Washington state). CMS was looking well beyond
the 8th SOW and considering the program over the next 10 to 12 years.
After considerable internal discussion, external stakeholder groups offered
advice on how transformational change could be achieved. The meetings of
the stakeholder groups, including representatives of QIOs, were organized
according to the main provider settings addressed by the QIO program:
home health agency providers, hospitals, nursing homes, and physicians'
practices. The discussion at the physicians' meeting was wide ranging and
touched on many of the issues raised by the 8th SOW, such as the role of
QIOs in promoting the use of health information technology in physicians'
practices. Discussions among the stakeholders and CMS indicated much
uncertainty about the roles of QIOs in the 8th SOW, as well as in the
future. CMS planned to prepare a report on the substance of the meetings
and to provide feedback to the participants.
Policy Direction
As described in the evolution of the QIO program in Chapter 2, there
have been significant changes in policy direction with each new contract,
including occasional additional changes within a 3-year contract period. In
the past, CMS released a version of the new QIO contract well in advance
of the request for proposal for the first contract cycle so that the QIOs had
time to plan their work and respond to the request for proposal. The tran-
sition from the 7th to the 8th SOW was not easy because negotiations within
DHHS and among DHHS, CMS, and the Office of Management and Bud-
OCR for page 352
352 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
get over the new QIO contract and funding took longer than it did in previ-
ous years. The contract for the 8th SOW, which had been expected to be
available in the summer of 2004 for implementation on August 1, 2005,
was not formally released until April 2005; and all of the final budget fig-
ures were not available until May 20, 2005. Subsequent altered versions of
the contract were released in June, September, and November 2005. All
QIOs bidding in the first round received guidance from CMS on most but
not all of the QIO budget before their responses were due to CMS (personal
communication, D. Adler, American Health Quality Association, May 16,
2005).
In the 8th SOW, the tasks of the QIO program reflect a major change
from measurement-based quality improvement to assisting providers with
achieving transformational change (Rollow, 2004). Early summaries of the
8th SOW, as well as the request for proposal, raised many questions. The
QIOs speculated about what "transformational change" really meant and
how it would be accomplished (CMS, 2004a; AHQA, 2005). CMS had to
post questions and answers on its website to clarify its intent for the QIO
contract bidders. The long-range strategic planning meetings mentioned
above, which were held after the release of the QIO contract for the 8th
SOW, defined the goal as soliciting advice from the stakeholders on how to
achieve transformational change in their care settings, how to measure it,
and how CMS and QIOs could support that change.
Priority setting is also a key need for the QIO program; beyond the goal
of "transformational change" and the six quality aims for health care estab-
lished by IOM (safety, timeliness, effectiveness, efficiency, equity, and
patient-centeredness), it is difficult to discern priorities within the 8th SOW.
Compared with the 7th SOW, the 8th SOW involves considerably more
tasks, more measures for evaluation, and more identified participant groups.
The evaluation formulas are complex, with many different subscores and
many different weights, making it impossible to determine where a QIO
should focus its time and resources (see Table A.6 in Appendix A). Because
failure on any one task could jeopardize noncompetitive renewal for the
next SOW, one might assume that all tasks are of equal priority.
The QIOs voiced concerns about CMS's priorities and focus in the 7th
SOW. During the IOM committee site visits, 6 of 11 QIOs described frus-
tration with CMS's inconsistent or changing priorities, including contract
changes in the middle of an SOW. One QIO specifically criticized the lack
of continuity between SOWs, describing difficulty with a "stop-start" ef-
fect. Several QIOs expressed a desire for fewer, more well-defined priority
areas so that the QIOs could focus their efforts on just a few priorities.
The participants of a focus group held by the IOM committee discussed
the direction that the QIO program is taking in the 8th SOW. Themes of
concern included the challenge of working with an increased number of
OCR for page 353
CMS OVERSIGHT 353
identified participants with limited resources, overly complex evaluation
formulas, and the lack of flexibility in the contract. The participants also
believed that increased competition might lead to decreased collaboration
among QIOs, that the contract length was too short to create culture
change, and that administrative reporting requirements should be de-
creased. Overall, the focus group participants believed that DHHS as a
whole needs to align its priorities to provide incentives for quality improve-
ment, such as through the implementation of regulatory requirements and
pay for performance.
Program Coordination
The QIO program is only one of several health care qualityrelated
efforts under way within CMS, which increases the need for coordination
within Medicare and CMS as a whole. Some of that coordination may take
place when other offices within CMS desire to use the QIO apportionment
to fund their research or other activities or to use the apportionment for
policy planning at broader levels.
Support Contracts and Special Studies
Chapter 7 described the various review and funding mechanisms for
the special studies and support contracts. At the beginning of the SOW, the
program indicates priorities for special studies, but unsolicited proposals
may be considered and funded later in the contract period for the SOW.
However, no apparent mechanism exists for coordinating projects and fund-
ing priorities among those projects. Also, it is unclear how CMS shares
information about ongoing studies with the QIO community or what it
does with the results of all studies. As one QIOSC representative stated in
an interview, "I have no idea what CMS does with special studies' results."
In the IOM committee telephone interviews, 8 of 20 QIO CEOs com-
mented on the pros and cons of pilot testing. All found pilot testing to be
favorable from the standpoint of having experience with the task at hand
before all QIOs approach that task. However, two CEOs cautioned that
sometimes pilot studies are not always the answer, as they can be too state
specific and may not have been translated for a wider audience. One CEO
commented further that the oversight of special projects is sometimes as-
signed to a middle manager at CMS with no expertise in the topic area of
the project.
Quality Coordination Team
Recently, CMS made efforts to coordinate the quality improvement
efforts across all programs of CMS. On September 14, 2004, CMS an-
OCR for page 354
354 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
nounced the creation of a new Quality Coordination Team to support and
act as staff to the redesigned Quality Council (CMS, 2004c). The adminis-
trator of CMS, Mark McClellan, chairs the Quality Council; and its mem-
bers include the director of each major CMS office. The Quality Council
strives to coordinate all CMS efforts related to quality as well as to align
those efforts with the quality improvement activities of other public and
private organizations (Jencks, 2004). The Quality Coordination Team is
led by Steve Jencks, former director of the Quality Improvement Group.
Almost all team members are staff from CMS.
In July 2005, CMS released the Quality Council's Quality Improve-
ment Roadmap to improving the quality of care (CMS, 2005b). This
roadmap included five major strategies: working through partnerships, pub-
lic reporting, paying for quality performance, promoting efficient systems
(such as electronic health systems), and increasing the availability and im-
proved use of innovative technologies. Major activities of the Quality Coor-
dination Team include direct support of the Quality Council, such as moni-
toring of work groups, facilitation of partnerships and collaboratives, and
participation in breakthrough projects. Topics chosen for focus in the work
groups include performance measures and pay for performance, health in-
formation technology, the Medicare Part D prescription drug benefit, and
CMS Regional OfficeCentral Office communications. Breakthrough proj-
ects include Fistula First, which seeks to improve vascular access in patients
with end-stage renal disease; a project that seeks to raise immunization
rates in specific settings; and the Institute for Healthcare Improvement's
100,000 Lives campaign (personal communication, S. Jencks, CMS, July
21, 2005). The Quality Coordination Team strives to facilitate partnerships
within CMS, with other federal agencies (such as the Centers for Disease
Control and Prevention and the Agency for Healthcare Research and Qual-
ity), and with nongovernmental organizations (such as the Institute for
Healthcare Improvement). The Quality Council and Quality Coordination
Team have no direct responsibility for the QIO program, but many CMS
staff on the team are directly responsible for the operation of the QIO
program.
Measures Selection and Coordination
Some QIO functions, such as the selection of quality measures, require
coordination with national stakeholder organizations as well with various
offices in CMS. CMS identified four criteria for the selection of measures:
· the measures must be scientifically and clinically sound,
· the measures must be reproducible,
· the measures should not add burden to the provider, and
OCR for page 355
CMS OVERSIGHT 355
· the measures should use existing data sources (CMS, 2004b).
CMS worked collaboratively with the CMS Survey and Certification pro-
gram as well as with the Joint Commission on Accreditation of Healthcare
Organizations, the American Medical Association, and the National Com-
mittee for Quality Assurance to align measure specifications to minimize
reporting burdens on providers. Ultimately, the groups seek endorsement of
selected measures by the National Quality Forum. In the 7th SOW, CMS
contracted with the Health Services Advisory Group (Arizona's QIO) to
maintain measures by the identification, standardization, and endorsement
of measures with updating and retirement of the measures as needed (CMS,
2004b). The QIO program also funded work through other parts of CMS
that contributed to the development and refinement of other measures, such
as the support contract for the Consumer Assessment of Healthcare Provid-
ers and Systems (CAHPS) family of surveys ($33.4 million during the 7th
SOW) (personal communication, C. Lazarus, March 17, 2005).
Roles and Relationships
Overall, the QIOs express great concern about the relationships be-
tween and among QIOs, Program Officers, Government Task Leaders,
QIOSCs, and CMS. The QIOs want better definitions of the roles of each of
these individuals or groups and streamlining of the management process.
During the IOM committee's site visits, the QIOs discussed many chal-
lenges related to CMS oversight of the QIO program. Seven of 11 QIOs
expressed concerns over the relationship between CMS and the QIOs and
communications problems. Specific examples included references to isola-
tion of groups within CMS and poor communication between CMS, the
Regional Offices, and the QIOs. The QIOs referred to "tension" in these
interactions. Three QIOs specifically mentioned difficulties with ambigu-
ous or poorly defined information, and three were frustrated with "micro-
management" of the program. Five QIOs wanted more flexibility in the
program in terms of either quality improvement topic areas or how goals
are achieved.
In interviews with five organizations representing seven QIOSCs, QIO
staff members also mentioned difficulties in their relationships with CMS's
Central Office and with their Government Task Leaders. As described
above, all QIOs believed that the relationship with their Government Task
Leaders was key, and they provided a range of responses as to whether or
not that relationship was positive. QIOSC staff believed that they had lim-
ited to no direct interaction with CMS's Central Office and wanted to see a
reduction of silos and increased communication among and between the
Government Task Leaders and CMS's Central Office.
OCR for page 356
356 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
Elaborating on this issue, during the telephone interviews, one QIO
CEO said, "CMS develops these contracts to develop expertise in an area,
but then CMS doesn't listen to or take the advice of experts, so we end up
with a program directed by Government Task Leaders rather than experts.
This structure could use improvement." To get better service, this CEO
stated, "There should be direct knowledge transfer [to QIOs] rather than
having to get approval by a Government Task Leader at every step." The
CEO asserted that "the Government Task Leaders don't seem to have any
great urgency in approving materials, so lots of time can transpire. Govern-
ment Task Leader delay cuts into the time available for technical assistance
to QIOs." Another CEO reported that "there is often lack of clear direction
from CMS on desired outcomes or that expectations change during the
project. Sometimes change is inevitable as information is gathered, but that
can substantially change the QIOSC resources. Reasons for change are more
palatable if they are clinical rather than political."
In these telephone interviews, the QIO CEOs also expressed concern
about the interaction between the QIOSCs and CMS. CEOs pointed out
that the QIOSCs have dual audiences, both the QIOs and CMS, and thus
have a sense of being caught in the middle. Four respondents whose organi-
zations held QIOSC contracts mentioned having difficulty in being able to
respond to QIOs as a direct consequence of CMS delay. One said, "QIOSCs
are caught between a rock and a hard place, with CMS being the rock that
they have to be responsive to over the QIOs. What the QIOs want goes
nowhere until CMS wants it to."
Overall Program Evaluation
Compared with the considerable effort that CMS put into designing
complex formulas to evaluate the contract performance of each QIO (see
Chapter 10), creating the databases necessary to evaluate their performance,
and monitoring the progress of each QIO, it appears to have spent little
time on evaluating other aspects of the program and the program as a whole.
Although priorities for special studies are set by the Science Council of the
Office of Clinical Standards and Quality, as described in Chapter 7, CMS
has not developed a system for tracking all the various special study and
support contracts, considering the balance of topics and spending, and de-
termining how they might serve program priorities. Also, no system exists
for broadly sharing the knowledge acquired through the studies or even
letting all the QIOs and other Project Officers and Government Task Lead-
ers know which QIO is working on a particular special study topic.
The Quality Improvement Group was unable to provide the IOM com-
mittee with information on the various contracts at a level of detail suffi-
cient for the committee to know what the contracts are supposed to accom-
OCR for page 357
CMS OVERSIGHT 357
plish and whether accomplishments have been made, although individual
Project Officers are required to assess their own projects. This lack of infor-
mation prevents an assessment of the overall value and impact of the spend-
ing on special studies and support contracts. In the 7th SOW, nearly 31 per-
cent ($355 million) of the total program's apportionment ($1,154.3 million)
was spent on special studies and support contracts (personal communica-
tion, C. Lazarus, March 17, 2005).
CMS does not have a mechanism or formula with which it can evaluate
individual QIOs overall. Although the program can determine whether a
specific QIO has achieved a passing score on its contract performance, it
cannot distinguish outstanding QIOs from mediocre QIOs in a holistic
sense. Although the QIOs vary widely on many organizational criteria, it is
unclear which, if any, of those factors contribute to better performance, as
it was not feasible to identify the better-performing QIOs. For example, at
the end of the 7th SOW, one of the three QIO contracts for which re-
competition was conducted in the first round was held by an organization
that had been awarded one of the highest number of special study con-
tracts. Also, as mentioned in Chapter 10, the committee's attempts to group
QIOs according to their overall performance on the quality improvement
subtasks were unsuccessful. The web-based data collection tool attempted
to gather opinions about other QIOs by the QIO community itself, but the
results were inconclusive.
More importantly, neither CMS nor independent researchers have per-
formed a conclusive evaluation of the impacts of the 53 QIOs on quality
improvement nationally. Also, CMS has not performed a programwide
evaluation to examine in detail the synergy, or lack thereof, between the
spending on special studies, QIOSCs, and support contracts and the spend-
ing on the core contracts.
During the IOM committee's site visits, 4 of 11 QIOs independently
related frustration with the contract evaluation process. They believed that
the goals were too stringent and that too much emphasis was placed on
short-term quantitative results. In the web-based data collection tool, 52
QIOs rated the clarity and timeliness of the evaluation process. Overall, the
process did not receive high marks, with only one QIO giving a score of
"excellent" on one of the three dimensions indicated in Table 13.8. More
than half of the QIOs answered "fair" or "poor" for each of the three
dimensions.
In contrast to the QIO evaluations of the 7th SOW, QIOSC evaluations
were informal and were primarily based on the completion of a set of
deliverables, according to interviews with five organizations representing
seven QIOSCs. All believed that their "success" was very subjective and
based on the personal satisfaction of their Government Task Leaders.
OCR for page 358
358 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM
TABLE 13.8 QIO Ratings of Evaluation Process
Overall Timeliness
Clarity of Clarity of of Information
Quantitative Portion Qualitative Portion About Evaluation
Rating of Evaluation of Evaluation (methodology, process, etc.)
Excellent 1 0 0
Good 18 21 10
Fair 19 19 19
Poor 14 12 23
NOTE: The data in the table represent the number of QIOs responding as indicated. Data
are for a total of 52 QIOs.
SOURCE: IOM committee web-based data collection tool.
SUMMARY
This chapter has discussed CMS's oversight of the QIO program. The
following are some of the main themes of this chapter, which are reflected
in the findings and conclusions presented in Chapter 2:
· Multiple offices and divisions within CMS have responsibility for
the QIO program. QIOs expressed frustration with the lack of coordina-
tion and communication between and among personnel and with the time-
liness of the information provided to them. They criticized the lack of coor-
dination by CMS, which leads to competing agendas for different managers
within the QIO program.
· One of the greatest concerns for QIOs was the time lag to the receipt
of performance data because it affects their quality interventions as well as
their contract performance assessments.
· QIOs oppose routine recompetition for the core contract because
of the loss of momentum that it causes, the decreased incentive that
QIOs have to share knowledge, and the chance that they might lose their
contract.
· Overall, QIOs believe that the 3-year contract period is too short to
achieve measurable change and is complicated by the concurrent lag in the
time to receipt of performance data. They also believe that the timeline for
QIOSC contracts should begin earlier so that the QIOSCs may help the
QIOs immediately upon the start of a new SOW.
· Although CMS is developing a strategic plan for the QIO program
12 years into the future, the program still lacks distinct, focused priorities.
Neither the core contracts nor the associated evaluation schemes prioritize
the QIO activities.
OCR for page 359
CMS OVERSIGHT 359
· Evaluations of the QIO core contract are based on overly complex
formulas. They hold the QIO accountable for provider improvements on
specified measures for short-term quantitative results. In contrast, QIOSC
evaluations are mainly subjective and are based primarily on the satisfac-
tion of the Government Task Leader and completion of a set of deliverables.
· CMS lacks any formal means of evaluation of the whole QIO pro-
gram, its success on improving quality, or the distinction of the perfor-
mance of one QIO over another.
REFERENCES
AHQA (American Health Quality Association). 2005. Proceedings of the AHQA Annual Meet-
ing and Technical Conference. San Francisco, CA: American Health Quality Association.
Beck C, Richard H, Tu J, Pilote L. 2005. Administrative data feedback for effective cardiac
treatment: AFFECT, A Cluster Randomized Trial. Journal of the American Medical As-
sociation 294(3):309317.
Bradley EH, Carlson MDA, Gallo WT, Scinto J, Campbell MK, Krumholz HM. 2005. From
adversary to partner: have quality improvement organizations made the transition? Health
Services Research 40(2):459476.
CMS (Centers for Medicare and Medicaid Services). 2002. 7th Statement of Work (SOW).
[Online]. Available: http://www.cms.hhs.gov/qio [accessed April 9, 2005].
CMS. 2003. HHS Privacy Impact Assessment (PIA). November 18. [Online]. Available:
www.cms.hhs.gov/privacyact/hcqis.pdf [accessed April 21, 2005].
CMS. 2004a. Proceedings of the QualityNet 2004 Conference. Washington, DC: Centers for
Medicare and Medicaid Services.
CMS. 2004b. The Quality Improvement Organization Program: CMS Briefing for IOM Staff.
[Online]. Available: http://www.medqic.org/dcs/ContentServer?cid=1105558772835&
pagename=Medqic%2FMQGeneralPage%2FGeneralPageTemplate&c=MQGeneralPage
[accessed December 26, 2005].
CMS. 2004c. Notice from CMS Concerning Positions of Steve Jencks and Bill Rollow. Unpub-
lished. Baltimore, MD: Centers for Medicare and Medicaid Services.
CMS. 2004d. Quality Improvement Organization Manual. September 16. [Online]. Available:
http://www.cms.hhs.gov/manuals/110_qio/qio110index.asp [accessed May 11, 2005].
CMS. 2005a. 8th Statement of Work (SOW). [Online] Available: http://www.cms.hhs.gov/qio
[accessed April 9, 2005].
CMS. 2005b. Executive Summary. In: Quality Improvement Roadmap. [Online]. Available:
http://www.cms.hhs.gov/quality/quality%20roadmap.pdf [accessed September 30, 2005].
CMS. 2005c. 8th Statement of Work (SOW), Version #080105-1. [Online]. Available: http://
www.cms.hhs.gov/qio [accessed November 4, 2005].
Hughes E. 2004. 8th SOW Contract Issues. Presentation at QualityNet 2004 Conference.
Washington, DC: Centers for Medicare and Medicaid Services.
Jencks SF. 2004. The Health Care Quality Improvement Partnership. Powerpoint Presenta-
tion to Quality Improvement Organization Subcommittee, October 4, Baltimore, MD.
Jost TS. 1991. Policing cost containment: The Medicare Peer Review Organization Program.
University of Puget Sound Law Review 3:483526.
Rollow WC. 2004. Evaluating the HCQIP Program. Powerpoint Presentation to Quality Im-
provement Organization Subcommittee, October 4, Baltimore, MD.
OCR for page 360
Representative terms from entire chapter:
main perfor
