Click for next page ( 326


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 325
13 CMS Oversight CHAPTER SUMMARY This chapter examines how the Centers for Medicare and Medicaid Services (CMS) oversees and manages the Quality Improvement Organization (QIO) program as a whole. First, the integration of the program into CMS's organizational structure is discussed, in- cluding the use of personnel who help with oversight. Next, com- munications, information technology, and data services are dis- cussed both in the context of how they are used in the operations of the program and how they are used as a resource for manage- ment. Then, contract issues are presented, including how contracts are competed, awarded, implemented, and monitored. Finally, there is an examination of how CMS provides overall guidance to the Quality Improvement Organization program through strategic planning, policy decision making, coordination, and overall pro- gram evaluation. ORGANIZATIONAL STRUCTURE OF QIO PROGRAM IN CMS Oversight of the Quality Improvement Organization (QIO) program involves coordination of the efforts of multiple personnel in several offices within the Centers for Medicare and Medicaid Services (CMS), each of which has distinct roles. The administrative office of CMS, located in Balti- more, Maryland, is commonly referred to as the "Central Office." Two offices within CMS's Central Office share the responsibility for manage- ment of the QIO program: the Office of Clinical Standards and Quality, the "Program Office," and the Office of Acquisition and Grants Management, the "Contracts Office." Other groups have indirect roles in the manage- ment of the QIO program. The QIO and End-Stage Renal Disease Steering Committee manages the daily operations of the QIO program. The mem- bership on the QIO and End-Stage Renal Disease Steering Committee com- prises the Associate Regional Administrator for each of the four Regional 325

OCR for page 325
326 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Offices affiliated with the QIO program and both the director and the deputy director of each of three groups within the Office of Clinical Stan- dards and Quality: the Quality Improvement Group, the Information Sys- tems Group, and the Quality Measurement and Health Assessment Group. The committee, currently chaired by the director of the Quality Improve- ment Group, meets weekly and primarily discusses operational issues (per- sonal communication, J. V. Kelly, June 28, 2005). Program Office Overall responsibility for the QIO program lies in CMS's Office of Clinical Standards and Quality, with direct oversight provided by the Qual- ity Improvement Group (Jost, 1991; CMS, 2004c) and with support pro- vided by other groups within that office. The Program Office monitors the QIO program, coordinates with the Office of Internal Customer Support on financial matters, and creates and interprets policy related to the QIO program's operations. The office is divided into six groups, each of which may have one or more of the following divisions: Quality Improvement Group, Quality Measurement and Health Assessment Group, Information Systems Group, Quality Coordination Team, Coverage and Analysis Group, and Clinical Standards Group. In the Institute of Medicine (IOM) committee's web-based data collection tool, 52 QIOs rated the Program Office on several functions. Overall, the office received higher scores on "clarity" than on "timeliness" (Table 13.1). Concerns over clarity and timeliness also arose during the IOM com- mittee and staff site visits. Four QIOs mentioned that the information that they receive is often ambiguous, and eight related frustration with the time- liness of access to information or data related to their tasks (referred to here as data lags). Data lags, however, may also be attributable to the measure- ment process, based on claims (this is discussed more later in this chapter). Contracts Office Many groups contribute to the development of a QIO contract, includ- ing the Office of Clinical Standards and Quality, the Office of Acquisition and Grants Management, and Regional Office Divisions of Quality Im- provement (CMS, 2004b). However, responsibility for the QIO contract ultimately rests with the Acquisition and Grants Group of the Office of

OCR for page 325
CMS OVERSIGHT 327 TABLE 13.1 QIO Ratings of CMS Program Office Clarity of Overall Overall Clarity of Timeliness of Program Support of Information Information on Ratings Direction QIO Work on Core Tasks Core Tasks Excellent 2 3 1 0 Good 33 33 26 8 Fair 15 9 22 30 Poor 2 7 3 14 NOTE: The data in the table represent the number of QIOs responding as indicated. Data are for a total of 52 QIOs. SOURCE: IOM committee web-based data collection tool. Acquisitions and Grants Management. The Contracting Officer, a repre- sentative of the Acquisition and Grants Group, is the only person with the authority to release the contract or make modifications to the contract. The Contracting Officer oversees all contracts for the QIO program, and several contract specialists are each assigned to specific QIOs. As of June 2005, nine contract specialists were each assigned to work directly with between five and seven QIOs (personal communication, J. V. Kelly, June 30, 2005). The QIOs expressed frustration with their interactions with the Con- tracts Office. During the site visits, two QIOs raised issues about conflicting messages between the Program and Contracts Offices. Additionally, at CMS's annual technical conference for the QIO program (QualityNet 2004), many QIO staff related difficulties with being asked to perform du- ties not specified within their contracts. They were asked to perform these duties by different sources, such as their Program Officer or Government Task Leader, or through a Transmittal of Policy System (TOPS) document (all of these are described later in this chapter). Although CMS presenters clarified that the Contracting Officer has the final say on required duties, the QIOs expressed frustration with conflicting messages from different individuals and groups at CMS (Hughes, 2004). The QIOs rated the contracts office on many functions. Thirty-five of 52 QIOs stated that they had interaction with the Contracts Office only on an as-needed basis. The majority of QIOs rated the Contracts Office as "good" or "fair" on all questions (Table 13.2). Regional Offices CMS has 10 Regional Offices around the country. In four of these Re- gional Offices (Boston, Dallas, Kansas City, and Seattle), CMS established

OCR for page 325
328 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM TABLE 13.2 QIO Ratings of CMS Contracts Office Overall Overall Timeliness Expertise/ Clarity of Timeliness of of Contract Understanding Ratings Communications Communications Modifications of QIO Tasks Excellent 6 6 3 4 Good 26 25 27 19 Fair 14 17 15 19 Poor 6 4 7 10 NOTE: The data in the table represent the number of QIOs responding as indicated. Data are for a total of 52 QIOs. SOURCE: IOM committee web-based data collection tool. Divisions of Quality Improvement that act as liaisons between the QIOs and CMS's Central Office (Jost, 1991; CMS, 2004b). The remaining six CMS Regional Offices do not have any direct responsibility for the QIO program. The four Regional Offices with Divisions of Quality Improve- ment (referred to in the QIO program as "Regional Offices") assist QIOs with technical issues on a daily basis by interpreting CMS policy, monitor- ing finances, and providing feedback. The staff of the Divisions of Quality Improvement include an Associate Regional Administrator, Project Officers, and Scientific Officers. The Asso- ciate Regional Administrator oversees daily operations, including develop- ment and the implementation of goals, participation in consortium meet- ings, maintenance of stakeholder relationships, and management of funds (CMS, 2004b). Before the 7th SOW, Divisions of Quality Improvement existed in all 10 CMS Regional Offices and were generally staffed only by Project Officers. As the program focus shifted toward quality improvement, oversight was condensed into the four Regional Offices mentioned above, as new skills were needed to parallel the skills needed at the QIO level. New staff included epidemiologists, clinicians, biostatisticians, data managers, and communications specialists (CMS, 2004b). Today, staffing at each Re- gional Office varies in terms of both the numbers of personnel and the skill sets of those personnel (CMS, 2004b). CMS also divided the country into four consortiums that correlated with the four Regional Offices with Divisions of Quality Improvement. These consortiums (Northeast, Midwest, Southern, and Western) include the one Regional Office's with QIO oversight in that area and any other Regional Offices in that area that are not directly involved in the QIO pro- gram. The consortiums act to improve communications and share resources among the 10 Regional Offices and enhance consistency in the QIO pro- gram as a whole (CMS, 2004b).

OCR for page 325
CMS OVERSIGHT 329 Project Officers Project Officers monitor technical aspects of the QIO core contract (CMS, 2004b). All Project Officers participate in a week-long basic training session, with some officers completing optional advanced Project Officer training or performance-based contracting training. Each QIO is assigned one Project Officer, but a single Project Officer works with multiple QIOs. The QIOs reported that they have frequent contacts with their Project Of- ficers: half (26 of 52) reported weekly contact, and 92 percent (48 of 52) reported at least monthly contact (Figure 13.1). The Project Officer provides direct technical assistance to each QIO, serves as the advocate for the QIO within CMS, and is an expert resource for the QIOs in terms of contract content and CMS policy. The Project Officers manage QIO contracts by monitoring the progress of the QIOs, acting as a direct liaison to the Contracting Officer at CMS, and participat- ing in strategic planning. Monitoring activities include scheduled calls with individual QIOs and review of the data on the Dashboard section of CMS's intranet site (see below). Official monitoring visits are discussed in greater detail later in this chapter. The Project Officers also have communications and coordination responsibilities at both the local and the national levels. Table 13.3 shows the number of full-time Project Officers at each Regional Other, 1 As needed, 3 Monthly, 14 Weekly, 26 Semi-monthly, 8 FIGURE 13.1 Frequency of Project Officer contact with QIOs reported by 52 QIOs. The numbers in the figure represent the number of QIOs responding as indicated. SOURCE: IOM committee web-based data collection tool.

OCR for page 325
330 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM TABLE 13.3 Numbers of Project Officers and Contracts for Each CMS Regional Office Regional Number of Number of Average Number of Office Project Officers QIO Contracts Contracts per Project Officer Boston 5 16 3.2 Dallas 4.6 11 2.4 Kansas City 4 13 3.25 Seattle 4 13 3.25 SOURCE: CMS (2004b). TABLE 13.4 QIO Ratings of Project Officers Expertise/ Expertise/ Clarity of Timeliness Understanding Understanding Rating Responsesa of Responsesa of Review Tasks of HCQIP Tasks Excellent 34 36 25 21 Good 13 13 19 24 Fair 4 3 7 6 Poor 1 0 1 1 NOTE: The data in the table represent the number of QIOs responding as indicated. Data are for a total of 52 QIOs. HCQIP = Health Care Quality Improvement Program. aResponses to questions raised or issues posed by the QIO. SOURCE: IOM committee web-based data collection tool. Office, as well as the total number of QIO contracts monitored in that region as of June 2004 (CMS, 2004b). The QIOs rated the Project Officers on various functions. Overall, the Project Officers received high ratings in all areas, with the majority of QIOs rating their Project Officers as "excellent" or "good" in each area (Table 13.4). Scientific Officers The Scientific Officers support the Project Officers by providing scien- tific and clinical expertise (CMS, 2004b). Scientific Officers are not as- signed to specific QIOs but, instead, assist all QIOs in the region covered by the Regional Office with specific technical needs. They also assist the

OCR for page 325
CMS OVERSIGHT 331 Semi-Weekly, 6 Only as needed, 17 Quarterly, 3 Monthly, 22 FIGURE 13.2 Frequency of Scientific Officer contact with QIOs reported by 48 QIOs. The numbers in the figure represent the number of QIOs responding as indicated. SOURCE: IOM committee web-based data collection tool. QIOs in other regions, if they are requested to do so. As of June 2004, the Boston Regional Office had five Scientific Officers on staff, and the three other Regional Offices each had four Scientific Officers (CMS, 2004b). The QIOs reported extremely variable interactions with the Scientific Of- ficers (Figure 13.2). Scientific Officers evaluate measurement methodologies and surveys, analyze QIO data, review manuscripts, provide clinical expertise, and man- age special studies (CMS, 2004b). Scientific Officers possess specific skills in areas such as statistics, epidemiology, clinical science (Medical Officer), and data management. Scientific Officers may complete any of the training sessions described for Project Officers, but they are not required to do so. Scientific Officers also participate in official monitoring visits, described later in this chapter. In addition to their basic duties, Scientific Officers often serve as Government Task Leaders (see below). Table 13.5 shows the QIO ratings of Scientific Officers on a variety of functions. In general, QIOs rated Scientific Officers highly in all areas, with most QIOs providing "excellent" or "good" ratings for their Scien- tific Officers. Government Task Leaders Each task of the QIO contract and each special study are assigned a single Government Task Leader to provide direct oversight. The Govern-

OCR for page 325
332 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM TABLE 13.5 QIO Ratings of Scientific Officers Clarity of Timeliness Timeliness of Rating Responsesa of Responsesb Manuscript Reviewsc Excellent 17 17 18 Good 24 29 14 Fair 6 2 1 Poor 0 0 2 aData are for a total of 47 QIOs. bData are for a total of 48 QIOs. cData are for a total of 35 QIOs. SOURCE: IOM committee web-based data collection tool. ment Task Leader may be located in either the Regional or Central Office. In the IOM committee telephone interviews, 11 of 20 QIO chief executive officers (CEOs) expressed problems with Government Task Leaders. Three CEOs, including two with QIO support center (QIOSC) contracts, specifi- cally mentioned, unprompted, that many Government Task Leaders lack substantive expertise in their topic areas. Some of their comments were as follows: "What QIOSCs need to do the best job are exceptional CMS Gov- ernment Task Leaders. They blend a knowledge of breaking research with pragmatism and good political instincts." "There should be better coordination among the Government Task Leaders at CMS. They tend to get siloed in their specialties and do not understand the scope of what QIOs are doing." "You can usually attribute the difference [in timeliness] to the rela- tionship with the CMS Government Task Leader; if it is positive, you get things approved in a timely manner." Difficult relationships with Government Task Leaders were echoed in inter- views with staff from five organizations representing seven QIOSCs. All of them believed that the relationship often depended on the Government Task Leader's experience in the topic area. One staff member stated that the Government Task Leader used the QIOSC as an extension of his or her personal staff. Two staff members indicated that the rate of turnover of their Government Task Leaders was high and that their skills and experi- ence with their assigned topic areas varied.

OCR for page 325
CMS OVERSIGHT 333 TABLE 13.6 Full-Time CMS Employees for the QIO Program FTEa Percentage Area of CMS Count of Total Regional Offices Total count 42.45 32 Dallas 9.2 Boston 11.25 Seattle 11.0 Kansas City 11.0 Quality Improvement Total count 36.5 28 Group (Office of Division of Contract Operations and 14.5 Clinical Standards and Support Quality) Division of Quality Improvement Policy 14.0 for Acute Care Division of Quality Improvement Policy 4.5 for Chronic and Ambulatory Care Front office staff 3.5 Information Systems Group 24.0 18 (Office of Clinical Standards and Quality) Quality Measurement and 20.5 16 Health Assessment Group (Office of Clinical Standards and Quality) Office of Acquisition and Grants Management 8.5 6 aFTE = full-time equivalent. SOURCE: Personal communication, J. V. Kelly, September 8, 2005. Full-Time Employees As of September 2005, the full-time employee count for the QIO pro- gram was 131.95 (personal communication, J. V. Kelly, September 8, 2005). This includes all CMS employees who work on the core contract, special studies, or developmental work. Most employees (62 percent) work in one of the groups of the Office of Clinical Standards and Quality. The break- down is presented in Table 13.6. COMMUNICATIONS AND INFORMATION TECHNOLOGY SERVICES Communications QIO Manual and Contract Many conduits of communication exist within the QIO program (CMS, 2004b). A primary source of program information is the QIO manual,

OCR for page 325
334 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM which lays out basic program policy on the basis of legal and agency re- quirements and which is unlikely to change during the course of a contract. The QIO contract itself is another source of information for QIOs. The contract includes a statement of work, a document that delineates detailed work requirements, a list of deliverables, evaluation criteria, and a budget. The scope of work (SOW) is a section of the statement of work that pro- vides an overall nontechnical description of the required activities during the contract cycle. According to the J-1 attachment of the QIO contract (the glossary), the abbreviation "SOW" can be used to refer to either the scope of work or statement of work but declares that the terms themselves are not interchangeable (CMS, 2002). Memos and Letters CMS uses TOPS documents to inform the QIOs quickly about antici- pated changes in policy, including draft statements (Jost, 1991; CMS, 2004b). Although TOPS documents deal with policy changes, Standard Data Processing System (SDPS) memos inform QIOs about operational con- cerns. Examples include one-time requests for information, emergency alerts, and administrative announcements (CMS, 2004b). SDPS memos may come from different sources, but all memos must be cleared by the Informa- tion Systems Group of the Office of Clinical Standards and Quality. CMS uses contractor clarification letters to inform QIOs of alterations or additions to their contracts. The letters may also clarify requirements or respond to specific questions. Two types of clarification letters are used. The first type is an unofficial letter that explains an issue or question but does not result in a contract modification (personal communication, J. V. Kelly, May 31, 2005). The second type is one that is a precursor to a con- tract modification, informs QIOs of forthcoming contract changes, and ul- timately, results in a contract modification. No matter the source, all letters must be cleared by the Contracting Officer in the Acquisition and Grants Group. For day-to-day work and specific questions, CMS may use e-mail or conference calls to communicate with the QIOs. These formal letters and memos are all sent by e-mail to each QIO and are also posted in appropri- ate sections of QIONet, CMS's internal intranet website (described later in this chapter). Figure 13.3 shows QIO satisfaction with the clarity and the timeliness of TOPS memos. Overall, most QIOs believe that clarity was "good" but that timeliness was "fair." Regional Office Communications The Regional Offices coordinate much of the communication between CMS and the QIOs (CMS, 2004b). Informal interactions often occur daily

OCR for page 325
CMS OVERSIGHT 335 A B Excellent , 1 Excellent, 1 Poor, 3 Poor, 4 Good, 11 Fair, 12 Fair, 36 Good, 36 FIGURE 13.3 Clarity (A) and timeliness (B) of TOPS memos reported by 52 QIOs. The numbers in the figure represent the number of QIOs responding as indicated. SOURCE: IOM committee web-based data collection tool. via e-mail and telephone. Formal interactions occur at the 9- and 18-month evaluations (discussed later in this chapter). Some Project Officers expressed frustration that limited travel budgets do not permit more than two on-site evaluation visits. Finally, the Regional Offices interact with each other as well as with CMS Regional Offices that do not oversee the QIOs. The Project Officers of the four Regional Offices that oversee QIOs participate in a monthly community-of-practice call; this is a regularly scheduled tele- conference that allows officers to exchange ideas and information. Interac- tion with CMS Regional Offices not associated with the QIO program is less formalized but still occurs, especially when national programs (like Nursing Home Compare) are launched. Medicare Quality Improvement Community The Medicare Quality Improvement Community (MedQIC) (formerly known as the Medicare Quality Improvement Clearinghouse) is a public website available to anyone via the Internet at http://www.medqic.org (CMS, 2004b). MedQIC currently features support for seven areas: struc- tural and systems change, physicians' offices, hospitals, home health agen- cies, nursing homes, underserved populations, and managed care organiza- tions. These areas are subject to change with the evolution of the SOWs and refinement of the website. The site serves as a resource for quality improve- ment efforts and includes bibliographies, tool kits, flowcharts, and sugges-

OCR for page 325
350 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM must move ahead on its own. The QIOs in the first round were particularly affected and had to start the new tasks with no QIOSC materials. Award Fees In the QIO contract for the 8th SOW, the Award Fee Plan involves a combination of cost plus fixed fee and cost plus award fee mechanisms. CMS also built in several types of incentives in addition to the base fee. The base fee is 1 percent of contract costs, excluding pass-through costs (reim- bursable expenses) and special studies (CMS, 2005a). The Full Pass Perfor- mance Award Fee is an award of 1 percent of contract costs per applicable subtask (excluding pass-through and special studies costs) for QIOs that meet all full pass performance expectations. The Excellent Pass Performance Award Fee is an additional award of 1 percent of contract costs per appli- cable subtask (excluding pass-through and special studies costs) for QIOs that meet the excellent pass criteria. Finally, a Group Award Fee of 2 per- cent of contract costs per subtask is awarded to QIOs that meet the follow- ing three criteria: the QIO receives a full pass on evaluation standards for that subtask, no more than five QIOs have failed to achieve at least a conditional pass on that subtask, and the composite scores for all QIOs meet or exceed specific achieve- ment standards delineated for each subtask in the J-2 attachment of the QIO contract for the eighth. The Group Award Fee is designed to encourage sharing and collaboration among QIOs, for it is in the best interest of each QIO to ensure that all QIOs pass so that all will receive the additional fee. The contract also speci- fies that the QIOs will be paid a fixed fee for information systems, contrac- tual requirements, and special studies costs. In the 8th SOW, QIOs may also qualify for an interim award fee based on performance as of January 2007 (CMS, 2005c). This includes up to 50 percent of the Full Pass Performance Award Fee and up to 50 percent of the Group Award Fee. QIOs that do not qualify for Interim Award Fee payments may receive full award amounts in November 2007. QIOs quali- fying for an Interim Award Fee may receive the remainder of the fee at that time as well. In the QIO contract for the 8th SOW, CMS presents detailed information on the measures and calculations used to assess performance for these payments.

OCR for page 325
CMS OVERSIGHT 351 OVERALL PROGRAM GUIDANCE For a public program as diverse and multifaceted as the QIO program, some of the most important functions at the federal level include strategic planning, broad policy guidance and priority setting, coordination with other programs of the U.S. Department of Health and Human Services (DHHS), and evaluation of the whole program. In a federal program, even one that does not require an annual appropriation, these guidance func- tions take place in the context of the federal budget. That context inevitably creates some uncertainties. Also, as the QIO program becomes more inte- grated with other CMS activities, the independence of the program's plan- ning and operations will likely be affected. Strategic Planning In the months preceding the start of the 8th SOW, CMS began an am- bitious long-range planning process for the QIO program with the help of a consultant and the Process Improvement QIOSC (Qualis Health, under its contract as the QIO for Washington state). CMS was looking well beyond the 8th SOW and considering the program over the next 10 to 12 years. After considerable internal discussion, external stakeholder groups offered advice on how transformational change could be achieved. The meetings of the stakeholder groups, including representatives of QIOs, were organized according to the main provider settings addressed by the QIO program: home health agency providers, hospitals, nursing homes, and physicians' practices. The discussion at the physicians' meeting was wide ranging and touched on many of the issues raised by the 8th SOW, such as the role of QIOs in promoting the use of health information technology in physicians' practices. Discussions among the stakeholders and CMS indicated much uncertainty about the roles of QIOs in the 8th SOW, as well as in the future. CMS planned to prepare a report on the substance of the meetings and to provide feedback to the participants. Policy Direction As described in the evolution of the QIO program in Chapter 2, there have been significant changes in policy direction with each new contract, including occasional additional changes within a 3-year contract period. In the past, CMS released a version of the new QIO contract well in advance of the request for proposal for the first contract cycle so that the QIOs had time to plan their work and respond to the request for proposal. The tran- sition from the 7th to the 8th SOW was not easy because negotiations within DHHS and among DHHS, CMS, and the Office of Management and Bud-

OCR for page 325
352 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM get over the new QIO contract and funding took longer than it did in previ- ous years. The contract for the 8th SOW, which had been expected to be available in the summer of 2004 for implementation on August 1, 2005, was not formally released until April 2005; and all of the final budget fig- ures were not available until May 20, 2005. Subsequent altered versions of the contract were released in June, September, and November 2005. All QIOs bidding in the first round received guidance from CMS on most but not all of the QIO budget before their responses were due to CMS (personal communication, D. Adler, American Health Quality Association, May 16, 2005). In the 8th SOW, the tasks of the QIO program reflect a major change from measurement-based quality improvement to assisting providers with achieving transformational change (Rollow, 2004). Early summaries of the 8th SOW, as well as the request for proposal, raised many questions. The QIOs speculated about what "transformational change" really meant and how it would be accomplished (CMS, 2004a; AHQA, 2005). CMS had to post questions and answers on its website to clarify its intent for the QIO contract bidders. The long-range strategic planning meetings mentioned above, which were held after the release of the QIO contract for the 8th SOW, defined the goal as soliciting advice from the stakeholders on how to achieve transformational change in their care settings, how to measure it, and how CMS and QIOs could support that change. Priority setting is also a key need for the QIO program; beyond the goal of "transformational change" and the six quality aims for health care estab- lished by IOM (safety, timeliness, effectiveness, efficiency, equity, and patient-centeredness), it is difficult to discern priorities within the 8th SOW. Compared with the 7th SOW, the 8th SOW involves considerably more tasks, more measures for evaluation, and more identified participant groups. The evaluation formulas are complex, with many different subscores and many different weights, making it impossible to determine where a QIO should focus its time and resources (see Table A.6 in Appendix A). Because failure on any one task could jeopardize noncompetitive renewal for the next SOW, one might assume that all tasks are of equal priority. The QIOs voiced concerns about CMS's priorities and focus in the 7th SOW. During the IOM committee site visits, 6 of 11 QIOs described frus- tration with CMS's inconsistent or changing priorities, including contract changes in the middle of an SOW. One QIO specifically criticized the lack of continuity between SOWs, describing difficulty with a "stop-start" ef- fect. Several QIOs expressed a desire for fewer, more well-defined priority areas so that the QIOs could focus their efforts on just a few priorities. The participants of a focus group held by the IOM committee discussed the direction that the QIO program is taking in the 8th SOW. Themes of concern included the challenge of working with an increased number of

OCR for page 325
CMS OVERSIGHT 353 identified participants with limited resources, overly complex evaluation formulas, and the lack of flexibility in the contract. The participants also believed that increased competition might lead to decreased collaboration among QIOs, that the contract length was too short to create culture change, and that administrative reporting requirements should be de- creased. Overall, the focus group participants believed that DHHS as a whole needs to align its priorities to provide incentives for quality improve- ment, such as through the implementation of regulatory requirements and pay for performance. Program Coordination The QIO program is only one of several health care qualityrelated efforts under way within CMS, which increases the need for coordination within Medicare and CMS as a whole. Some of that coordination may take place when other offices within CMS desire to use the QIO apportionment to fund their research or other activities or to use the apportionment for policy planning at broader levels. Support Contracts and Special Studies Chapter 7 described the various review and funding mechanisms for the special studies and support contracts. At the beginning of the SOW, the program indicates priorities for special studies, but unsolicited proposals may be considered and funded later in the contract period for the SOW. However, no apparent mechanism exists for coordinating projects and fund- ing priorities among those projects. Also, it is unclear how CMS shares information about ongoing studies with the QIO community or what it does with the results of all studies. As one QIOSC representative stated in an interview, "I have no idea what CMS does with special studies' results." In the IOM committee telephone interviews, 8 of 20 QIO CEOs com- mented on the pros and cons of pilot testing. All found pilot testing to be favorable from the standpoint of having experience with the task at hand before all QIOs approach that task. However, two CEOs cautioned that sometimes pilot studies are not always the answer, as they can be too state specific and may not have been translated for a wider audience. One CEO commented further that the oversight of special projects is sometimes as- signed to a middle manager at CMS with no expertise in the topic area of the project. Quality Coordination Team Recently, CMS made efforts to coordinate the quality improvement efforts across all programs of CMS. On September 14, 2004, CMS an-

OCR for page 325
354 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM nounced the creation of a new Quality Coordination Team to support and act as staff to the redesigned Quality Council (CMS, 2004c). The adminis- trator of CMS, Mark McClellan, chairs the Quality Council; and its mem- bers include the director of each major CMS office. The Quality Council strives to coordinate all CMS efforts related to quality as well as to align those efforts with the quality improvement activities of other public and private organizations (Jencks, 2004). The Quality Coordination Team is led by Steve Jencks, former director of the Quality Improvement Group. Almost all team members are staff from CMS. In July 2005, CMS released the Quality Council's Quality Improve- ment Roadmap to improving the quality of care (CMS, 2005b). This roadmap included five major strategies: working through partnerships, pub- lic reporting, paying for quality performance, promoting efficient systems (such as electronic health systems), and increasing the availability and im- proved use of innovative technologies. Major activities of the Quality Coor- dination Team include direct support of the Quality Council, such as moni- toring of work groups, facilitation of partnerships and collaboratives, and participation in breakthrough projects. Topics chosen for focus in the work groups include performance measures and pay for performance, health in- formation technology, the Medicare Part D prescription drug benefit, and CMS Regional OfficeCentral Office communications. Breakthrough proj- ects include Fistula First, which seeks to improve vascular access in patients with end-stage renal disease; a project that seeks to raise immunization rates in specific settings; and the Institute for Healthcare Improvement's 100,000 Lives campaign (personal communication, S. Jencks, CMS, July 21, 2005). The Quality Coordination Team strives to facilitate partnerships within CMS, with other federal agencies (such as the Centers for Disease Control and Prevention and the Agency for Healthcare Research and Qual- ity), and with nongovernmental organizations (such as the Institute for Healthcare Improvement). The Quality Council and Quality Coordination Team have no direct responsibility for the QIO program, but many CMS staff on the team are directly responsible for the operation of the QIO program. Measures Selection and Coordination Some QIO functions, such as the selection of quality measures, require coordination with national stakeholder organizations as well with various offices in CMS. CMS identified four criteria for the selection of measures: the measures must be scientifically and clinically sound, the measures must be reproducible, the measures should not add burden to the provider, and

OCR for page 325
CMS OVERSIGHT 355 the measures should use existing data sources (CMS, 2004b). CMS worked collaboratively with the CMS Survey and Certification pro- gram as well as with the Joint Commission on Accreditation of Healthcare Organizations, the American Medical Association, and the National Com- mittee for Quality Assurance to align measure specifications to minimize reporting burdens on providers. Ultimately, the groups seek endorsement of selected measures by the National Quality Forum. In the 7th SOW, CMS contracted with the Health Services Advisory Group (Arizona's QIO) to maintain measures by the identification, standardization, and endorsement of measures with updating and retirement of the measures as needed (CMS, 2004b). The QIO program also funded work through other parts of CMS that contributed to the development and refinement of other measures, such as the support contract for the Consumer Assessment of Healthcare Provid- ers and Systems (CAHPS) family of surveys ($33.4 million during the 7th SOW) (personal communication, C. Lazarus, March 17, 2005). Roles and Relationships Overall, the QIOs express great concern about the relationships be- tween and among QIOs, Program Officers, Government Task Leaders, QIOSCs, and CMS. The QIOs want better definitions of the roles of each of these individuals or groups and streamlining of the management process. During the IOM committee's site visits, the QIOs discussed many chal- lenges related to CMS oversight of the QIO program. Seven of 11 QIOs expressed concerns over the relationship between CMS and the QIOs and communications problems. Specific examples included references to isola- tion of groups within CMS and poor communication between CMS, the Regional Offices, and the QIOs. The QIOs referred to "tension" in these interactions. Three QIOs specifically mentioned difficulties with ambigu- ous or poorly defined information, and three were frustrated with "micro- management" of the program. Five QIOs wanted more flexibility in the program in terms of either quality improvement topic areas or how goals are achieved. In interviews with five organizations representing seven QIOSCs, QIO staff members also mentioned difficulties in their relationships with CMS's Central Office and with their Government Task Leaders. As described above, all QIOs believed that the relationship with their Government Task Leaders was key, and they provided a range of responses as to whether or not that relationship was positive. QIOSC staff believed that they had lim- ited to no direct interaction with CMS's Central Office and wanted to see a reduction of silos and increased communication among and between the Government Task Leaders and CMS's Central Office.

OCR for page 325
356 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Elaborating on this issue, during the telephone interviews, one QIO CEO said, "CMS develops these contracts to develop expertise in an area, but then CMS doesn't listen to or take the advice of experts, so we end up with a program directed by Government Task Leaders rather than experts. This structure could use improvement." To get better service, this CEO stated, "There should be direct knowledge transfer [to QIOs] rather than having to get approval by a Government Task Leader at every step." The CEO asserted that "the Government Task Leaders don't seem to have any great urgency in approving materials, so lots of time can transpire. Govern- ment Task Leader delay cuts into the time available for technical assistance to QIOs." Another CEO reported that "there is often lack of clear direction from CMS on desired outcomes or that expectations change during the project. Sometimes change is inevitable as information is gathered, but that can substantially change the QIOSC resources. Reasons for change are more palatable if they are clinical rather than political." In these telephone interviews, the QIO CEOs also expressed concern about the interaction between the QIOSCs and CMS. CEOs pointed out that the QIOSCs have dual audiences, both the QIOs and CMS, and thus have a sense of being caught in the middle. Four respondents whose organi- zations held QIOSC contracts mentioned having difficulty in being able to respond to QIOs as a direct consequence of CMS delay. One said, "QIOSCs are caught between a rock and a hard place, with CMS being the rock that they have to be responsive to over the QIOs. What the QIOs want goes nowhere until CMS wants it to." Overall Program Evaluation Compared with the considerable effort that CMS put into designing complex formulas to evaluate the contract performance of each QIO (see Chapter 10), creating the databases necessary to evaluate their performance, and monitoring the progress of each QIO, it appears to have spent little time on evaluating other aspects of the program and the program as a whole. Although priorities for special studies are set by the Science Council of the Office of Clinical Standards and Quality, as described in Chapter 7, CMS has not developed a system for tracking all the various special study and support contracts, considering the balance of topics and spending, and de- termining how they might serve program priorities. Also, no system exists for broadly sharing the knowledge acquired through the studies or even letting all the QIOs and other Project Officers and Government Task Lead- ers know which QIO is working on a particular special study topic. The Quality Improvement Group was unable to provide the IOM com- mittee with information on the various contracts at a level of detail suffi- cient for the committee to know what the contracts are supposed to accom-

OCR for page 325
CMS OVERSIGHT 357 plish and whether accomplishments have been made, although individual Project Officers are required to assess their own projects. This lack of infor- mation prevents an assessment of the overall value and impact of the spend- ing on special studies and support contracts. In the 7th SOW, nearly 31 per- cent ($355 million) of the total program's apportionment ($1,154.3 million) was spent on special studies and support contracts (personal communica- tion, C. Lazarus, March 17, 2005). CMS does not have a mechanism or formula with which it can evaluate individual QIOs overall. Although the program can determine whether a specific QIO has achieved a passing score on its contract performance, it cannot distinguish outstanding QIOs from mediocre QIOs in a holistic sense. Although the QIOs vary widely on many organizational criteria, it is unclear which, if any, of those factors contribute to better performance, as it was not feasible to identify the better-performing QIOs. For example, at the end of the 7th SOW, one of the three QIO contracts for which re- competition was conducted in the first round was held by an organization that had been awarded one of the highest number of special study con- tracts. Also, as mentioned in Chapter 10, the committee's attempts to group QIOs according to their overall performance on the quality improvement subtasks were unsuccessful. The web-based data collection tool attempted to gather opinions about other QIOs by the QIO community itself, but the results were inconclusive. More importantly, neither CMS nor independent researchers have per- formed a conclusive evaluation of the impacts of the 53 QIOs on quality improvement nationally. Also, CMS has not performed a programwide evaluation to examine in detail the synergy, or lack thereof, between the spending on special studies, QIOSCs, and support contracts and the spend- ing on the core contracts. During the IOM committee's site visits, 4 of 11 QIOs independently related frustration with the contract evaluation process. They believed that the goals were too stringent and that too much emphasis was placed on short-term quantitative results. In the web-based data collection tool, 52 QIOs rated the clarity and timeliness of the evaluation process. Overall, the process did not receive high marks, with only one QIO giving a score of "excellent" on one of the three dimensions indicated in Table 13.8. More than half of the QIOs answered "fair" or "poor" for each of the three dimensions. In contrast to the QIO evaluations of the 7th SOW, QIOSC evaluations were informal and were primarily based on the completion of a set of deliverables, according to interviews with five organizations representing seven QIOSCs. All believed that their "success" was very subjective and based on the personal satisfaction of their Government Task Leaders.

OCR for page 325
358 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM TABLE 13.8 QIO Ratings of Evaluation Process Overall Timeliness Clarity of Clarity of of Information Quantitative Portion Qualitative Portion About Evaluation Rating of Evaluation of Evaluation (methodology, process, etc.) Excellent 1 0 0 Good 18 21 10 Fair 19 19 19 Poor 14 12 23 NOTE: The data in the table represent the number of QIOs responding as indicated. Data are for a total of 52 QIOs. SOURCE: IOM committee web-based data collection tool. SUMMARY This chapter has discussed CMS's oversight of the QIO program. The following are some of the main themes of this chapter, which are reflected in the findings and conclusions presented in Chapter 2: Multiple offices and divisions within CMS have responsibility for the QIO program. QIOs expressed frustration with the lack of coordina- tion and communication between and among personnel and with the time- liness of the information provided to them. They criticized the lack of coor- dination by CMS, which leads to competing agendas for different managers within the QIO program. One of the greatest concerns for QIOs was the time lag to the receipt of performance data because it affects their quality interventions as well as their contract performance assessments. QIOs oppose routine recompetition for the core contract because of the loss of momentum that it causes, the decreased incentive that QIOs have to share knowledge, and the chance that they might lose their contract. Overall, QIOs believe that the 3-year contract period is too short to achieve measurable change and is complicated by the concurrent lag in the time to receipt of performance data. They also believe that the timeline for QIOSC contracts should begin earlier so that the QIOSCs may help the QIOs immediately upon the start of a new SOW. Although CMS is developing a strategic plan for the QIO program 12 years into the future, the program still lacks distinct, focused priorities. Neither the core contracts nor the associated evaluation schemes prioritize the QIO activities.

OCR for page 325
CMS OVERSIGHT 359 Evaluations of the QIO core contract are based on overly complex formulas. They hold the QIO accountable for provider improvements on specified measures for short-term quantitative results. In contrast, QIOSC evaluations are mainly subjective and are based primarily on the satisfac- tion of the Government Task Leader and completion of a set of deliverables. CMS lacks any formal means of evaluation of the whole QIO pro- gram, its success on improving quality, or the distinction of the perfor- mance of one QIO over another. REFERENCES AHQA (American Health Quality Association). 2005. Proceedings of the AHQA Annual Meet- ing and Technical Conference. San Francisco, CA: American Health Quality Association. Beck C, Richard H, Tu J, Pilote L. 2005. Administrative data feedback for effective cardiac treatment: AFFECT, A Cluster Randomized Trial. Journal of the American Medical As- sociation 294(3):309317. Bradley EH, Carlson MDA, Gallo WT, Scinto J, Campbell MK, Krumholz HM. 2005. From adversary to partner: have quality improvement organizations made the transition? Health Services Research 40(2):459476. CMS (Centers for Medicare and Medicaid Services). 2002. 7th Statement of Work (SOW). [Online]. Available: http://www.cms.hhs.gov/qio [accessed April 9, 2005]. CMS. 2003. HHS Privacy Impact Assessment (PIA). November 18. [Online]. Available: www.cms.hhs.gov/privacyact/hcqis.pdf [accessed April 21, 2005]. CMS. 2004a. Proceedings of the QualityNet 2004 Conference. Washington, DC: Centers for Medicare and Medicaid Services. CMS. 2004b. The Quality Improvement Organization Program: CMS Briefing for IOM Staff. [Online]. Available: http://www.medqic.org/dcs/ContentServer?cid=1105558772835& pagename=Medqic%2FMQGeneralPage%2FGeneralPageTemplate&c=MQGeneralPage [accessed December 26, 2005]. CMS. 2004c. Notice from CMS Concerning Positions of Steve Jencks and Bill Rollow. Unpub- lished. Baltimore, MD: Centers for Medicare and Medicaid Services. CMS. 2004d. Quality Improvement Organization Manual. September 16. [Online]. Available: http://www.cms.hhs.gov/manuals/110_qio/qio110index.asp [accessed May 11, 2005]. CMS. 2005a. 8th Statement of Work (SOW). [Online] Available: http://www.cms.hhs.gov/qio [accessed April 9, 2005]. CMS. 2005b. Executive Summary. In: Quality Improvement Roadmap. [Online]. Available: http://www.cms.hhs.gov/quality/quality%20roadmap.pdf [accessed September 30, 2005]. CMS. 2005c. 8th Statement of Work (SOW), Version #080105-1. [Online]. Available: http:// www.cms.hhs.gov/qio [accessed November 4, 2005]. Hughes E. 2004. 8th SOW Contract Issues. Presentation at QualityNet 2004 Conference. Washington, DC: Centers for Medicare and Medicaid Services. Jencks SF. 2004. The Health Care Quality Improvement Partnership. Powerpoint Presenta- tion to Quality Improvement Organization Subcommittee, October 4, Baltimore, MD. Jost TS. 1991. Policing cost containment: The Medicare Peer Review Organization Program. University of Puget Sound Law Review 3:483526. Rollow WC. 2004. Evaluating the HCQIP Program. Powerpoint Presentation to Quality Im- provement Organization Subcommittee, October 4, Baltimore, MD.

OCR for page 325