Click for next page ( 56


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 55
2 Assessment of the QIO Program: Findings and Conclusions CHAPTER SUMMARY This chapter describes the study's evaluation approach, summa- rizes the key findings discussed in detail in Part II, and presents the committee's conclusions about the impact of the Quality Im- provement Organization program. The committee concludes that although the evidence regarding the program's effects on quality of care is limited and inconclusive, the program has the potential to be a valuable asset as providers become more involved with public reporting and incentive payments from Medicare and other insurers. The historical overview of the Quality Improvement Organization (QIO) program in Chapter 1 describes a public program that has changed direction, functions, and methods frequently over its 35-year history. Given the absence of earlier evaluations of the program, uncertainties exist with regard to its potential roles in the future. As a base for the recommenda- tions for the future of the program presented in Chapters 4 and 5, this chapter details the committee's findings and conclusions concerning the program's impacts. Specifically, the chapter addresses whether: There has been improvement in the quality of health care services provided to Medicare beneficiaries. The QIO program has contributed to that improvement. Certain components of the QIO program should be eliminated or strengthened. There should be a continuing federal role in technical assistance for quality improvement and, if so, whether that role should be stronger. It becomes clear from the descriptive details presented in Part II that the 55

OCR for page 55
56 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM QIO program is richly varied. The organizations holding the QIO contracts for each state often differ dramatically in structure, in the services they provide, and in their quality intervention programs, perhaps reflecting dif- ferences in the health care services in their communities. This variation makes it difficult to generalize about all QIOs and the program at the na- tional level, as well as to determine what changes in structure might be of most benefit to the program and yield the greatest improvements in the quality of care. The historical overview presented in the previous chapter shows how the program has changed over time. Nonetheless, many of the organiza- tions holding state contracts and some of their key leadership have persisted with the program through much of its evolution and have demonstrated remarkable flexibility in adapting to each new statement of work. Through that evolution, the QIOs have built partnerships with the providers and other key stakeholders in their states, such as state departments of health and local chapters of the American Association for Retired Persons. The committee gathered anecdotal evidence of many favorable and some less favorable relationships between QIOs and providers, brought to the com- mittee's attention at national conferences, site visits, and informal discus- sions with providers and consumers. In addition, results of large national surveys of physicians, hospitals, nursing homes, and home health agencies reveal generally favorable attitudes toward their local QIO. (The provider surveys are discussed in detail in Chapter 10.) The committee finds these relationships and the QIOs' capacity for adaptability to be valuable assets. The remainder of this chapter begins with a discussion of the first two questions raised above: (1) whether care for Medicare beneficiaries has im- proved, and (2) whether the improvements, if any, can be attributed to the QIO program. Alternatives to the current QIO program, along with their advantages and disadvantages, are considered next. The chapter then ad- dresses program infrastructure at both the national and state levels. Next is a discussion of the main functions of the QIOs, followed by a review of structural issues, such as funding and board composition. Finally, the chap- ter addresses the oversight responsibilities of the Centers for Medicare and Medicaid Services (CMS). Detailed data and analyses supporting the com- mittee's findings and conclusions are presented mainly in Part II and the appendixes, relevant portions of which are referenced throughout the dis- cussion here. EVIDENCE OF QUALITY IMPROVEMENT IN MEDICARE The Medicare program has carefully tracked the growth in the numbers of beneficiaries, the expenditures made on their behalf, and the increasing variety and use of covered services over the history of the program; how-

OCR for page 55
ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 57 ever, it has produced much less information on the quality of those services. The former director of the QIO program in CMS examined Medicare ser- vices by comparing data for 22 clinical inpatient and outpatient measures against state baseline performance reported through the QIO program from 19981999 to 20002001. It was found that the weighted national aver- age, as well as the state with the median performance, had improved on 20 of the 22 indicators (Jencks et al., 2003). A recent national study employing 9 of the same hospital quality indicators used by Jencks and colleagues (2003), along with publicly reported data on Medicare and all other pa- tients, found that the level of hospital performance had improved on all but 1 measure (Jha et al., 2005). Although these studies were less than ideal--in particular, the number of measures used in each study was limited, and the measures focused primarily on care provided within hospitals--their results indicate that health care for Medicare beneficiaries has improved. There are signs of improvement as well in clinical activities targeted as national pri- orities on which QIOs and other organizations have focused, such as mammograms and care for heart attacks and diabetes (Leatherman and McCarthy, 2005). Improvements in care for beneficiaries in managed care were also noted from 2000 to 2004 (NCQA, 2005). On the other hand, much evidence, cited in Chapter 1 of this report and in Performance Mea- surement: Accelerating Improvement (IOM, 2006), indicates that the qual- ity of care varies greatly from provider to provider by geographic location, race or ethnicity, and income of the beneficiary and that many people do not receive all the services they need, particularly appropriate preventive care. There is substantial room for further improvement. Can the gradual improvements in care that have been accomplished be attributed to the QIO program? Because of the nature of their evaluation designs, the studies mentioned above cannot be used to determine the cause of the improvements documented or attribute them to the QIO program. During site visits and focus group discussions conducted for this study, the committee heard frequent anecdotal evidence from providers and the QIOs regarding positive impacts on quality resulting from the multiple forms of technical assistance provided by the QIOs (see Chapter 8). Some studies of specific, limited QIO interventions or collaboratives have also documented improvements over time, but likewise cannot be used to conclude that the improvements were due to the QIOs' efforts (Marciniak et al., 1998; Kiefe et al., 2001; Gould et al., 2002; Chu et al., 2003; Daniel et al., 2004; Dellinger et al., 2005). In addition, preliminary data on several measures of the quality of care under the 7th scope of work (SOW) suggest that the QIOs may have had a positive impact on the care received from nursing home, home health agency, and hospital providers that participated inten- sively with their QIOs ("identified participants") in comparison with that received from all such providers statewide. The data also suggest that those

OCR for page 55
58 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM identified participants achieved greater improvements on measures related to the clinical area of their particular intervention relative to improvements achieved by participants who collaborated with QIOs on interventions un- related to those measures (personal communication, W. C. Rollow, July 8, 2005; personal communication, J. Kelly, September 8, 2005). The analyses were incomplete as of this writing, however, and it is unclear whether these apparent differences will be significant. Given the lack of consistent and conclusive evidence in scientific jour- nals and the lack of strong findings from the committee's analyses, it is not possible to determine definitively the extent of the impact of the QIOs and the national QIO infrastructure on the quality of health care received by beneficiaries. Many confounding factors make it difficult to attribute the results obtained thus far. Also, the literature does not address the QIO program as a whole; rather, it merely addresses the impacts of specific qual- ity improvement activities of individual QIOs or quality measures aggre- gated at the state or national level. Other aspects of the program, such as the impact of QIO case reviews on quality and the value of QIO Support Centers (QIOSCs), have received little or no scrutiny from evaluators. (See Chapter 8 for a discussion of QIOSCs.) One challenge to evaluating the QIO program is that QIOs recruit voluntary participants and generally partner with multiple stakeholder or- ganizations to conduct quality improvement interventions (see Chapter 8). It would be difficult to identify true control groups; random assignment has not been tried because of practical and political implications; studies often rely only on changes that have been observed compared with perfor- mance at the baseline; it is difficult to measure impacts within the time frame of the study; and it is nearly impossible to distinguish the impacts of a QIO from those of its partners and other environmental factors. Also, the nature of the interventions varies depending on the provider type and the QIO, as well as from one provider to another, because the intervention methods used are determined largely by each QIO. The voluntary nature of provider participation with QIOs introduces the possibility of bias in the self-selection of participants, which in turn limits the value of compari- sons of the rates of change for identified participant groups with the rates for the entire state. Also, because the QIO has the responsibility to raise the level of quality statewide as well as for identified participants, a large difference between the two rates of quality improvement may indicate a particularly successful program for identified participants but a very weak statewide program. These research limitations and inconclusive findings in the literature are not limited to studies of the QIO program, but also ham- per other studies of improvements in health care quality. In addition to studies related to the QIOs, the literature review per- formed for this study focused on more generic quality improvement studies

OCR for page 55
ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 59 in health care. In addition to 23 studies focused specifically on the QIO program and various QIO-led quality improvement interventions, the com- mittee examined 9 articles, including 2 Cochrane reviews (the latter system- atically assessed a wide range of studies according to explicit, high stan- dards). (This literature review is discussed further in Chapter 9, and selected articles are summarized in Tables A.1 and A.2 in Appendix A.) The studies paint an inconclusive picture of the effectiveness of quality improvement programs, whether the interventions are conducted by QIOs or other orga- nizations, for both Medicare and non-Medicare services. In part, this lack of conclusive data results from the research challenges mentioned above. Few of the studies used randomized controlled trials or control groups. An examination of additional literature on the transfer of knowledge about various quality improvement methods also provided inconclusive findings on the effectiveness of these methods; no specific method has been identified as best. Studies do show the importance of commitment on the part of an organization's leadership, staff empowerment, and the develop- ment of a plan to promote quality improvement (Bradley et al., 2005; CMS, 2005a). Analysis of the management literature on effecting change to im- prove patient safety in health care organizations reveals that several human resource management practices are helpful in that process. In particular, to promote successful change and the adoption of new practices, it is impor- tant for management to promote "ongoing communication; training; use of mechanisms for measurement, feedback, and redesign; sustained attention; and worker involvement" (IOM, 2004:118). Knowledge transfer is considered an important part of the QIO pro- gram, although evidence that it occurs is meager. A significant portion of the functions of the QIOSCs relates to knowledge transfer among the QIOs: training the trainer, the establishment and maintenance of communities of practice, and the identification and promotion of best practices. During the site visits, telephone interviews, and focus group discussions conducted by the committee, the chief executive officers (CEOs) of the QIOs emphasized the need to include early adopters, leaders, and champions, in addition to middle and late adopters, in quality interventions to promote their uptake by practitioners and institutions beyond those involved in the interventions. On the other hand, focusing solely on providers that are lead- ers and champions and may be relatively easy to recruit to a QIO technical assistance program could limit the participation of providers that may need the QIO's assistance the most. The QIO program considers sharing among QIOs to be important and has established mechanisms for this purpose. For example, the shift in the 7th SOW from a relative failure rate definition for QIOs that must re- compete to an absolute target for automatic renewal meant that QIOs were more willing to share and help each other. They were no longer competing

OCR for page 55
60 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM to avoid the penalty associated with being in the bottom quartile; they were all trying to meet absolute targets and could cooperate without jeopardy. The 8th SOW maintains similar incentives. While the QIO contracts and QIO collaboratives are based on sharing and knowledge transfer, other forces in health care are pushing providers in the opposite direction, toward more competition. Under the pressures of public reporting, no providers will want to see their name on the bottom half of the quality-of-care list, which would indicate that the care they pro- vide is of below-average quality. Pay-for-performance schemes are likely to reinforce that competition, depending on the reward structure. Unintended consequences of public reporting and pay for performance should be exam- ined, but no evidence yet exists on what impacts, if any, these programs will have on competition, and it is unclear how QIO collaboratives will work as the environment becomes more competitive for providers. In some commu- nities, providers may welcome the use of public reporting and use it to benchmark and identify other providers from which they could learn. In any case, the program should be considering new options for QIO interven- tions and mechanisms, such as web-based training and self-guided study tools, to extend the QIOs' reach to more providers. Because of the limitations of the scientific literature, the committee at- tempted to use program-generated data, including the performance scores used by CMS to evaluate each QIO's contract performance, to identify high- and low-performing QIOs (see Chapter 10). CMS conducts separate per- formance assessments for each SOW task and subtask. The scores related to Task 1 (technical assistance for quality improvement) define "quality" for the program. The scores used by the committee are based on specified clini- cal measures for hospitals, nursing homes, home health care agencies, and physician offices during the 7th SOW through December 2004. To retain its QIO contract during the next SOW without competing, a QIO must attain a passing score for each task; no total or average score based on all tasks is assigned to each QIO. The committee's analyses revealed the following: A QIO's score in one provider setting did not correlate with that in any other setting. No correlation existed between spending per beneficiary in a state on the QIO's technical assistance in a particular setting and the QIO's per- formance measure for that setting. No correlation existed between the state performance score in a par- ticular setting and measures of the provider's satisfaction with the QIO. Some regional variations in the performance scores were noted, as were variations in contract rounds for the home health care provider scores (see Chapter 13 for a discussion of contract rounds).

OCR for page 55
ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 61 The committee was unable to ascertain why one QIO might have a higher performance score than another in a particular setting. One might expect that many skills needed to carry out a quality improvement interven- tion would be transferable from one provider setting to another. They may in fact be transferable, but perhaps some QIO staffs are not well organized to effect that transfer of knowledge, or the provider communities and their stakeholder organizations may differ in ways that affect the success of the QIOs' interventions. Also, the role played by providers' voluntary partici- pation in the impacts of the QIO program is unclear. One study of hospital and ambulatory care measures showed that absolute improvement on 21 of the 22 measures tracked was greater for those states starting at a lower- quality baseline than for those starting at a higher-quality baseline (Jencks et al., 2003). While many inferences can be drawn about why a trend can- not be identified to link improvement and setting of care, it must be noted that some skills and expertise required to provide assistance in each setting are not necessarily the same. Given limited resources and the need to com- plete other tasks, such as communications and beneficiary protection, QIOs must choose how best to allocate their resources. The committee heard consistent reports of extensive variations among the QIOs: some are outstanding, while others are mediocre. These judg- ments appear to reflect an overall perspective on the organizations rather than the QIOs' performance on specific tasks. Yet there are no objective global measures, based on contract performance scores for quality improve- ment, that could indicate which QIOs belong in which category or even whether there are in fact significant differences in overall performance. For example, some QIOs score high on improving care in one setting but not in others. The complex and detailed formulas used by CMS to assess each state's performance on the core tasks of the SOW can be used to determine the QIOs' performance only on each task separately. For example, 6 of the 36 QIOs in the first two contract rounds failed on at least one assessment standard (task or subtask) and had to recompete for an 8th SOW contract. Those 6 QIOs included some that would not generally be viewed as belong- ing in the bottom tier on overall performance, some that had received sub- stantial contracts from CMS to conduct special studies or to serve as a QIOSC, and some that were named "best" at particular tasks by other CEOs according to the committee's web-based data collection tool. ALTERNATIVES TO THE CURRENT QIO PROGRAM In considering alternatives to the current QIO program, the committee decided it was necessary to step back and ask some fundamental questions. First, should the federal government exercise a stronger role in the pro- vision of technical assistance to health care providers to promote more rapid

OCR for page 55
62 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM improvements in health care quality? The evidence base for the committee's Performance Measurement report (IOM, 2006) and for the present study (see the introduction to this report) indicates that the health care received by Americans is seriously inadequate, and that the health care system needs to produce better-quality care that meets standards supported by the best scientific evidence. While these shortcomings have been in the headlines and discussed extensively in the provider community for many years, progress toward improved quality continues to be painfully slow. The com- mittee concludes that the public sector needs to play a substantial role in improving the quality of care for all Americans. This is especially so for those who depend upon federal programs, such as Medicare, Medicaid, the State Children's Health Insurance Program, the Department of Defense's TRICARE, and the programs of the Veterans Health Administration and the Indian Health Service, which provide coverage and care to roughly 100 million people (IOM, 2002). The creation of a performance measurement and reporting system and the implementation of payment systems that re- ward quality care are important steps in accelerating improvement, but they are insufficient to achieve the six quality aims outlined by the IOM (IOM, 2000, 2001). The committee believes the federal government is well suited to promote better-quality care for all Americans because it spends more than $513 billion annually on their care. The magnitude of this investment can generate positive changes in quality throughout the health care system (IOM, 2002). As the IOM concluded previously, "the federal government must assume a stronger leadership role to address quality concerns" (IOM, 2002:x). Second, given a role for the federal government in providing technical assistance for quality improvement, should that assistance be offered at no cost to providers, as is now the case through the QIOs, or should providers be expected to pay for the assistance? The following findings can help an- swer this question: Many providers have resources that allow them to purchase techni- cal assistance from private organizations, such as those discussed in Chap- ter 3 and listed in Table B.1 in Appendix B. Also, many providers spend considerable sums to hire quality improvement experts and to conduct im- provement programs internally. While the extent and distribution of such expenditures are unknown, they are generally considered to be more com- mon among hospitals, large physician groups, and managed care organiza- tions than among medium and small physician practices and other provider settings. The committee does not wish to discourage such behavior. At the same time, particular attention needs to be paid to disparities in access to and use of technical assistance resources within the provider community. Some providers are ready and willing to undertake the internal

OCR for page 55
ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 63 changes needed to improve quality but lack the resources to purchase the technical assistance they need to achieve these reforms. Continuing to achieve improvement at the current rate would be costly in lives and dollars and is therefore unacceptable. Regardless of the current level of expenditures by providers for tech- nical assistance to improve quality, the end result is clear--improvements in quality nationally are progressing too slowly. While many providers may be willing to expend their own resources to improve quality when payment becomes based on such improvements, others are likely to be left behind. Some providers simply do not have sufficient resources to purchase the necessary services readily. The expansion of pay-for-performance systems to cover a greater portion of a provider's revenues is likely to worsen this situation. Providers with low levels of quality according to the given metrics may receive no increase or perhaps even a decrease in reimbursements. In some areas, poor-quality providers may be the only ones available to a portion of the population, and their failure would further exacerbate disparities in health care access and quality. The committee concludes that it is important to improve the level of care offered by all providers, even if they cannot afford to purchase techni- cal assistance privately, and that some level of technical assistance should be available through the federal government as a public good. As providers improve their care processes for Medicare beneficiaries, it is likely that other patients will benefit as well. PROGRAM INFRASTRUCTURE Given the committee's limited ability to attribute quality improvements in Medicare directly to the efforts of the QIO program, it is necessary to consider whether the current program should continue. Although it may appear obvious, the committee believes the existence of 41 separate organi- zations holding QIO contracts dedicated to providing quality improvement services in every state, the District of Columbia, Puerto Rico, and the Virgin Islands is a significant asset. The cadre of trained experts in QIOs (see Chap- ter 7) is a potentially valuable resource for offering technical assistance in quality improvement and for helping hospitals, nursing homes, home health agencies, and outpatient physician practices collect data on their perfor- mance and aggregate and analyze those data to improve the care they deliver. The QIO program has created an infrastructure across the United States with staff trained and experienced in various quality improvement tech- niques. The QIO program serves as a focal point within CMS for assisting health care service providers in improving the care they offer to Medicare

OCR for page 55
64 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM beneficiaries and others. It is also the only program within CMS that ac- tively addresses health disparities (Jost, 2005). The following are examples of the QIOs' expertise and experience drawn from the committee's QIO and Regional Office site visits, telephone interviews, and web-based data collection tool: All but seven QIO contracts are staffed with at least one employee who is a Certified Professional in Healthcare Quality (through the Health- care Certification Board, the National Association for Healthcare Quality, or a similarly recognized professional accreditation in quality improvement). Eleven QIOs have from 10 to 34 such certified staff. Eighty-eight percent of QIO CEOs believe their leadership staff have substantial competencies for such functions as collaboration, relationship building, team development, and performance measurement. Many CEOs stated that their QIOs have individuals with other leadership skills avail- able to carry out the tasks of the 8th SOW. A national survey of the QIOs by the Best Practices QIOSC (Qualis Health, 2004) showed that the quality improvement management staff of many QIOs were familiar at a minimum with various quality improvement techniques and programs, such as the Baldrige criteria, a collaborative meth- odology, human factors, International Standards Organization (ISO) 9000 criteria, Lean principles, and the Six Sigma program (see Chapter 7 for a discussion of QIO staff training and Chapter 9 for a description of each method). Some QIOs indicated having used some of those tools; only a small number of QIOs reported having staff certification for specific pro- grams, such as Six Sigma. Collaboratives were a major method used in the 6th and 7th SOWs, and many QIO staff were trained in its use. This is reflected in the fact that 98 percent of the QIOs reported being familiar with the method, and 95 percent of the QIOs reported having used it (per- sonal communication, J. Kelly, June 29, 2005). Almost half of the respon- dents were familiar with even the least common methods--those based on ISO 9000 criteria and Lean principles (Qualis Health, 2004). The QIOs had a separate task in the 7th SOW to reduce disparities between an identified group, such as a particular underserved or rural popu- lation, and a reference group. In the 8th SOW, the disparities task was folded into all the quality improvement technical assistance tasks rather than remaining separate (see Chapter 8). All QIOs gained experience in assisting hospitals with reporting mea- sures during the 7th SOW, when nearly all hospitals decided to participate voluntarily in public reporting to avoid a payment reduction. QIOs were also involved with the public rollout of comparative nursing home mea- sures (see Chapter 11).

OCR for page 55
ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 65 Not only does each QIO have technical expertise in quality improve- ment, data management, and data analysis, but significant expertise in these areas also exists within CMS at the national and regional levels (see Chap- ter 13). The Central and Regional Office program staff may appear to be small (132 full-time equivalents) relative to the size of the program they oversee, the variety of tasks they perform, and their responsibilities for im- proving the quality of care for Medicare beneficiaries. Among the staff, however, are people with advanced training and experience in medicine, nursing, quality improvement, epidemiology, scientific research, and data systems (CMS, 2004; personal communication, J. Kelly, September 8, 2005). Although frustration and complaints with regard to the staff at CMS were voiced frequently during the committee's site visits to QIOs, telephone interviews, and other informal meetings, 85 percent of the QIOs rated their Project Officer "good" or "excellent" on expertise; the Scientific Officers received similarly high ratings on their performance (see Chapter 13). The QIO program's administrator reported that the program has sufficient staff and expertise to conduct the work required under the 8th SOW (personal communication, W. C. Rollow, July 8, 2005). In addition, the program has established potentially useful national communications networks and a data repository that currently serve both the QIOs and the public reporting of hospital data (CMS, 2004) (see Chapter 13 for further discussion of the data systems). The committee notes that at both the state and the national levels, the QIOs and CMS have established important working relationships with pro- viders, their professional associations, and various other stakeholder groups and convened parties around specific issues (CMS, 2004; Westat, 2005; National Health Policy Forum, 2004). The QIOs and CMS have also been major participants in such collaborative efforts as the Hospital Quality Al- liance and the Ambulatory Care Quality Alliance. Both the convening abil- ity of the QIOs and the clout of CMS to bring national organizations to the table are key ingredients for promoting widespread, coordinated quality improvement. The committee recognizes the expertise available within the QIO pro- gram and the enthusiasm, commitment, and dedication to quality improve- ment exhibited by staff and leadership from the Central Office at CMS, the Regional Offices, and all the organizations holding QIO contracts. The com- mittee concludes that the potential exists for the QIO program to have a measurable positive impact on improving the quality of care for Medicare beneficiaries, serving useful and important functions in the rapidly chang- ing world of Medicare.

OCR for page 55
ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 71 Broader representation of various disciplines and expertise--The QIOs have diverse constituents, including hospitals, physician practices, nursing homes, and home health care agencies. However, the QIO boards appear to have relatively few members from health professions other than physicians. The committee found that physicians dominated nearly all the QIO boards, that two-thirds of all board members listed in the IOM's web- based data collection tool were physicians, and that all responding QIO CEOs (39 of 41) listed representatives of office-based practices on their boards. In particular, nurses (especially those in home health care), physi- cian assistants, pharmacists, and physical and occupational therapists ap- pear to be in short supply on QIO boards and are nonexistent on most boards, even though these types of health care professionals could play critical roles in improving the quality of care. Boards need to become more inclusive, encouraging participation from a variety of stakeholders and rel- evant experts. The growing responsibilities of QIOs during the 7th SOW included assisting stakeholders with data measurement, collection, aggrega- tion, analysis, and reporting as part of their provision of technical assis- tance for quality improvement. The committee expects those functions to increase in importance, and the minimal representation of QIO board mem- bers with expertise and experience in information technology management and oversight is a concern. Board members from outside the health care sector with experience in quality improvement, process measurement, and performance accountability could also contribute to the governance of QIOs. At the same time, when considering the addition of consumers, health professionals, and others to its board, a QIO must balance the need to keep the board's membership to a workable number. The creation of alternative mechanisms, such as advisory committees, may be necessary to obtain the varied input needed while keeping the board to a manageable size. Board member development and assessment--Studies have shown that organizational leadership can benefit from the implementation of a systematic plan for individual board member development, an annual as- sessment of each board member's performance, and an evaluation of the board as a whole (Orlikoff and Totten, 2005). Such assessments are par- ticularly important in light of recent scandals involving both for-profit and not-for-profit health care entities; passage of the Sarbanes-Oxley Act of 2002 (P.L. 107-204), which brings greater accountability and transparency to the for-profit corporate world; and efforts in the not-for-profit sector by Board Source, Independent Sector, the Aspen Institute, and the Center for Healthcare Governance, which have recommended new requirements for board accountability. Just under a quarter of the 41 organizations holding contracts under the 7th SOW reported that they had formal mechanisms in place for evaluating the performance of individual board members; the same proportion of QIOs evaluate the overall performance of their boards.

OCR for page 55
72 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Financial oversight and strategic guidance--Only 23 of the 41 orga- nizations with QIO contracts have a board finance committee, only 15 re- ported that they have an audit committee, and only 8 reported that they have a strategic planning committee. Thus there appears to be a need for increased attention to these important areas of board responsibility to pro- vide adequate oversight and strategic direction to QIO management. Overall responsibility--In addition to financial oversight and strate- gic guidance, the QIO board should be responsible for assessing the overall performance of the CEO and senior staff, as well as the accomplishment of defined goals and priorities. Physician-Access or Physician-Sponsored QIOs Outmoded Requirement The legislative requirement that QIOs be physician-access or physician- sponsored organizations was intended to assure physicians that their clini- cal work would be reviewed by their peers on the basis of local practice standards. Although certain clinical assessments and case reviews still re- quire medical expertise, the physicians need not be local because quality of care is now defined according to evidence-based national guidelines and standards rather than local patterns of practice. The committee concludes that this requirement is now outmoded and unnecessarily limits the compe- tition for QIO contracts from other entities. Limited Competition The committee finds that few entities other than out-of-state QIOs have been serious competitors for QIO core contracts in the past or are expected to compete for the 8th SOW, given the wide and complex assortment of required tasks and structural requirements. Other entities that are not physician-access or physician-sponsored organizations might have the ca- pacity to perform all or portions of that contract (see Chapter 3 for further discussion of "other entities"). Yet there has been a history of very limited competition for QIO contracts, and there have been few opportunities to replace QIOs that did relatively poorly or that failed on their previous contract: All but two of the QIOs in the 7th SOW held their state's contract under the 6th SOW. Fully 70 percent of the 7th SOW contractors were also contractors under the 1st SOW.

OCR for page 55
ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 73 No outside entities bid against the previous QIO for three of the seven state contracts opened for competition for the 7th SOW. As of the end of the 7th SOW, five contracts had been recompeted and one was to be; there were no competitors for three of five contracts (personal communication, S. Pazinski, November 14, 2005). Conflicts of Interest Federal regulations for QIOs are designed to foster impartiality and to prevent conflicts of interest between members of the QIO board and the health care providers whose care is the focus of the QIO assessment through case review (see Chapter 7). These regulations contribute to a perception of neutrality and respect for the QIO, as well as acceptance of its quality im- provement and case review tasks, among the provider community and ben- eficiaries. They also mean that the QIO boards have derived limited benefit from the involvement of executives from the provider community, such as hospital CEOs whose hospitals might come under the review of a QIO. Under the 8th SOW, organizations that hold QIO contracts are not permitted to contract with providers in their state(s) for technical assistance or to review services for Medicare beneficiaries that are similar to the assis- tance or services they provide under the QIO contract. Although some pro- viders and insurers in some states may wish to purchase additional QIO services beyond those supported by Medicare funds, the QIO is not allowed to negotiate such a contract (personal communication, D. Schulke and T. Ketch, American Health Quality Association, June 30, 2005). This prohibi- tion appears to be based on a perception that lucrative contracts with the QIO could influence relations with certain providers and the review of their cases. The potential for such conflicts appears less likely now, however, given the decreased emphasis on case review and sanctions since the early days of the Professional Standards Review Organization program. There is clearly a market demand for the additional services of some QIOs, and under its current level of funding, the Medicare QIO program is unlikely to meet fully the future need for technical assistance with quality improve- ment. The committee finds that many QIOs have demonstrated a capacity to develop external revenues (not from CMS) and extensive relations with providers and provider organizations within their own states. At least 15 QIOs receive more than half of their revenues from sources other than the core contract. The committee concludes that if the case review functions were no longer maintained within each QIO, the need for this contracting prohibition might be eliminated. Reasonable controls, however, would be necessary to prevent favoritism in the QIOs' selection of recipients of their technical assistance with Medicare quality improvement.

OCR for page 55
74 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Confidentiality Restrictions The attitudes of the early predecessors to the QIOs and physician pres- sure have generated strict confidentiality restrictions on the QIOs' treat- ment of clinical data. The committee concludes that these restrictions pro- tect the privacy of providers and are not well suited to the current need for transparency, public reporting, and consumer access to information (Gaul, 2005). The restrictions also constrain the use of data on quality measures to support the quality improvement process. These restrictions were estab- lished mainly through regulations, and the secretary of DHHS could make policy changes consistent with Health Insurance Portability and Account- ability Act (HIPAA) regulations without the need for new legislation (per- sonal communication, T. Jost, Washington and Lee University School of Law, December 21, 2004). Consistent with CMS's concerns regarding the use of data and protec- tion of providers, CMS required that for this study, the committee use mea- surement data on the performance of each QIO only in the aggregate for the nation as a whole, or deidentify the data to protect the QIOs. The IOM had never requested access to physician or patient data, which would have required protections. The committee finds that the program's lack of trans- parency concerning its key contractors is incompatible with the broader trends within Medicare and the health care system to disclose providers' quality measures through public reporting. OVERSIGHT OF THE QIO PROGRAM3 CMS has the challenge of managing the QIO program in the field, as well as integrating it into the operational responsibilities of the Medicare program. The main oversight functions required of the CMS Quality Im- provement Group, which runs the QIO program, include the following: Operation of complex data processing and communications systems for the program Management and evaluation of QIO core contracts and contracts for QIOSCs, special studies, and support contracts Strategic planning needed to ensure the continued usefulness of the QIO program to the Medicare program Chapter 13 describes the various communications and management mecha- nisms used by the QIO program and its data systems. 3Most of the information in this section was drawn from the committee's site visits, tele- phone interviews, and web-based data collection tool.

OCR for page 55
ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 75 Data Processing and Communications Communications among the QIO Central Office, the Regional Offices, and the QIOs are maintained through e-mails, list serves, restricted and public websites, and electronic reporting tools. To some extent, the QIOs are expected to operate in the virtual world, and they are encouraging pro- viders to move in that direction as well. Training for CMS staff and pro- grams, including QIO staff, is conducted over the Internet; communities of practice stay in touch with monthly conference calls; and meetings are video broadcast across the country to minimize travel. Some of these mechanisms do not work as well as the QIOs would like: CEOs claim that the frequency of conference calls makes it impossible to participate every time; that they do not receive policy memoranda and other necessary guidance promptly; and that reporting systems, such as those for case review and program ac- tivities, are difficult to use. Management communications among CMS Project Officers, QIOs, and CMS Contract Officers are particularly problematic. A QIO can be caught between two different interpretations of an issue, and the interpretation of the Contracts Office has priority. The CMS Project Officer provides routine guidance to the QIO and often serves as a communications link when policy questions arise. While the Project Officer may request an activity outside the scope of the QIO contract and beyond the budget, however, the Con- tracts Office may subsequently refuse to consider a contract modification. The Standard Data Processing System supports the communications tools mentioned above, as well as the flow and processing of data from medical records. It is key to the QIO program, and 63 percent of the QIOs rated the value of the system as excellent or good, although some QIOs mentioned the need to update the system and integrate it with their own equipment. It will be important for QIO and CMS staff to be closely involved with national and regional initiatives concerning data exchange, to understand strategic policy issues related to health information technol- ogy, and to keep their software and hardware current. The main concern QIOs expressed is that the data reported through the Dashboard section of the QIONet internal CMS website and used to monitor the progress of the QIOs' quality improvement efforts often are not timely (see Chapter 13). Although there is some conflicting evidence about the value of rapid data feedback (Beck et al., 2005), the QIOs and providers frequently stressed the usefulness of real-time data for quality improvement and for the care of specific patients. As CMS increases the measures required for public reporting by hospi- tals and physicians, the challenges to the Standard Data Processing System will increase. (CMS conducts much of the processing of the data for nurs- ing home and home health care measures.) The committee concludes that

OCR for page 55
76 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM the growing volumes of data, the expanded numbers of data sources, the increased need for the timeliness of data processing, and the interest in user-friendly data reports for different stakeholders will exert pressures on the current system. At the same time, the transition to electronic health records may reduce the demand for data abstraction, increase the need for data-auditing services, and improve the timeliness of the data. Long-range planning for these changes is essential to smooth operations and should include an array of people with detailed operational expertise early in the process. The implementation of a national system for quality measurement and reporting will have an additional impact on the demand for data pro- cessing and other activities (IOM, 2006). The committee recognizes, how- ever, that full implementation of both electronic health records and the new national public reporting system will take time. Therefore, paper-based tools that can be used prospectively, as opposed to chart abstraction, will also be important. With DHHS fully supporting providers' move to the use of electronic health records, as well as electronic communications, Medi- care could serve as a guide to the future if its data systems keep pace with the transition. QIO Contract Management and Evaluation CMS's day-to-day management of each QIO contract is a major re- sponsibility of staff in the Regional Offices, who perform detailed on-site assessments at 9 and 18 months into the contract period and in the interim keep in touch with the QIOs to which they are assigned. However, most of a QIO's performance rating for contract renewal is based on data on pro- viders' performance on quality measures and complex formulas, with sepa- rate calculations for each task. The increased complexity of tasks in the 8th SOW led to a significantly more complicated set of formulas and incentive awards than those in the 7th SOW (see Chapter 10). The absence of an overall evaluation plan, guidance, and program priorities has created a situ- ation in which each of CMS's Government Task Leaders designs the evalu- ation formulas for his or her own tasks in the SOW. The formulas do not draw upon consistent time frames, goals, or definitions. The resulting com- plexity makes it difficult to construct a coherent overall assessment of a QIO. It is unclear, therefore, whether these formulas adequately assess "im- provement" and "quality," and the committee finds that the program's sys- tem for evaluating the QIOs requires further development to enhance its effectiveness. As recommended in a 1994 IOM letter report, staff with ex- pertise and experience in program evaluation, as well as outside assistance, are needed to design formative and summative evaluations of the QIOs and to contribute to the design of contract performance reviews (IOM, 1994). The complex method for evaluating the contract performance of QIOs

OCR for page 55
ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 77 under their core contract is in sharp contrast to the limited, relatively simple methods used in the 7th SOW to assess the performance of the QIOSCs, QIOs conducting special studies, and organizations carrying out support contracts. Although these contracts are assessed for timeliness and com- pleteness of deliverables and spending, CMS relies primarily on the judg- ments of the Project Officers. It may be noted that contracts for QIOSCs, special studies, and program support accounted for almost one-third of the total apportionment for the 7th SOW. The value of these expenditures is diminished by the lack of a comprehensive listing of these various contracts and summaries of their contents and results, which are thus not diffused throughout the quality improvement community. Although many of the support contracts are related tangentially to the QIO program, QIOs may be unaware of those contracts or of a special study that is under way and would be of interest to them. CMS is in the process of developing a manage- ment and accountability plan to better administer all of its contracts. Few details were available as of this writing, however, and implementation of the plan remains to be seen. One factor that may have contributed to the previously discussed lim- ited competition from other entities for the 8th SOW was the timing of submission of proposals for the core contract. The request for proposal was released on April 1, 2005, and the deadline for submission of proposals was April 15, 2005, before the SOW was final and the full amount of funds available was known. An official version of the 8th SOW was issued on May 20, 2005, but some elements were still under discussion within DHHS and the Office of Management and Budget that could have required future and unknown modifications to the SOW. In the first contract round, the QIOs were required to submit their bids before the release of the final SOW, and the bids had to be based on a draft SOW and spending estimates. Ma- jor revisions to the 8th SOW were made on November 4, 2005, well past the beginning of the new contract period. Because late revisions can affect QIO staffing and work schedules, QIOs reported that the timing has been frustrating and has hampered their ability to prepare for the start of the new contract (personal communication, D. Schulke and T. Ketch, Ameri- can Health Quality Association, June 30, 2005). Additionally, delayed re- leases may deter organizations with no previous experience in holding a QIO contract from bidding because of the inadequate amount of time to prepare. During the committee's site visits and telephone interviews, several QIOs indicated that the 3-year length of the contract is also a problem. They believe this period of time is insufficient to obtain data describing the initial impact of their quality improvement projects and to allow them to adjust their interventions and later be remeasured for their contract perfor- mance evaluations. Given the experience with the preparation for the 8th

OCR for page 55
78 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM SOW, as well as the increased expectations of this SOW, it appears that CMS would also benefit from a longer contract period. Strategic Planning Two positive signs concerning policy planning for the QIO program suggest to the committee that CMS is moving in the right direction. First, the Quality Improvement Group has initiated a long-range planning pro- cess to better position the program 12 years into the future. Although some strategic planning discussions have recognized the importance of coordi- nated care and crosscutting issues in improving the overall safety and qual- ity of care, representatives of hospitals, home health care agencies, physi- cian practices, and nursing homes have not had opportunities to meet to discuss appropriate measures and mechanisms for tracking a patient being released from one facility and placed in another. Much of the internal stra- tegic planning process has focused on individual tasks and separate Gov- ernment Task Leaders; the external planning process with stakeholders has followed the same pattern of separating each of the four main health care settings. The committee concludes that implementation of the new Part D prescription drug benefit and therapeutic pharmacy strategies offers impor- tant opportunities for QIOs to focus on the coordination of care across provider settings. Strategic planning separately for different providers and care settings is inconsistent with future measures of quality and perfor- mance that will be encouraged by the national performance measurement and reporting system recommended by this committee (IOM, 2006). Another positive sign concerning strategic planning is the rejuvenation of the CMS Administrator's Quality Council and its staffing by the Quality Team. This management structure could give the QIO program an opportu- nity to integrate its activities with those of other CMS functions, such as public reporting, in the early planning stages. The council has adopted a vision of the QIO program--"the right care for every person every time"-- and has made it the vision of the whole agency. The Quality Council is coordinating the development of a number of quality-related projects and presented a Quality Improvement Roadmap to guide the agency's ongoing quality-related work. The current roadmap is more a listing of projects and strategy options than a guide with priorities. Nonetheless, the QIOs are identified as having a major role in one of five major strategies: "assisting practitioners and providers in making care more effective, particularly in- cluding the use of effective electronic health systems" (CMS, 2005b:13). Given the commitment throughout CMS to improving the safety and qual- ity of care nationally, the QIO program could play an important part in carrying out this strategy.

OCR for page 55
ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 79 The committee concludes that thorough evaluations of the impact of the QIO program as a whole and of the effectiveness of specific quality improvement interventions, based on the articulation of clear and well- defined goals and priorities, would provide useful input to the long-range strategic planning process. Evaluations of QIO efforts in physician offices will be particularly important since the QIOs have had relatively little expe- rience in that provider setting, and there is little evidence of what approaches are most successful in stimulating major system changes. SUMMARY While there is evidence that the quality of care for Medicare beneficia- ries has been improving, the evidence concerning specific quality improve- ment efforts, the overall impact of individual QIOs, and the impact of the QIO program in the aggregate is limited and inconclusive. Nevertheless, the committee considers the QIO program to be a potentially rich resource as the health care system moves toward the increased use of performance mea- surement and pay for performance. The QIO program has shown consider- able adaptability over its 35-year history. It currently has a strong founda- tion of quality improvement experts in each state; a network of collaborative relationships with providers and stakeholders; and a national infrastructure to support data collection, reporting, aggregation, and auditing, as well as research to further the development and the use of quality measures. To make more effective use of these valuable resources, however, a major re- structuring of the program is needed. In Chapters 4 and 5, the committee presents its recommendations and rationales for these changes. First, how- ever, Chapter 3 presents a discussion of the changing health care environ- ment and new functions that will affect QIOs and other organizations con- ducting similar activities. It reviews the committee's first report and how implementation of that report's recommendations concerning the establish- ment of a national system for performance measurement and reporting will likely affect the QIO program. The chapter also examines other entities that might compete to perform the needed functions. REFERENCES Beck C, Richard H, Tu J, Pilote L. 2005. Administrative data feedback for effective cardiac treatment: AFFECT, a cluster randomized trial. Journal of the American Medical Asso- ciation 294(3):309317. Bradley EH, Carlson MDA, Gallo WT, Scinto J, Campbell MK, Krumholz HM. 2005. From adversary to partner: Have quality improvement organizations made the transition? Health Services Research 40(2):459476.

OCR for page 55
80 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Chu LA, Bratzler DW, Lewis RJ, Murray C, Moore L, Shook C, Weingarten SR. 2003. Im- proving the quality of care for patients with pneumonia in very small hospitals. Archives of Internal Medicine 163(3):326332. CMS (Centers for Medicare and Medicaid Services). 2002. 7th Statement of Work (SOW). [Online]. Available: http://www.cms.hhs.gov/qio [accessed April 9, 2005]. CMS. 2004. The Quality Improvement Organization Program: CMS Briefing for IOM Staff. [Online]. Available: http://www.medqic.org/dcs/ContentServer?cid=1105558772835& pagename=Medqic%2FMQGeneralPage%2FGeneralPageTemplate&c=MQGeneral Page [accessed December 26, 2005]. CMS. 2005a. Best Practice Methods Special Study: Report of First Year Scan of QIO Inpatient Practice. Unpublished. Baltimore, MD: Centers for Medicare and Medicaid Services. CMS. 2005b. Quality Improvement Roadmap. [Online]. Available: http://www. medicaldevices.org/public/issues/documents/CMSMedicareroadmap.pdf [accessed De- cember 26, 2005]. Daniel DM, Norman J, Davis C, Lee H, Hindmarsh MF, McCulloch DK, Wagner EH, Sugarman JR. 2004. A state-level application of the chronic illness breakthrough series: Results from two collaboratives on diabetes in Washington state. Joint Commission Jour- nal on Quality and Safety 30(2):6979. Dellinger EP, Hausmann SM, Bratzler DW, Johnson RM, Daniel DM, Bunt KM, Baumgardner GA, Sugarman JR. 2005. Hospitals collaborate to decrease surgical site infections. The American Journal of Surgery 190(1):915. Gaul GM. 2005, July 26. Once health regulators, now partners. Washington Post. p. A1. Gould BE, Grey MR, Huntington CG, Gruman C, Rosen JH, Storey E, Abrahamson L, Conaty AM, Curry L, Ferreira M, Harrington KL, Paturzo D, Van Hoof TJ. 2002. Improving patient care outcomes by teaching quality improvement to medical students in commu- nity-based practices. Academic Medicine 77(10):10111018. IOM (Institute of Medicine). 1994. An Assessment of the HCFA Evaluation Plan for the Medicare Peer Review Organization. Washington, DC: National Academy Press. IOM. 2000. To Err Is Human: Building a Safer Health System. Kohn LT, Corrigan JM, Donaldson MS, eds. Washington, DC: National Academy Press. IOM. 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. Wash- ington, DC: National Academy Press. IOM. 2002. Leadership by Example: Coordinating Government Roles in Improving Health Care Quality. Corrigan JM, Eden J, Smith BM, eds. Washington, DC: National Academy Press. IOM. 2004. Keeping Patients Safe: Transforming the Work Environment of Nurses. Page A, eds. Washington, DC: The National Academies Press. IOM. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. Jencks SF, Huff ED, Cuerdon T. 2003. Change in the quality of care delivered to Medicare beneficiaries, 19981999 to 20002001. Journal of the American Medical Association 289(3):305312. Jha AK, Li Z, Orav EJ, Epstein AM. 2005. Care in U.S. hospitals--the hospital quality alliance program. New England Journal of Medicine 353(3):265274. Jost TS. 2005. Racial and Ethnic Disparities in Medicare: What the Department of Health and Human Services and the Centers for Medicare and Medicaid Services Can, and Should Do. National Academy of Social Insurance Study Panel on Sharpening Medicare's Tools to Reduce Racial and Ethnic Disparities. Washington, DC: National Academy of Social Insurance. Kiefe CI, Allison JJ, Williams OD, Person SD, Weaver MT, Weissman NW. 2001. Improving quality improvement using achievable benchmarks for physician feedback: A randomized controlled trial. Journal of the American Medical Association 285(22):28712879.

OCR for page 55
ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 81 Leatherman S, McCarthy, D. 2005. Quality of Health Care for Medicare Beneficiaries: A Chartbook. New York: The Commonwealth Fund. Marciniak TA, Ellerbeck EF, Radford MJ, Kresowik TF, Gold JA, Krumholz HM, Kiefe CI, Allman RM, Vogel RA, Jencks SF. 1998. Improving the quality of care for Medicare patients with acute myocardial infarction: Results from the Cooperative Cardiovascular Project. Journal of the American Medical Association 279(17):13511357. National Health Policy Forum. 2004. Quality Improvement in Maryland: Partnerships and Progress. Washington, DC: National Health Policy Forum. NCQA (National Committee for Quality Assurance). 2005. The State of Health Care Quality Report: Industry Trends and Analysis. Washington, DC: National Committee for Qual- ity Assurance. NORC (National Organization for Research at the University of Chicago). 2004. Final Re- port: Physician Meetings on Take-Up of Electronic Health Records. Unpublished. Wash- ington, DC: National Organization for Research at the University of Chicago. Orlikoff JE, Totten MK. 2005. The pros and cons of board compensation. Does the promise justify the means? Healthcare Executive 20(1):4648. Qualis Health. 2004. HCQIP Improvement Methodologies Survey Results. Seattle, WA: Qualis Health. Rollow W. 2005. QIO Program: Update and Policy Considerations. PowerPoint Presentation to the Committee on Redesigning Health Insurance, June 13, Washington, DC. Westat. 2005. Survey for Provider Satisfaction with Quality Improvement Organizations. Unpublished. July 1. Rockville, MD: Westat.