Click for next page ( 83


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 82
3 Performance Measurement, Quality Improvement, and Other Entities CHAPTER SUMMARY This chapter summarizes the key points of the committee's first report, Performance Measurement: Accelerating Improvement and reviews the functions of the performance measurement and report- ing system proposed therein. It also examines alternative organiza- tional structures for quality improvement and potential roles for the QIO program and other capable organizations in government and the private sector. The legislative mandate for the Institute of Medicine's (IOM) evalua- tion of the Quality Improvement Organization (QIO) program requested an overview of the program and an assessment of the program's impact (see the introduction to this report). The committee assumes that another ques- tion is implicit in that mandate: On the basis of the QIO program's past performance, what should its role be in the future? The committee believes examining a program's past is of particular value in considering changes that should be made in the future. It should also be noted that the com- mittee's judgments about future roles for the QIO program are based on certain expectations regarding the changing health care environment. Those expectations derive from the committee's work on the two other studies in the IOM's Redesigning Health Insurance project--on performance mea- surement and payment incentives (also described in the introduction to this report). The first section of this chapter summarizes the case for an orga- nized performance measurement and reporting system, along with the rec- ommendations made in the committee's first report, Performance Measure- ment: Accelerating Improvement (IOM, 2006). The second section examines the potential roles for the QIO program and other organizations in the new health care environment of the future. The committee's report on payment incentives will be published in 2006. 82

OCR for page 82
PERFORMANCE MEASUREMENT 83 THE NEED FOR AN ORGANIZED PERFORMANCE MEASUREMENT AND REPORTING SYSTEM The introduction to this report and the Performance Measurement re- port document serious problems with the quality of health care in the United States: substantial variations in provider performance among both indi- vidual and institutional providers; substantial variations in provider perfor- mance by the patient's geographic location, race or ethnicity, and insurance status; high error rates; and high and rising costs. The gap remains wide at present between the level of performance called for in the IOM's Quality Chasm report (IOM, 2001) and envisioned by the committee and the care that is provided (IOM, 2001). A congressional mandate separate from that for the present study, Section 238 of the Medicare Prescription Drug, Im- provement, and Modernization Act of 2003 (P.L. 108-173), requested that the IOM consider how the application of two key strategies within the health care environment--public reporting and payment incentives--can be used to promote quality improvement. Both mandates were assigned to this committee. These two strategies, together with quality improvement initia- tives, can influence the way care is delivered. The committee believes these strategies have the potential not only to improve the quality and increase the value of care delivery, but also to improve overall health outcomes. Implementation of these strategies will require the use of performance mea- sures. Indeed, a performance measurement system can provide support and guidance for the formulation of national priorities and goals and for quality improvement strategies to achieve those goals. Moreover, a common per- formance measurement infrastructure is needed to support the efforts of private and public insurance plans to realign incentives. Many performance measure sets are currently used for quality improve- ment in the health care delivery system, creating unnecessary burdens on providers and confusion among consumers and purchasers. A multitude of organizations have created their own quality measures for purposes specific to their particular needs, leading to an uncoordinated proliferation of mea- sures. Those efforts have contributed to a "nonsystem" of performance measurement. In cataloging more than 800 measures, the committee found many of the measure sets to be duplicative, with slight differences in the specific details of the numerator or the denominator. Public reporting meth- ods also vary greatly, with little evidence on how best to present the data to the public and on what impact the reports have on health care delivery. Efforts have been made to harmonize the various measures. For ex- ample, those used in the QIO program are generally consistent with some of the common measure sets, and the Centers for Medicare and Medicaid Services (CMS) has been collaborating with the relevant associations to eliminate the slight differences in specifications among measures used for

OCR for page 82
84 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM various purposes in the Medicare program. Measurement remains incom- plete, however, failing to address important aspects of the health care sys- tem. For example, measures of efficiency, equity, and patient-centeredness are less developed than measures of effectiveness. Processes of care are mea- sured more frequently than structures and outcomes. Also, the measures used cover a disparate set of clinical conditions and populations. As a re- sult, current measures are not robust enough to support the strategies of performance measurement and reporting and payment incentives. Regardless of one's political persuasion--whether one believes the health care system should be primarily publicly run, governed by the free market, or governed by a mix of the two approaches--it is generally recog- nized that coherent and comprehensive performance measurement and re- porting are fundamental to achieving better-quality care and represent a public good. Such a system should ultimately include data from all payers, all patients, and all providers. Strong national leadership is needed to create such a public good; it has not evolved spontaneously and is unlikely to develop without effective public guidance. The committee believes an organized system is necessary to align all current measurement efforts and to accelerate the diffusion and pace of performance measurement and reporting. Many issues innate to the mea- surement of performance--such as risk adjustment to account for differ- ences in patient populations and the severity of cases from one provider to another and adjustment for patient compliance--would be difficult to ad- dress adequately outside of such an organized system. Because performance measurement and reporting are a public good and would serve a broad and diverse audience, investment from society in general should be considered. In sum, the committee believes the chaos that characterizes the current nonsystem inhibits the rate of improvement in health care quality, and that strong federal leadership is necessary to create a system for measurement and reporting that can achieve the vision of the Quality Chasm series of reports. A National System for Performance Measurement and Reporting In the Performance Measurement report, the committee recommends the establishment of a new, independent governing board, the National Quality Coordination Board (NQCB), to be housed as an independent en- tity within DHHS. This recommendation reflects and builds upon a history of earlier efforts to establish a national performance measurement and pub- lic reporting system, including recommendations of the Strategic Frame- work Board and the Advisory Council for Health Care Quality. (Chapter 2 of Performance Measurement reviews in detail previous attempts in the

OCR for page 82
PERFORMANCE MEASUREMENT 85 United States and other countries to establish such a system.) The commit- tee envisions the NQCB as working with and not supplanting existing stake- holder groups from both the public and the private sectors to ensure that all of the following functions of a high-performing system for measurement and reporting are carried out: Specification of a purpose and aims for American health care Establishment and prioritization of short- and long-term national goals Designation or development and promulgation of standardized measures A guarantee of adequate data collection, validation, and aggregation Establishment of public reporting methods responsive to stake- holder needs Identification of a research agenda for the development of new measures Evaluation of the impact of the overall system The work of the NQCB will need to be grounded in sound, scientific evidence. The board should be given adequate funding and contract author- ity, should have structural independence, and should be free of undue influence from special interests in completing its tasks. It should also have standards-setting authority and external accountability. Its members should have substantive expertise, should have experience gained in the field, and should be able to guide existing organizations with ongoing efforts that address its goals and functions. The collaboration of stakeholders will con- tribute to an organized national system that, although not a solution in and of itself, has great potential to accelerate improvements in health care qual- ity for the entire health care delivery system. The NQCB as envisioned by the committee will harmonize and refocus measurement and reporting efforts for three main purposes: accountability, quality improvement, and population health. With respect to accountabil- ity, the committee believes performance measurement and reporting should support patients' decisions in choosing providers, as well as give purchasers information to aid in the selection of providers and the development of health insurance networks. Quality oversight organizations should be able to use these data in their accreditation and certification activities. Perfor- mance data should also support decisions on the quality improvement in- terventions to be undertaken in provider settings. In addition, the data and analyses of the NQCB should inform stakeholders about how well the sys- tem is functioning in addressing population health issues, such as access to health care, disparities in health care, and health promotion.

OCR for page 82
86 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Key Functions of the NQCB Certain functions essential to the new system, such as the definition of goals and priorities, should be carried out by the NQCB itself at the na- tional level; other functions can be performed by various organizations and agencies, with national coordination provided by the board. Specification of the board's purpose and aims is the first step in creating the new system (see Figure 3.1). This should be done at the national level, along with the establishment and prioritization of national goals for the delivery of quality health care. Without a set of priorities, efforts will not be targeted effi- ciently to the areas of greatest need. On the other hand, the development of national priorities and goals for the health care delivery system, similar to those developed in the Healthy People series of reports in the public health system, is not meant to stifle local innovation. In fact, the committee recom- mends that innovation and achievement of local priorities in pursuit of the national goals be encouraged, recognizing that such local efforts can help improve current approaches to quality measurement. The promulgation of standardized measures built upon the measures currently in use or under development is another key function of a high- performing measurement and reporting system. The NQCB could begin by endorsing a minimum set of such measures for use by all providers in am- bulatory care, acute care, health care plans, and long-term care settings; for the treatment of end-stage renal disease; and for longitudinal measure- ment of health outcomes and health care costs for a given condition or patient. To this end, the committee identified a starter set of measures drawn from current leading measure sets developed or used by such pro- grams as the End Stage Renal Disease program in Medicare; the Leapfrog Group; the Agency for Healthcare Research and Quality's National Qual- ity and Disparities Reports; the Ambulatory care Quality Alliance; the National Committee for Quality Assurance's Health Plan Employer Data and Information Set (HEDIS); CMS's Nursing Home Compare and Home Health Compare; the Hospital Quality Alliance; and the Consumer Assess- ment of Healthcare Providers and Systems' (CAPHS) health plan, hospital, and ambulatory surveys of consumers' views on health care they received (see Table A.3 in Appendix A). The measures in this starter set will need to be defined with consistent specifications and detailed elements. They will also need to be updated periodically to reflect advances in performance measurement. The continual evaluation of the measurement and reporting system, including feedback from providers and users of the data, will con- tribute to the updating and refinement of the measures. As noted above, the committee's review of existing performance mea- sures revealed both duplication and gaps. The gaps included important as- pects of the health care system not adequately measured (limited scope), a

OCR for page 82
PERFORMANCE MEASUREMENT 87 Purpose To continuously reduce the impact and burden of illness, injury, and disability, and to improve the health and functioning of the people of the United States Aims Effective, safe, timely, patient-centered, efficient, and equitable care Establishment of Goals Promulgation of Standardized Measures Data Collection and Aggregation Public Reporting Accountability Quality Improvement Population Health Pay for performance Health care organizations Access to services Quality oversight Patients and clinicians Health behaviors Professional certification Disease surveillance Were the six aims achieved? Research Agenda Impact Assessment Intended consequences Unintended consequences FIGURE 3.1 Functions of a national system for performance measurement and reporting. SOURCE: Performance Measurement: Accelerating Improvement, adapted from Strategic Framework Board (McGlynn, 2003). time window too narrow for the assessment of health care delivery, a provider-centric rather than a patient-centric focus, and a narrow focus of accountability (see Table 3.1). To address these gaps, the committee identified four approaches for improving performance measurement and reporting:

OCR for page 82
88 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Comprehensive measurement--Through comprehensive perfor- mance measurement, aims and conditions not being adequately measured will become apparent. Longitudinal measurement--Longitudinal measurement examines the quality of services as patients move through the delivery system over time and across settings of care. For example, longitudinal measures would be necessary to follow a patient through an episode of care or for 30 days after hospital discharge. Individual-level, population-based, and systems-level meas- urements--Individual-level, population-based, and systems-level measure- ments assess how well the delivery system is providing care to individuals and populations (by geographic location, race or ethnicity, or some other designation). Shared accountability--All providers treating a patient should share responsibility for the patient's health outcomes. TABLE 3.1 Gaps in Current Performance Measure Sets Relevant Design Gap Principles Description Limited scope of measurement: Principle 1: A performance measurement Few measures of patient-centered Comprehensive system should advance the core care, equity, or efficiency. Few measurement purpose of the health care system measures for children or those at and foster improvements in all six the end of life. Many important aims identified in the Quality conditions unrepresented in Chasm report (IOM, 2001): safety, measures. effectiveness, patient-centeredness, timeliness, efficiency, and equity. Narrow time window: Most Principle 3: Standardized performance measures focus on a single point in Longitudinal measures should characterize time and do not assess care across measurement health and health care both within settings. and across settings and over time. A provider-centric focus: Current Principle 7: Measurement and measures should measures focus on existing silos of Individual assess the health and health care of care (e.g., physician's office, patient-level, both individuals and populations hospital). population-based, and the many systems within and systems-level which care is provided. measurement Narrow focus of accountability: Principle 8: Measurement should not be Most measures focus on an Shared constrained by the absence of a individual provider's actions. accountability current, identifiable, single responsible agent. SOURCE: IOM (2006).

OCR for page 82
PERFORMANCE MEASUREMENT 89 To the extent that measures are useful at all levels of the health care system and relevant to all stakeholders, reporting is likely to be less burden- some to providers, and improvement efforts are more likely to be success- ful. Also, performance measurement will be more efficient and timely if data used to calculate measures are, to the extent feasible, generated in real time and as a byproduct of patient care processes rather than collected retrospectively. While real-time data collection and reporting can be done either electronically or manually, it will require changes in procedures for many providers. The committee recommends that the performance mea- surement and reporting system be implemented rapidly, but without over- whelming providers. Technical support to providers will be essential. Performance measurement and reporting require various related func- tions, such as data collection, validation, and aggregation. The committee does not specify in detail the activities to be conducted or coordinated by the NQCB. Many of these activities are ongoing under the direction or operation of various public and private organizations across the country, including the QIO program. The committee recognizes the importance of current stakeholders in the new system that will be coordinated through the NQCB. The committee also recognizes that its focus has been on roles for CMS in the performance measurement and reporting system. This is the case because the Medicare program has implemented a public reporting system, is moving ahead with the development of payment incentives, and is such an important purchaser of services; because the studies' mandates relate to the federal program; and because CMS sponsored the studies car- ried out by the committee. Although the committee does not define specific roles for CMS and its relevant programs under the new system, leaving the formulation of such details and their implementation to the NQCB, it does anticipate a strong collaborative role for CMS given its expertise and cur- rent responsibilities. The discussion of various activities and organizations that follows in this chapter is meant to be illustrative, not to provide a work plan for the new board. The committee foresees increased requirements for providers to report on performance measures in the future because of calls for greater account- ability. These requirements will place an increased burden on providers even if all reporting requirements are aligned. Providers will need assistance in minimizing this reporting burden. The validation and aggregation of data will require substantial investments. The NQCB will need to determine the optimum means of carrying out these functions and the organizations best suited to performing them, whether at the local or national level. A robust system for the development, maintenance, collection, validation, and aggre- gation of performance measures will facilitate reporting for purposes of improving the quality of health care.

OCR for page 82
90 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM The continued advancement of performance measurement and report- ing will depend on a research agenda. The NQCB, working with stakehold- ers, should develop an agenda for addressing the gaps noted above, as well as other systemic needs. The formulation of such an agenda will require work in four specific areas: development of new measures, resolution of methodological issues, determination of the best methods of reporting so the data can be used by consumers, and assessment of the overall impact of the system. It will be essential as well to monitor the system for any unin- tended, negative consequences, such as adverse selection and inappropriate data manipulation. The committee realizes that the establishment of a functioning NQCB and a national measurement and reporting system and the achievement of the board's performance goals will not be accomplished overnight. Never- theless, it is essential to the health care system and to the health of the U.S. population that such a national system be established. During the transi- tion, while the NQCB is being established and a performance measurement and reporting system is being implemented, it will be especially important for CMS to play a strong leadership role in coordinating the goals and measures of its programs with those developing nationally within the new system and to support the necessary infrastructure. FUNCTIONS OF THE NATIONAL QUALITY COORDINATION BOARD AND IMPLICATIONS FOR QIOS This section reviews the various functions likely to be carried out in some fashion by the NQCB and the national system for performance mea- surement and reporting it establishes, potential contributions of the QIO program, and possible roles for other organizational options and other en- tities. The discussion here is based on the committee's recommendations in its first report, but goes further to encompass quality improvement efforts as well. In the Performance Measurement report, the committee acknowledges many private and public organizations that have played significant roles in the development and use of performance measures. Chapter 2 and Appen- dix B of that report provide details on those organizations that are not repeated here. Many of these organizations have been working with the QIOs in developing measures, reporting on the CMS Compare websites, and providing technical assistance. The committee expects that many of these organizations will continue to perform their current functions, but under the coordination of the NQCB. If, however, any of these organiza- tions could not or did not wish to carry out the functions required for the national system, the NQCB would have the authority to contract for those services. There may also be other services required for the national system

OCR for page 82
PERFORMANCE MEASUREMENT 91 for which additional contracts would be necessary. To the extent that con- tracting under the NQCB was done on a competitive basis, organizations holding QIO contracts, as well as other entities discussed in this chapter, could have an expanded role to play. After strongly endorsing an ongoing role for CMS in the performance measurement and public reporting system, the committee considered whe- ther CMS should have a continuing role in quality improvement, as dis- cussed in Chapter 2. Other possible locations within the federal govern- ment could house the quality improvement program, such as the Agency for Healthcare Research and Quality, which has contributed significantly to research on quality measures and improved clinical pathways; the Veterans Health Administration, which has made dramatic improvements in care within its own system; and the NQCB, which will be directing quality mea- surement and reporting activities. Moving responsibility for the quality improvement program from CMS to another federal agency offers some advantages. These include opportuni- ties for CMS to concentrate on measurement and payment issues and to pursue a strong regulatory approach, when such an approach appears nec- essary, without fear of jeopardizing providers' willingness to participate in the quality improvement program. Moreover, other federal agencies might better manage the program by integrating it with their ongoing quality im- provement activities. On the other hand, the disadvantages of moving quality improvement efforts from CMS to another site also deserve consideration. The first is CMS's loss of the QIO apportionment, which supports other quality- related projects. Also lost would be the opportunity to achieve closer coor- dination among the offices responsible for public reporting, conditions of participation, Medicaid, the State Children's Health Insurance Program, Medicare payment, and quality improvement. The QIO program has worked closely in the past with the offices tasked with measure develop- ment and management of conditions of participation to ensure a consistent approach and to synchronize the details of the measures so as to minimize the burden on providers. Under the NQCB and the new performance mea- surement and reporting system, those functions will become even more im- portant, and the QIO program's value will increase to the extent that it is able to coordinate its measurement work. Some QIOs have worked with Medicaid but at the state level, primarily on data analysis and case review, rather than on technical assistance to Medicaid providers. They serve as External Quality Review Organizations to state Medicaid programs, re- viewing care provided by managed care organizations paid on a capitation basis by the state; survey consumers of that care; and perform special analy- ses of data collected by the state program. The potential of the QIO program's coordination with Medicaid within CMS has yet to be realized.

OCR for page 82
92 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM In the end, the committee decided it has serious reservations about moving the quality improvement program to another federal agency, for several reasons: The experience of the Veterans Health Administration with quality improvement is unique, based on its own management control over provid- ers, and is not readily replicable in the private sector. The Agency for Healthcare Research and Quality has had little expe- rience with running an operational program of the scale of the QIO pro- gram nationwide. Moving the quality improvement function would diminish opportu- nities for coordination within CMS, including Medicaid, and with other quality-related activities. It would also stifle the progress made through the strong efforts of the current administrator, as well as the rejuvenation of the administrator's Quality Council staffed by the Quality Coordination Team (see Chapter 13), and diminish the potential support and guidance of the NQCB. The next issue the committee addressed was whether it would be better to allow the NQCB to manage the QIO program directly. The advantages of that arrangement would be the potential for close synchronization of QIO and NQCB priorities; the opportunity for the NQCB to have an op- erational arm that would reach every state; and the explicit expansion of QIO activities to cover the entire population, not just Medicare beneficia- ries. In addition, the NQCB could help strengthen relationships and coordi- nate activities between public- and private-sector entities. Yet the commit- tee chose not to pursue this option, for the following reasons: As recommended in the Performance Measurement report, the NQCB should not become a large federal entity assuming responsibility for the operation of all facets of the performance measurement and re- porting system. Rather, the NQCB should be an independent entity that would rely, to the extent possible, upon existing stakeholder organiza- tions, such as CMS, to perform specific functions. The focus and most immediate priority of the NQCB should be on performance measurement and reporting; assuming other duties for quality improvement would require additional roles and resources that could over- whelm the board's formative efforts. However, the extent to which the NQCB should and could be involved in quality improvement activities in the future is yet to be determined. The NQCB has not yet been established, so its assumption of the management of an ongoing federal program would be premature.

OCR for page 82
PERFORMANCE MEASUREMENT 93 As noted above, a number of organizations have made important con- tributions to the development and use of performance measures. Working with these other organizations is necessary to advance the state of quality improvement, as the functions of performance measurement, reporting, and quality improvement are clearly interrelated. Coordination between the NQCB and the QIO program should therefore be explored to the extent that it can advance the goals and priorities of all these organizations. For example, such coordination could lead to more direct links between private-sector and QIO efforts. The NQCB will be working closely with private organizations--for example, the Institute for Healthcare Improve- ment, the Institute for Clinical Systems Integration, and such provider groups as the Physician Consortium for Performance Improvement and na- tional specialty societies--to guide the development of performance mea- sures. Combined with the emergence of policy levers such as pay for perfor- mance, coordination between the NQCB and the QIOs could link the QIOs more directly to these private-sector efforts to hold providers responsible for the quality of care they provide. Important relationships such as these tend not to develop by themselves. This type of coordination between the NQCB and the QIO program offers a unique opportunity that the commit- tee believes should be explored. The committee recognizes that the functions ultimately to be assigned to CMS by the NQCB, as well as those currently under CMS's purview through the QIO program, could be conducted by CMS directly through its own staff or, alternatively, through contracts with other private technical assistance organizations. For the QIO program, such direct operations would, of course, presume an increase in CMS's hiring ceiling and all the related administrative changes necessary to hire additional staff. The size of the agency would dramatically increase. Such an arrangement could offer several advantages: Given that program policy is currently set by federal staff, this ar- rangement would allow management to follow the entire operational pro- cess to ensure implementation of that policy. The federal agency would have more direct control over all activities in each state and over each employee. Communications among employees across the United States and be- tween staff in each state and the policy directors and managers in the Cen- tral Office might be easier to arrange. Most of the QIO contracting functions could be eliminated. However, the disadvantages of CMS running the QIO program through its own staff are more compelling:

OCR for page 82
94 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Hiring practices and salaries under the federal personnel system cre- ate more constraints than are faced by private organizations and could make it difficult for CMS to hire people with the necessary expertise and experience. The federal personnel system is relatively inflexible and does not respond as quickly as can private companies to changing program priorities and personnel needs. CMS has chosen to manage many other important Medicare func- tions through contracts with intermediaries, carriers, information technol- ogy vendors, and other private organizations, and is accustomed to various contracting arrangements. Thus, the committee supports CMS's continuing to operate the QIO pro- gram through contractors rather than operating the program directly through federal staff. The committee then considered whether the QIO program should con- tinue with its current contractors, many of which have held QIO core con- tracts for multiple scopes of work (SOWs), or other entities should become eligible bidders. The committee believes the bidding process should be opened up for several reasons: The new SOW, based on the committee's recommendations, will be significantly different from past SOWs. Some current organizations holding state QIO contracts may excel at case review, but may be less successful at providing technical assistance for quality improvement, the focus of the next SOW. Some organizations holding QIO state contracts that have excellent technical assistance programs may want to expand to neighboring states where regionally organized activities would be appropriate. Some other entities may be better positioned to perform the tasks of the QIO core contract than are current QIO contractors. The same rationale generally applies to other contracts in the QIO pro- gram, such as contracts for QIO Support Centers (QIOSCs), special studies, operation of the data system, and general program support. It should be noted that the committee's discussion of this issue relates to that portion of the legislative mandate for the QIO study in Section 109 of the Medicare Prescription Drug, Improvement, and Modernization Act of 2003 (P.L. 108-173(d)(1)(D)) which calls for a review of "the extent to which other entities could perform such quality improvement functions as well as, or better than, quality improvement organizations." By definition, only those organizations that meet the requirements to compete for QIO core contracts do so, and bidding for QIOSC contracts and special studies

OCR for page 82
PERFORMANCE MEASUREMENT 95 is open only to organizations that hold a QIO contract. In the few instances in which there has been competition for a QIO contract, it has generally come from an out-of-state QIO. No other entities (private businesses or government agencies) are performing the same range of functions carried out by the QIOs under the SOW for the core contract with Medicare or meet the structural and organizational requirements to become a QIO. Therefore, no direct evidence exists as to how and how well these other organizations might perform those same functions, and it is impossible to judge whether in the future, some other entities might do a better job than the QIOs. Thus, the committee interpreted this request from the Congress to mean an examination of other organizations that could possibly assume the functions now performed by QIOs. The QIO program has functioned as a closed system of contractors with relatively little change from one SOW to the next. The committee's recommendations in Chapters 4 and 5 are designed to open up the contract- ing process in the QIO program to more competition and to encourage participation by new and different organizations. There should be opportu- nities for competition not only for the core SOW for each state, but also for QIOSC contracts, special studies, and support contracts that might be used to promote quality improvement and sustain functions for the NQCB. CMS should take advantage of existing federal contracting mechanisms to select the best bidders efficiently. Thus, the committee envisions the QIO program as providing QIO core services in each state through contracts with the best-qualified organization bidding on each state's contract. Given the committee's recommendations in the following two chapters that include eliminating QIO requirements for physician access or sponsorship, changing board representation, and limiting QIO functions, other entities in addition to organizations currently serving as QIOs might be expected to be well positioned to bid on the core contracts. Of course, the anticipated funding level of future QIO contracts will also affect the amount of competition. Rather than limit federal contracting to what the QIO program has done in the past, the committee considers here hypothetical and potential contracting options for CMS and, possibly, the NQCB that would allow them to offer technical assistance to providers for performance measure- ment, quality improvement, and various other services to support the na- tional system for performance measurement and reporting. As mentioned above, the present discussion of activities is at a general level because the precise functions of the NQCB and the national system it will coordinate have yet to be detailed and established. Nevertheless, many of the salient activities or their elements are now under way, although they are not occur- ring in a coordinated fashion. For the purposes of this discussion, the com- mittee assumes that the NQCB, CMS, and other stakeholders will define

OCR for page 82
96 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM the specific functions to be performed under the contract, invite bids for contracts to perform the necessary activities, and choose from among the various bidders. In this context, collaboratives and other quality improve- ment methods, whether conducted by QIOs or other organizations, often have not been rigorously examined; the body of peer-reviewed evidence on these efforts is small, and it shows mixed effectiveness. The committee expects there will be an important need for technical support to providers for quality improvement in each state. Many of the other functions related to the national performance measurement and re- porting system, however, may be conducted more efficiently at the regional or national level. Of the general tasks discussed below, some should prob- ably be carried out in each state and should therefore be part of the QIO core contract. Both organizations that hold existing QIO contracts and other entities might choose to bid on future QIO contracts. Contracts for other functions not part of the core contract in each state likewise might attract bids from some QIOs and other entities. Finally, certain func- tions might appropriately be funded by the QIO program from its Medi- care apportionment at the national level. A significant portion of the ap- portionment for the QIO program has traditionally been spent on projects and activities related only tangentially to the QIOs and could continue to be used to support functions related to quality improvement under the guidance of the NQCB. The following are examples of functions that could support the national system for performance measurement and reporting: Data collection--All types of providers nationwide need assistance with the collection and reporting of performance measures. Because perfor- mance data come from medical charts, administrative claims, registries, and electronic health records, and because providers have different levels of so- phistication in the use of those types of records, assistance will be necessary to ensure that the data reported are uniform. The QIOs have worked with providers in various health care settings through improvement collabora- tive interventions and other technical assistance efforts. QIOs also have helped hospitals responding to the reporting efforts of the Hospital Quality Alliance to collect accurate data efficiently. The collection of uniform data is integral to any quality improvement effort, and technical assistance will be needed in each state to promote the reporting of measures, particularly by those providers ill equipped to staff such services internally or unable to purchase them. Therefore, the provision of such assistance should remain part of the QIOs' core contract. Even so, either in addition to or instead of QIO assistance, many providers rely on other entities, including private businesses, consultants, and professional associations, for assistance with data collection and reporting (see Table B.1 in Appendix B).

OCR for page 82
PERFORMANCE MEASUREMENT 97 Data validation--After data have been reported and aggregated into performance measures, it would be useful to review the original data with each provider to ensure that they validly reflect what is intended, and that all providers understand how to interpret and compare their own perfor- mance with overall provider performance as determined through the aggre- gated data. It would also be useful to offer the provider suggestions for improvement. Although this would be a natural core task for the QIOs, it is beyond the current scope of the QIO program to reach every provider. Other organizations having good working relationships with providers, such as state and national professional societies and private vendors of data ser- vices, would also be able to assist the NQCB with data validation. Other such entities, for example, HealthShare and THA, a subsidiary of the Texas Hospital Association, might bid on this data validation task in their own states if the NQCB decided to let state-level contracts for measure valida- tion (see Table B.1 in Appendix B). Data aggregation--The NQCB will need data aggregated at the na- tional level and at various other geographic levels, as well as down to the patient level, for analysis and reporting purposes. Whether one or more data repositories will be needed has yet to be determined. Currently, the QIO program supports a private data repository that serves the needs of the QIOs as well as those of the Hospital Quality Alliance's public reporting program (see Chapter 11 for more detail on public reporting of data). The QIO program also supports secure data communications systems that can be used for data transmission. Some private organizations, such as Medstat and the RAND Corporation, also have large data repositories and data aggregation capabilities (see Table B.1 in Appendix B). Even if the NQCB decided that data aggregation functions should be a public responsibility, it might choose to contract for some of the related activities, either through the QIO program or with one or more of the private entities profiled in Appendix B. Data auditing--The accuracy of performance data will become in- creasingly important as more providers participate in a coordinated report- ing system and as financial rewards and public recognition are based on the data collected. Although the QIOs have conducted case reviews in the past and are capable of data auditing, the data-auditing function is inconsistent with their role of providing technical assistance to promote quality improve- ment and with their collaborative relations with the provider community. The committee believes regulatory functions such as data auditing should not be handled by the QIOs, but should be consolidated and conducted at a level higher than the state. A QIO might thus choose to bid on a contract covering a different region without jeopardizing its own provider relations. Other organizations are performing such functions as well.

OCR for page 82
98 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Public reporting--The NQCB's reputation as an independent public body will be important when it issues national reports on the findings ob- tained from the evaluation of performance measures. Ideally, the measures in the National Healthcare Quality Report prepared by the Agency for Healthcare Research and Quality would be congruent with those of the NQCB, and the functions and goals of the two reports would be coordi- nated or integrated. The QIOs have experience with tailoring performance data to the needs of their states. They might fruitfully help consumers inter- pret and make use of the national reports of individual provider perfor- mance data to inform their choices of providers and treatments, and help providers identify general areas of care that may be in need of improve- ment. Undoubtedly, many professional societies and other entities will also use the public reports for their own members and clients. It may be noted that for providers to focus on specific, actionable internal efforts, other, more detailed and preferably real-time data will be necessary in addition to the publicly reported global data. Improvement of performance--Once baseline performance measures in various domains are available, the committee believes CMS should ex- pect all providers participating in the Medicare program to improve the quality of their services. The committee recognizes that many providers do not have the internal resources (staff expertise and funds) to make the nec- essary changes for quality improvement, but if they are given free technical assistance, they can make greater progress. This has been an important role for the QIOs, particularly during the last decade, and in the future should become their primary focus so they can handle the expected demand from those providers requesting assistance. Many providers also rely on internal quality improvement staff and other organizations to assist with quality improvement efforts, and are more likely to use those resources once public reporting and payment incentives become more common. Promotion of population health--In their core contracts, the QIOs have had responsibility for promoting some activities at the provider level to improve the health of populations, and this responsibility continues in the 8th SOW. In the context of the QIO program, specific activities include a focus on minorities and underserved populations. QIOs have also been charged in the 7th and 8th SOWs to increase rates of mammography and immunization and decrease rates of tobacco use, measures that help provid- ers promote public health (see Chapter 8). QIOs tend to collaborate with many state and local stakeholders in specific campaigns. Although other sections of CMS are involved with educating beneficiaries directly, resources from the QIO program at the national level could be directed toward the support of national, state, and local campaigns if the NQCB were to decide that such support was necessary to achieve the goals it has delineated.

OCR for page 82
PERFORMANCE MEASUREMENT 99 Evaluation of the system--The NQCB will need feedback on how the performance measurement and reporting system is working, when unin- tended consequences have been detected, when problems with the collec- tion and submission of data by providers are detected, or when there are difficulties with the measures themselves. Such feedback is necessary for continuous improvement of the system. The QIOs, along with the resources available in each state, are a good source of this information, and this func- tion could be built into the QIO core contract. Other evaluations of the system that might target operations in a sample of states could include par- ticipation by the relevant QIOs as necessary. The committee identified companies currently working on functions related to those listed above for various clients and gleaned relevant infor- mation from the organizations' websites (see Table B.1 in Appendix B). These companies are merely examples, but they do indicate the level and quantity of experience potentially available to CMS and, ultimately, the NQCB. In addition, several university health policy and research centers and professional associations have relevant capabilities. It is unlikely that any of these organizations have had experience with the full range of func- tions in the current QIO core contract, but some may offer unique and relevant areas of expertise. One organization suggested as an obvious "other entity" that could operate the QIO program is the Institute for Healthcare Improvement. How- ever, the institute's business model does not appear to be suited to the ex- panded growth necessary for this organization to provide direct services within each state; rather, its management prefers to work through the QIOs to extend the organization's impact. Similar limitations may apply to other organizations described in Table B.1 of Appendix B. Collaboratives and other quality improvement efforts, whether conducted by QIOs or other organizations, have often not been rigorously examined; the body of peer- reviewed evidence is small, and it shows mixed effectiveness. It was not feasible within the scope of this study to conduct assessments of the performance of the various organizations listed in Table B.1. In fact, because the activities of these other entities are so varied, it would be virtu- ally impossible to compare or rate them. It was also impossible to compare the performance of these organizations with that of the QIOs since none have performed the same set of functions on a similar scale. Thus the com- mittee does not endorse any of these organizations and can make no judg- ment about their expertise or suitability for specific tasks. The NQCB and CMS would have to use competitive contracting processes to choose the organization--a QIO or some other entity--that appeared to be most ap- propriate for a particular task. The committee cannot predict which organi-

OCR for page 82
100 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM zations among those listed in Table B.1 might bid or which would be most appropriate for a given task. Once the recommendations in Chapters 4 and 5 have been implemented and competition for QIO core contracts is no longer limited in practice to existing QIOs, there will be opportunities for these and other organizations to compete for contracts to provide QIO core services and other services related to the performance measurement and reporting system. It will be up to CMS and the NQCB to define the specific tasks to be accomplished, ensuring that the tasks in the QIO core contract promote the goals of the national performance measurement and reporting system, and to determine how to assess the bids and bidders. When evaluating proposals, CMS might consider the following criteria: The value of preexisting relationships with providers and other stake- holder organizations Relevant experience The technical expertise and demonstrated capacity of the bid- der's staff Staff turnover The creativity of the bidder's proposal The feasibility of the proposal Evaluations or references from current and recent clients The financial health of the organization The composition of the governing board and its processes for ensur- ing accountability To some extent, the amount of competition for a contract will depend upon how CMS and the NQCB decide to structure the tasks in the contract--what needs to be done on the ground in each state, what func- tions can be performed regionally or nationally, and how CMS and the NQCB will organize their management and oversight functions. CMS and the NQCB will need to consider the complexities of contract management and how to streamline the number of contracts and contractors that could result from restructured tasks. The more narrowly defined functions are, the more likely it is that there will be increased competition involving new entities. On the basis of the committee's perception of the history of the QIO program, its findings and conclusions about the program's current opera- tions, and its view of a future health care system that includes a national performance measurement and reporting system and performance incen- tives, recommendations for the QIO program are offered in the next two chapters. Chapter 4 focuses on the QIOs' key tasks, and Chapter 5 on how

OCR for page 82
PERFORMANCE MEASUREMENT 101 the QIOs' structure can be strengthened and how CMS's oversight and management capabilities can be improved. REFERENCES IOM (Institute of Medicine). 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press. IOM. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press.