6
Measuring Quality of Care

Morbidity and mortality from HIV disease declined dramatically in the mid-to-late 1990s due in large part to improvements in antiretroviral therapy and an increase in the proportion of individuals receiving therapy (Karon et al., 2001). The declines were unevenly distributed across HIV-infected populations, however, due to factors such as unequal access to care and variability in quality of care (Bozzette et al., 1998; Shapiro et al., 1999). A major focus of the Ryan White Comprehensive AIDS Resources Emergency (CARE) Act is not only to eliminate barriers to accessing care, but also to improve the quality of care that its clients receive.

Two major provisions of the 2000 CARE Act reauthorization emphasized aspects of quality of care. The first provision required that all Ryan White CARE Act (RWCA) grantees establish a quality-management program. Although Title III and IV grantees already had such programs in place, the new provision required Title I and II grantees to implement them.1 The purpose of the quality-management programs is to ensure that service providers adhere to established HIV clinical practices and Department of Health and Human Services (DHHS) guidelines to the extent possible; to ensure that quality-improvement strategies include support services that help people receive appropriate HIV health care

1  

Some Title I and II grantees already had quality-management programs in place prior to the reauthorization.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act 6 Measuring Quality of Care Morbidity and mortality from HIV disease declined dramatically in the mid-to-late 1990s due in large part to improvements in antiretroviral therapy and an increase in the proportion of individuals receiving therapy (Karon et al., 2001). The declines were unevenly distributed across HIV-infected populations, however, due to factors such as unequal access to care and variability in quality of care (Bozzette et al., 1998; Shapiro et al., 1999). A major focus of the Ryan White Comprehensive AIDS Resources Emergency (CARE) Act is not only to eliminate barriers to accessing care, but also to improve the quality of care that its clients receive. Two major provisions of the 2000 CARE Act reauthorization emphasized aspects of quality of care. The first provision required that all Ryan White CARE Act (RWCA) grantees establish a quality-management program. Although Title III and IV grantees already had such programs in place, the new provision required Title I and II grantees to implement them.1 The purpose of the quality-management programs is to ensure that service providers adhere to established HIV clinical practices and Department of Health and Human Services (DHHS) guidelines to the extent possible; to ensure that quality-improvement strategies include support services that help people receive appropriate HIV health care 1   Some Title I and II grantees already had quality-management programs in place prior to the reauthorization.

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act (e.g., transportation assistance, case management); and to ensure that demographic, clinical, and health care utilization information is used to monitor trends in the spectrum of HIV-related illnesses and the local epidemic (U.S. Congress, 2000a). The second provision in the reauthorization of RWCA directed the Institute of Medicine (IOM) to study two aspects of quality assessment: “The availability and utility of health outcomes measures and data for HIV primary care and support services and the extent to which those measures and data could be used to measure the quality of such funded services” (Ryan White CARE Act. 42 U.S.C. § 300ff-11 [2003]). “Other factors determined to be relevant to assessing an individual’s or community’s ability to gain and sustain access to quality HIV services” (Ryan White CARE Act. 42 U.S.C. § 300ff-11 [2003]). The focus on quality in the most recent reauthorization reflects several national trends in quality improvement and accountability. First, the health sector has seen growing interest in measuring and improving quality (IOM 2000, 2001a).2,3 In addition, numerous reports have identified a gap between care that should be provided based on available professional knowledge and technology and the care that patients actually receive (Advisory Commission on Consumer Protection and Quality in the Health Care Industry, 1988; IOM and NRC, 1999; IOM, 2000, 2001b). Significant disparities have also been identified in the quality of care for persons living with HIV disease (Cunningham et al., 1999; Shapiro et al., 1999; Stein et al., 2000). As a result, quality measurement and improvement efforts have become a central focus of the Health Resources and Services Administration HIV/AIDS Bureau (HRSA/HAB) and RWCA grantees. Another major trend in quality improvement and accountability is the fact that federal policy and funding decisions are increasingly being based on performance indicators. The passage of the Government Performance Results Act (GPRA)4 in 1993 and other legislation5 emphasized accountability across federal government sectors and programs by requiring all federal programs to establish measurable performance goals 2   Major groups with quality initiatives include the National Committee for Quality Assurance (NCQA), the Agency for Healthcare Research and Quality (AHRQ), the Foundation for Accountability (FACCT), the National Quality Forum, and the Leapfrog Group, among others. 3   For a more comprehensive review of quality-of-care activities in health care, see IOM, 2000, 2001a,b. 4   Government Performance Results Act., H.R. 826., 103d Cong., 1st Sess. 1993. 5   Federal Management Improvement Act of 1996 (P.L. 104-208); Chief Financial Officers Act of 1990.

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act and to document actual results that can be linked to the budgetary process (Kates et al., 2001). As a result, HRSA/HAB and RWCA grantees must document the impact of CARE Act funds on improving access to care and treatment and in addressing unmet needs. Key GPRA performance measures for HRSA regarding RWCA include eliminating barriers to care, eliminating health disparities, assuring quality of care, and improving public health and health care access. Progress on this broad agenda cannot be adequately measured by outcomes alone. Considering this context, the Committee interpreted this charge as a request to examine and make recommendations regarding the availability and usefulness of not only outcomes, but a broad continuum of measures for assessing and improving the quality of both primary care and support services funded through the CARE Act. Knowledge about what constitutes the best possible HIV/AIDS care is constantly changing, and the applicability of specific measures is limited by the evolving nature of the epidemic, including changes in the affected populations, the course and outcomes of infection, treatment standards, and policies and programs for financing and delivering care. Accordingly, while the Committee makes recommendations regarding measures that HRSA and grantees can use to monitor quality of care (see Recommendation 6-1), the Committee felt it could best contribute by providing a framework, criteria, and process for selecting measures that can evolve with the epidemic. The Committee divided its work into several discrete tasks: Reviewing current HRSA/HAB quality-of-care activities; Developing a conceptual framework for indicators that reflects the availability and quality of the full range of services across populations and organizations; Developing criteria for selecting indicators for assessing and improving HIV/AIDS services funded through RWCA; Reviewing quality measures used by RWCA grantees and other major HIV providers, recommended by authoritative groups, and contained in the research literature; Identifying areas that require further development; Developing a process HRSA might use to select and regularly update measures; and Offering recommendations based on the analysis and findings presented here. CURRENT HRSA/HAB QUALITY-OF-CARE ACTIVITIES HRSA/HAB and RWCA grantees have undertaken a variety of efforts to address the quality of care, treatment, and training in RWCA

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act programs.6 HAB has sponsored several major quality-of-care initiatives and provides a variety of technical assistance tools and training opportunities for grantees designed to help them implement quality-management programs targeting clinical, administrative, and supportive services. HAB’s quality initiatives focus on the service delivery system at various levels, including individual providers, Title I Eligible Metropolitan Areas (EMAs), and regions. Several of the initiatives have focused specifically on Title III/IV clinics and providers, although significant work is being done to implement quality-management programs through Title I and II. This section reviews the major quality initiatives of HRSA/HAB. A number of other quality efforts by HRSA/HAB, including special research projects and technical assistance guides, are presented in Table 6-1. HIV Quality of Care Project (HIVQUAL) Adapted from a model of quality improvement consultation developed by New York State, the HIVQUAL builds skills and capacity among Titles III and IV grantees to sustain quality improvement. The project is sponsored by HRSA/HAB and administered by the New York State Department of Health’s AIDS Institute. The project aims to improve the quality of care delivered to persons with HIV by using data to measure performance, tracking clinical indicators based on clinical practice guidelines, and providing consultations regarding Continuous Quality Improvement. The HIVQUAL project collects information on a number of quality indicators, which are discussed later in this chapter. Grantees submit collected data to the HIVQUAL Project, where they are compiled into aggregate reports and comparative tables for participants and HRSA. A software program (HIVQUAL 3) is used to facilitate the measurement of quality indicators. Participation among grantees is encouraged, but not required (HRSA, 2003c). Institute for Health Care Improvement HIV/AIDS Collaborative The Institute for Healthcare Improvement (IHI) HIV/AIDS Collaborative was an effort to use rapid improvement strategies to accelerate the pace of quality improvement among CARE Act grantees. IHI focused on developing a systemwide model to ensure the delivery of evidence-based clinical care and provide strong support for self-management. The long-range goal of the IHI Collaborative was to maximize the length and qual- 6   See McKinney and Marconi (2002) for a review of CARE Act–funded research on quality of care.

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act TABLE 6-1 Other HRSA/HAB Quality-of-Care Activities HRSA/HAB Quality Initiative Description Outcomes Evaluation Tools HRSA/HAB has developed several outcome evaluation guides that provide a framework for outcomes evaluation and sample measures (HRSA, 2003f): The Outcomes Evaluation Technical Assistance Guide: Primary Medical Care Outcomes (2000) includes suggested outcomes indicators for primary medical care. The Outcomes Evaluation Technical Assistance Guide for Case Management Services (2001) includes suggested outcomes indicators for case management and information on validated quality-of-life instruments. Outcomes Evaluation: Getting Started (2001). HRSA Care Action: The Resource Gap. Measuring Success: Evaluation, Outcomes, and Quality of HIV Care. Special Projects of National Significance HRSA/HAB has sponsored a number of Special Projects of National Significance (SPNS) that have examined access, quality of care, and outcomes (HRSA, 2003d). Some projects include: Assessing Existing Efforts to Increase Adherence to Medications (1999). Evaluating the Impact of IT on Improving Delivery and Quality of Care for HIV Seropositive Individuals (2002). Improving HIV Quality of Care Cooperative Agreement (1999). For more information on SPNS projects see: http://hab.hrsa.gov/special/evaluation2g.htm. HIV/AIDS Evaluation Monograph Series The monograph series of publications was created by HRSA/HAB to assist RWCA grantees in designing and implementing evaluation studies (HRSA, 2003c). Example publications include: Using Data to Assess HIV/AIDS Service Needs: A Guide for Ryan White CARE Act Planning Groups, Report #2. An Approach to Evaluating HAART Utilization & Outcomes in CARE Act–Funded Clinics, Report #5. Delivering HIV Services to Vulnerable Populations: What Have We Learned, Report #6. HRSA Center for HIV Quality Care The HRSA Center for Quality Care focuses on defining appropriate standards of care for clients of HRSA programs, determining the cost of that care, and conducting research on and comparing benefit packages and capitation rates in Medicaid managed care plans. The Center is also documenting community-based provider experience with managed care plans (HRSA, 2003a).

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act ity of life for patients with HIV/AIDS and satisfy patient and caregiver needs while maintaining or decreasing the total cost of care. Each participating health system identified a specific population of patients (either a subset of their patients or all of them) that could be monitored for the duration of the collaborative. Grantees then changed practices and systems to improve clinical management and office efficiency. Participants identified a set of quality-of-care indicators (discussed later in this chapter), while IHI provided guidance about how to implement and test changes in the care of their pilot population. Over 80 Title III and Title IV grantee organizations participated in the first collaborative, and Title I grantees are currently participating in a new collaborative (HRSA, 2003b; IHI, 2003). Client Demonstration Project The Client Demonstration Project collects client data on HIV-infected clients (tracked by unique identifiers) receiving services at CARE Act–funded providers at designated sites (EMAs or states). The client-level data permit analyses of various combinations of information on client demographics, medical status, and service utilization. The analysis of several characteristics, including CD4+ cell counts, CDC-defined disease stage, and tuberculosis status, is also possible. Though the project was originally designed to include clients who receive services from Title I and II providers, a substantial number of providers participating in the demonstration projects also receive funding from Title III and IV (HRSA, 2001, 2003a). Primary Care Assessment Tool The Primary Care Assessment Tool (PCAT) is a site-visit protocol developed for Title III–funded CARE Act programs. The tool, which includes a quality improvement component in the clinical section, is used to evaluate the clinical, fiscal, administrative, and support services of the grantees (HRSA, 2003e). CARE Act Data Report Each year all providers funded by CARE Act grantees are required to submit a CARE Act Data Report (CADR) to HRSA. Implemented in January 2002, CADR consolidated and replaced previous reporting instruments. The purpose of the CADR is to collect information on all clients who receive at least one CARE Act–eligible service during a calendar year, including information on the number and characteristics of clients

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act served, types of services provided, and characteristics of provider agencies. CADR also requires medical service providers to provide information on the number of clients who have been screened or treated for comorbid conditions, diagnosed with an AIDS-defining condition, and prescribed combination antiretroviral therapy. An unduplicated count of patients is not currently possible in CADR, so clients may be reported more than one time. CAREWare is a free software program that assists grantees in producing the CADR. Data from CADR are used to assess program effectiveness and quality under the GPRA (HRSA, 2002). Other HRSA Quality Initiatives HRSA also supports a number of other quality-of-care activities, including outcome evaluation tools, several Special Projects of National Significance, and the HIV/AIDS Evaluation Monograph Series (Table 6-1). FOUR-DIMENSIONAL CONCEPTUAL FRAMEWORK FOR HIV QUALITY ASSESSMENT In this section, the Committee presents a conceptual framework for assessing the quality of HIV/AIDS care. In developing its framework, the Committee relied on previous work for definitions and theory of patient-level quality, and coupled this with a public health perspective in extending previous work to include quality of care for populations. The Committee adopted the definition of health care quality first developed by the IOM in 1990. This report defined health care quality as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge” (IOM, 1990). This definition highlights several important aspects of quality (IOM, 2001a). First, numerous persons, providers, and organizations provide care to patients and all must be of good quality. Second, the definition acknowledges that good quality health care should produce outcomes that patients desire and that patients’ preference for treatment options may vary. Third, high-quality care does not necessarily lead to desirable outcomes. There are many factors beyond the control of providers or organizations, such as environmental or social factors, that can affect outcomes. The corollary is also true; desirable outcomes may occur despite poor quality of care. Finally, it speaks to the obligation of providers to inform themselves of most recent advances in their discipline (IOM, 2001a). The Committee also relied heavily on the seminal work of Avedis Donabedian who conceptualized quality along three dimensions: struc-

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act ture (“the settings in which [health care] takes place and the instrumentalities of which it is the product”), process (“whether what is known as ‘good’ medical care has been applied”) and outcomes (“in terms of recovery, restoration of function and of survival”) (Donabedian, 1966). The Committee also based its framework on a model developed by Holzemer and Reilly (1995) that extends the work of Donabedian by focusing on the interactions and linkages between structure, process, and outcomes at the client, provider, and the care setting levels. The Committee conceptualizes HIV quality assessment as having four key dimensions: the population of interest, the level assessed, the type of measures employed, and the spectrum of services to be evaluated (Figure 6-1). FIGURE 6-1 Dimensions of quality assessment.

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act Population of Interest The HIV-infected population consists of three groups with respect to use of services: (I) those who are not yet diagnosed and thus are not in care, (II) those who are diagnosed but are not currently in care,7 and (III) those who are diagnosed and are receiving care. These categories represent a dynamic continuum. Changes in HIV prevalence and the effectiveness of screening efforts affect the flow of persons from group I to groups II and III. Shifts between groups II and III can be rapid and frequent as individuals move in and out of care owing to changes in insurance coverage or as the need for hospital or other urgent care related to an HIV-related comorbid condition arises. When assessing care quality, the CARE Act has traditionally focused on individuals who are diagnosed and currently in care. Grantees have long sought to improve continuity of care for persons who drop out of care or receive only episodic care, and there has recently been an increased emphasis on those who are diagnosed but not yet in care. To date, few CARE Act resources have been directed to those who are not yet diagnosed. However, providers, consumers, public health officials, and Congress are focusing more attention on ensuring access to high-quality services for all who are HIV-infected (U.S. Congress, 2000b). Among those with HIV disease, many of those not receiving care were once in care and lost access or withdrew; it is generally agreed that a high proportion of these individuals would benefit from a return to regular care. A similar argument applies to those who know of their infection but have never sought or have been unable to gain access to care, as they may well have advanced disease amenable to chemotherapy. Those with early disease will benefit from monitoring and counseling regarding their general health before their disease progresses, and indications for antiretroviral therapy could be extended to this group in the future. In addition, society benefits from efforts to prevent transmission from all currently infected individuals. Thus, knowing the relative numbers of persons in groups I, II, and III gives direct information on the overall effectiveness of the health care system in providing access to needed care. Knowing the proportions of infected persons who are undiagnosed across areas allows inference regarding access to screening and case-finding services, and about the attractance associated with the perceived high quality of care in an area. Knowing the proportions of persons who are diagnosed but not in care across areas allows inference regarding the accessibility of existing care facilities and the ability of those facilities to maintain continuity of care for treated persons. 7   The category of those who are “not in care” also includes those who are not in regular care.

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act Level of Assessment The fundamental level for assessing quality of care is the individual patient. For example, if a clinician were assessing whether his or her patients were receiving high-quality care, it would be important to know whether patients for whom high active antiretroviral therapy (HAART) is indicated were offered such therapy. Such individual levels of quality also aggregate to the program level. For example, a measure of the quality of care provided by Title III clinics is the proportion of appropriate patients who are receiving HAART therapy. Provider organizations, including HRSA and its grantees, now commonly employ a variety of aggregated client-level measures of treatment quality in assessing clinics or practices, including monitoring, chemoprophylaxis, and antiretroviral therapy.8 However, the scientific community has given little attention to assessing the quality of care across whole communities, metropolitan areas, or states, regardless of whether patients get care at any particular facility.9 The measures and data available to assess quality vary markedly according to both the target population and the level of the assessment. For example, in assessing the quality of and access to services at the clinic level, one might assess the average wait time for an initial appointment or the use of appropriate chemoprophylaxis. While analysts have much less experience in measuring quality at the population level, they might assess access and chemoprophylaxis by measuring the average time elapsed between HIV and AIDS diagnoses. In addition, as indicated above, one might measure the overall accessibility of care by measuring the proportion of HIV-infected people who are in care, those who are diagnosed but not in care, and those who are not diagnosed. Type of Measure The Committee adopted the widely accepted taxonomy that classifies measures into three domains (Donabedian, 1966). Structural measures assess the characteristics and resources of an organization, such as staffing, supplies, equipment, and training. The influence of physical surroundings, availability of services, and staffing mix on the quality and costs of care is poorly understood. However, an abundance of literature has demonstrated a link between the experience of individual and institu- 8   While the data for these indicators of the quality of medical care are collected at the patient level, they are generally aggregated to make assessments at the provider, clinic, and program level. 9   HRSA, however, has recently initiated some work focused on developing quality measures at the EMA level.

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act tional providers (in terms of the volume and complexity of HIV services they offer) and improved clinical and economic outcomes (Kitahata et al., 1996; Turner et al., 1998; Aiken et al., 2002; Landon et al., 2002, 2003). Process measures focus on the actions taken in the course of providing medical care, such as conducting diagnostic tests, offering appropriate therapies, promoting self-care. Process measures are important because they “cause” outcomes to the extent that medical interventions have an effect and because the acts of providing care are convenient targets for quality improvement. Although information for process measures is usually obtained from medical records, analysts can also obtain important information by asking patients about their experiences (Cleary and Edgman-Levitan, 1997; Cleary, 1999; Wilson et al., 2002). Quality assessment often aims primarily at improving processes as the key to improving outcomes because they are easier to track and modify than outcomes. Outcome measures assess the changes in a patient’s health status. Intermediate outcome measures may assess patients’ viral load or CD4+ cell count, while longer-term outcome measures might examine morbidity, mortality, and health-related quality of life. These three major elements are interrelated and theoretically causally linked; that is, the structure of care affects the probability of good processes of care, which can in turn affect outcomes such as health status, quality of life, and patient satisfaction. Despite compelling arguments for analyzing outcomes as ultimate indicators of quality, there are practical and theoretical impediments to doing so, at least for the individual patient and the small (in a statistical sense) aggregations of patients often used to assess providers. First, individual patient outcomes are often not closely linked to quality of care offered or provided. A broad range of factors, many of which providers do not control, determines clients’ health outcomes. Accounting for these influences is extremely difficult (Brook et al., 1996). Strong links between care and outcomes are more likely when considering a large patient group that is well defined by medical condition or demographics, when interventions target outcomes, and when well-accepted physiological, biochemical, and psychological mechanisms link medical interventions with outcomes. When assessing the quality of specific clinics or programs, it is therefore important to ensure that the outcome measures are primarily affected by the program being assessed rather than some characteristic of the subpopulation being served, and that there is evidence for the quality of the actions and programs at that clinic. For this reason, process measures are often used in lieu of outcome measures at the program or clinic level, while monitoring outcomes is a simpler and more compelling way of assessing the impact of all care provided in an area.

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act care, mental health care, and substance abuse treatment. Any subject who answered yes to any of these questions was asked if those needs were met. Health Outcomes Outcome measures used are classified into three major groups: CD4+ cell counts, viral load counts, and emergency room visits and hospitalizations. Two sources measured CD4+ cell counts and viral load levels, and one source each measured patients’ emergency room visits and hospitalizations. Population-Level Quality-of-Care Measures Most quality assessment is designed to improve service quality and patient care, and thus focuses on the provider level, which by definition covers only people receiving treatment at a defined set of facilities or enrolled in a specific HMO. However, population-based measures are important in assessing the overall impact of RWCA in a region. Although several potential measures have been developed, research in this area has been limited. One recent study utilized a Pneumocystis carinii pneumonia (PCP) index, defined by zip code as the number of PCP-related hospitalizations for residents divided by the number of residents living with AIDS), to identify geographic areas within New York City where residents were at increased risk for PCP (Arno et al., 2002). The PCP index is an example of a population-based measure that can be used to identify areas in need of public health interventions and improvements in HIV service delivery. This measure has the added advantage of being easy to generate and requiring only a few data sources (surveillance and service utilization data). In a separate analysis, the Committee examined another potential population-level measure—the 12-month case fatality rate—which measures the proportion of people who die within 12 months after they are diagnosed with HIV infection. Figures 6-6 and 6-7 show a general decline in the 12-month case fatality rate after the introduction of HAART in the mid-1990s before the measure levels off. One notable aspect of these graphs is that they show two- and threefold variations in fatality rates among EMAs, and several EMAs show rising rates. Another potential population-based quality measure is the proportion of patients in an area who have an AIDS diagnosis within 12 months of their HIV diagnosis. This measure can be calculated using data from the HIV and AIDS reporting system. Currently, these data are only avail-

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act FIGURE 6-6 Percent of people who died within 12 months of AIDS diagnosis, by state/territory, 1994–2000. FIGURE 6-7 Percent of people who died within 12 months of AIDS diagnosis, by EMA, 1994–2000.

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act able for states with name-based HIV reporting in place as CDC does not accept code-based data. A recent report by the CDC highlights other potential measures that could be applied at the population level (CDC, 2003a). Using data from four supplemental surveillance studies12 conducted in selected geographic areas, CDC reports on a number of measures including: HIV-infected persons ever prescribed ART; HIV-infected persons currently prescribed ART; HIV-infected persons ever diagnosed with PCP; HIV-infected persons currently prescribed prophylaxis for PCP; and HIV-infected persons receiving a tuberculin skin test. Such measures would be difficult to obtain routinely from all relevant jurisdictions because they require estimates of the number of persons in a defined area who are infected and in care. However, a combination of intensive, targeted surveys, the current population surveys, the Area Resource File, and regional and local data could be used to estimate these populations. While population measures are critical in monitoring the overall health of the population in a region, population-based outcomes and other measures may be influenced by a number of factors. Kates and colleagues (2001) refer to this as the “problem of attribution,” that is the difficulty of ascribing client-level or local-level outcomes to a state or federal program. They note several challenges to the effective measurement of health programs: “Variation in health systems infrastructure, market dynamics, epidemiology, and demographics makes it difficult to connect federal level activities to effects at the local level. In addition, because clients of federal programs may receive services at multiple locations, each with multiple funding sources, it is difficult to align a federal program activity (or grant dollar) with a client level outcome such as reduction in morbidity and mortality. Indeed, individual client characteristics and varying levels of disease severity may require sophisticated case mix adjustment to make comparisons meaningful” (Kates et al., 2001, p. 147). 12   Data in the report are from (1) the Adult/Adolescent Spectrum of HIV Disease (ASD) project, a longitudinal medical record review study conducted in >100 selected facilities in 11 major U.S. cities; (2) the Survey of HIV Disease and Care (SHDC), a cross-sectional, population-based medical record review project conducted in 3 areas; (3) the Supplement to HIV/AIDS Surveillance (SHAS) project, an interview project of persons with HIV/AIDS conducted by 12 state/local health departments; and (4) the Missed Opportunities for Tuberculosis Prevention study, a medical record review of a population-based sample of persons newly diagnosed with HIV or AIDS in three major U.S. cities (CDC, 2003a). CDC has announced plans to discontinue two of these projects, ASD and SHAS, beginning in mid-2004. CDC is currently in the process of developing a new Morbidity Monitoring System that will use interview and chart data and will allow collection of HIV/AIDS data from a representative population sample (CDC, 2003c).

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act FINDINGS HRSA and RWCA-funded clinics and programs are doing an admirable job of defining, assessing, and attempting to improve the quality of care received by HIV-infected individuals. RWCA-funded clinics have established quality management and improvement programs, and states and EMAs either have or are establishing such programs. HRSA has sponsored numerous research and demonstration projects and a national quality-improvement collaborative to enable RWCA clients to receive the best-possible care. In many ways, RWCA-funded programs have a more advanced approach to measuring and improving quality of care than most general medical facilities. Yet, HRSA, RWCA grantees, and providers could still do much more to measure and improve quality of care. Current efforts are not guided by a common conceptual framework and measures are often not standardized. This is understandable given the recent development of quality-management programs, particularly under Titles I and II. From a programmatic point of view, the primary limitation is that the measures are not defined and measured in a comparable manner across jurisdictions. Thus, it would be very hard for HRSA to assess the relative quality of care across clinics, EMAs, or states, using the data currently available. A more organized and coherent approach would be beneficial. Based on its review of selected quality-of-care measure sets in the previous section, the Committee recommends a standardized set of structure, process, and outcome quality measures for assessing and facilitating quality improvement at the clinic level and at the EMA/state levels (see Recommendation 6-1). For the well-developed process measures of clinical care, measures were selected if at least four of the nine groups included them. Other measures were identified using the expert judgment of the Committee. A critical step in improving the current situation of uncoordinated data collection would be reaching a consensus on how different indicators should be defined and implemented and then putting into place a mechanism for ensuring the quality of the ensuing data. Although there are many ways of doing this, the NCQA and the National Quality Forum have developed procedures and policies that could be emulated or modified. For example, NCQA has technical advisory groups that provide advice on the importance, scientific soundness, and feasibility of possible measures, and it maintains a standing Committee on Performance Measurement that evaluates recommendations from technical experts and staff. The Committee on Performance Measurement includes representatives from the Office of Personnel Management, a state Medical Association, corporate health care purchasers, health care systems, business coa-

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act litions, professional organizations, the American Association of Health Plans, patient advocacy groups, insurance companies, a state department of human services, researchers, Centers for Medicare & Medicaid Services (CMS), CDC, the Veterans Administration, and the AHRQ. A comparable set of procedures could be established by HRSA. Finding 6-1 The RWCA legislation’s emphasis on development of outcome measures is appropriate because tracking outcomes is a crucial element for accountability, and helpful in quality improvement. However, outcome measures alone are not sufficient because outcomes may be influenced by many factors not under the control of grantees, and because structure and process measures can uniquely identify areas for specific improvement. Finding 6-2 Quality measures for HIV/AIDS that are in wide use are based on broadly accepted clinical guidelines and are appropriate. However, these measures are not standardized, which has hindered efficient evaluation of RWCA programs. Moreover, these measures are generally restricted to evaluating providers via examination of aggregated patient-level clinical data. Measures examining a more complete spectrum of services including, for example, those assessing support services, are important but are not yet in wide enough use to allow for consensus to emerge about specific measures for HIV care. Examples of potential measures of supportive services include the provision of case management, benefits advocacy, or substance abuse and mental health services. Finding 6-3 Measures of access to needed medical and nonmedical services are also lacking. Such measures have been developed for other diseases (e.g., cancer and heart disease) and could be adapted to HIV/AIDS. Examples of potential access measures include those that focus on continuity of care (e.g., identification of primary care provider), access to needed specialty care, and access to needed nonmedical care. Finding 6-4 Population-based measures are essential in monitoring HIV care in an EMA, region, or state and identifying areas for improvement. Current efforts to assess overall quality of care in EMAs and states are rudimentary and no population-based measures are in wide use. These measures should not be interpreted as direct measures of the quality of care being provided by specific clinics, but rather as reflecting the cumulative effects of many influences on case quality and outcomes. Additional work is needed to further develop these measures.

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act RECOMMENDATIONS The Committee’s recommendations are an effort to support and extend the excellent work of HRSA and RWCA grantees in quality assessment and improvement. Recommendation 6-1 Quality measures: HRSA should adopt quality measures that are comprehensive with respect to populations (diagnosed in care; diagnosed but not in care; not diagnosed, not in care), level of assessment (provider and population levels), types of measures (structure, process, and outcome), and spectrum of services (clinical and supportive services). At a minimum, HRSA and grantees should strongly consider inclusion of the following standard set of measures to assess the quality of care provided by RWCA-funded providers and EMA and state-level programs. Standard definitions and detailed criteria for these measures need to be developed by HRSA in collaboration with grantees, affected communities, and other stakeholders after a rigorous examination of the importance, scientific soundness, and feasibility of potential measures. I. CLINIC LEVEL MEASURES A. Structure 1. Proportion of providers with appropriate expertise and experience in treating patients with HIV*   2. Availability of case management services B. Process Screening for: 3. Cervical cancer* 4. Hepatitis B   a. And administration of hepatitis B vaccine if negative 5. Hepatitis C* 6. Syphilis* 7. Toxoplasmosis 8. Tuberculosis* Performance of the following clinical monitoring tests: 9. CD4+ cell count and HIV viral load (process)* Antiretroviral Treatment: 10. Provision of indicated antiretroviral treatment* 11. Provision of adherence counseling and monitoring Vaccinations and Prophylaxis for Opportunistic Infections: 12. Influenza vaccination 13. Pneumoccocal pneumonia vaccination 14. Hepatitis B vaccination if patient not immune 15. If indicated, Mycobacterium avium complex (MAC) 16. If indicated, Pneumocystis carinii pneumonia* (PCP)

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act C. Outcomes 17. Monitoring of CD4+ cell count and viral load values* 18. Proportion of patients with a history of an AIDS-defining opportunistic condition* 19. Proportion of patients who have >2 missed scheduled clinic appointments per year 20. Hospitalizations and emergency room visits without hospital admission* 21. Proportion of patients with unmet need for support services (e.g., assistance with obtaining housing).^ II. AREA (E.G., EMA OR STATE) LEVEL MEASURES   Outcome measures related to access and care: 22. Proportion of HIV-infected persons in an area who are not diagnosed^ 23. Proportion of diagnosed persons in area who are receiving regular care^ 24. Proportion of people with HIV who died within 12 months of an HIV diagnosis* 25. Proportion of people with HIV who progressed to AIDS within 12 months of initial HIV diagnosis* Key: * indicates high priority; ^ indicates high priority, but may require further development. Other measures are important, but of lower priority. Recommendation 6-2 Infrastructure development: The Secretary of Health and Human Services (HHS) should provide additional resources to HRSA and CDC to develop infrastructure for monitoring quality at the patient, clinic, and population levels. This infrastructure development strategy has three major components: HRSA should enhance support for information technology and personnel to enable clinics to collect, aggregate, and report a focused set of clinical and patient-reported data. HRSA should collaborate with CDC and other agencies to develop innovative population-based measures that can be captured using existing data sources or other community-based information-gathering activities, such as surveys of unmet needs. Congress should enhance flexibility in the administrative caps at the grantee level to promote infrastructure development. Recommendation 6-3 Collaborative quality activities: The Secretary of HHS should convene a working group, not restricted to, but including the NCQA, state insurance commissioners, state Medicaid officials, and representatives from HRSA, CDC, the CMS, providers of community, outpatient, and inpatient care, and members of the

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act relevant research communities, to consider strategies for promoting greater collaboration between public health departments and public-and private-sector providers in order establish tools and methods to assess systems of care and quality, building on the successful collaborative models developed by the CDC for immunization (e.g., Clinic Assessment Software Application and Assessment Feedback Incentives Exchange). REFERENCES Advisory Commission on Consumer Protection and Quality in the Health Care Industry. 1998. Quality First: Better Health Care for All Americans. Washington, DC: U.S. Government Printing Office. Aiken L, Clarke S, Sloane D, Sochalski J, Silber J. 2002. Hospital nurse staffing and patient mortality, burse burnout, and job dissatisfaction. Journal of the American Medical Association 288(16):1987–93. Arno PS, Gourevitch MN, Drucker E, Fang J, Goldberg C, Memmott M, Bonuck K, Deb N, Schoenbaum E. 2002. Analysis of a population-based Pneumocystis carinii pneumonia index as an outcome measure of access and quality of care for the treatment of HIV disease. American Journal of Public Health 92(3):395–8. Asch SM, Kerr EA, Hamilton EG, Reifel JL, McGlynn EA, Eds. 2000. Quality of Care for Oncologic Conditions and HIV: A Review of the Literature and Quality Indicators. Santa Monica, CA: Rand Health. Ball JK, Turner BJ. 1991. AIDS in U.S. Hospitals in 1986-1987: A National Perspective. Rockville, MD: Agency for Health Care Policy and Research. Bennett CL, Garfinkle JB, Greenfield S, Draper D, Rogers W, Mathews C, Kanouse DE. 1989. The relation between hospital experience and in-hospital mortality for patients with AIDS-related PCP. Journal of the American Medical Association 261:2975–9. Bozzette SA, Berry SH, Duan N, Frankel MR, Leibowitz AA, Lefkowitz D, Emmons CA, Senterfitt JW, Berk ML, Morton SC, Shapiro MF. 1998. The care of HIV-infected adults in the United States. HIV Cost and Services Utilization Study Consortium. New England Journal of Medicine 339(26):1897–904. Bozzette SA, Phillips B, Asch S, Gifford AL, Lenert L, Menke T, Ortiz E, Owens D, Deyton L. 2000. Quality Enhancement Research Initiative for Human Immunodeficiency Virus/Acquired Immunodeficiency Syndrome: Framework and Plan. Medical Care 38(6 Suppl 1):160–9. Brook RH, McGlynn EA, Cleary PD. 1996. Quality of health care. Part 2: Measuring quality of care. New England Journal of Medicine 335(13):966–70. CDC (Centers for Disease Control and Prevention). 2003a. Selected measures of access to and utilization of treatment and prophylaxis for HIV-infected persons. HIV/AIDS Surveillance Supplemental Report. 9(3). CDC. 2003b. IOM Request: Data on Early Progression and Early Deaths. (Email commuication P. Sweeney, CDC, March 20, 2003).

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act CDC. 2003c. Morbidity Monitoring System. (Email communication, Patricia Sweeney, October 16, 2003). Cleary PD. 1999. The increasing importance of patient surveys. British Medical Journal 319:720–1. Cleary PD, Edgman-Levitan S. 1997. Health care quality: Incorporating consumer perspectives. Journal of the American Medical Association 278(19):1608–12. Cunningham W, Andersen R, Katz M, Stein M, Turner B, Crystal S, Zierler S, Kuromiya K, Morton S, St Clair P, Bozzette S, Shapiro M. 1999. The impact of competing subsistence needs and barriers on access to medical care for persons with human immunodeficiency virus receiving care in the United States. Medical Care 37(12):1270–81. Donabedian A. 1966. Evaluating the quality of medical care. The Milbank Memorial Fund Quarterly 44(3 Supp):166–206. Gross PA, Asch S, Kitahata M, Freedberg KA, Barr D, Melnick DA, Bozzette S. 2000. Performance measures for guidelines on preventing opportunistic infections in patients infected with human immunodeficiency virus. Clinical Infectious Diseases 30(Suppl 1):S85–S93. Holzemer W, Reilly CA. 1995. Variables, variability, and variations research: Implications for medical informatics. Journal of the American Medical Informatics Association 2(3):183–9. HRSA (Health Resources and Services Administration). 2001. The Ryan White CARE Act. [Online]. Available: http://hab.hrsa.gov/data2/cdemosum.htm [accessed July 31, 2003]. HRSA. 2002. Instruction for completing the Ryan White CARE Act data report (CARE Act Data Report): Cross—Title Data Report. January, 2002. [Online]. Available: http://www.careactdatasupport.hrsa.gov/CADRforms/instruct.pdf [accessed July 31, 2003]. HRSA. 2003a. Report and Studies: Client Demonstration Projects. [Online]. Available: http://hab.hrsa.gov/data2/cdemosum.htm [accessed July 31, 2003]. HRSA. 2003b. Special Initiatives: HIV/AIDS Breakthrough Series Collaborative. [Online]. Available: http://hab.hrsa.gov/special/breakthrough.htm [accessed June 30, 2003]. HRSA. 2003c. Special Initiatives: HIV/QUAL Continuous Quality Program. [Online]. Available: http://hab.hrsa/gov/special/hivqual.htm [accessed June 30, 2003]. HRSA. 2003d. Special Initiatives: Initiatives/Collaboratives (by topic). [Online]. Available: http://www.hab.hrsa.gov/special/evaluation2g.htm [accessed July 31, 2003]. HRSA. 2003e. Tools for Grantees: Primary Care Assessment Tool (PCAT). [Online]. Available: http://hab.hrsa.gov/tools/pcat.htm [accessed July 31, 2003]. HRSA. 2003f. Outcomes Evaluation Technical Assistance Guide: Primary Medical Care Outcomes, Titles I and II of the Ryan White CARE Act. [Online]. Available: ftp://ftp.hrsa.gov/hab/guide.pdf [accessed July 31, 2003].

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act IHI (Institute for Healthcare Improvement). 2003. Improving Care for People Infected with HIV. [Online]. Available: http://www.ihi.org/collaboratives/breakthroughseries/HIV/charter.asp [accessed July 31, 2003]. IOM (Institute of Medicine). 1990. Medicare: A Strategy for Quality Assurance. Vol. 2. Lohr K, Ed. Washington, DC: National Academy Press. IOM. 2000. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press. IOM. 2001a. Envisioning the National Health Care Quality Report. Hurtado MP, Swift EK, Corrigan JM, Eds. Washington, DC: National Academy Press. IOM. 2001b. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press. IOM and NRC (Institute of Medicine and National Research Council). 1999. Ensuring Quality Cancer Care. Hewitt M, Simone JV, Eds. Washington, DC: National Academy Press. Karon JM, Fleming PL, Steketee RW, De Cock KM. 2001. HIV in the United States at the turn of the century: An epidemic in transition. American Journal of Public Health 91(7):1060–8. Kates K, Marconi K, Mannie TE. 2001. Developing a performance management system for a federal public health program: The Ryan White CARE ACT Titles 1 and 2. Evaluation and Programs Planning 24:145–55. Katz M, Cunningham W, Mor V, Andersen R, Kellogg T, Zierler S, Crystal S, Stein M, Cylar K, Bozzette S, Shapiro M. 2000. Prevalence and predictors of unmet need for supportive services among HIV-infected persons: impact of case management. Medical Care 38(1):58–69. Katz M, Cunningham W, Fleishman J, Andersen R, Kellogg T, Bozzette S, Shapiro M. 2001. Effect of case management on unmet needs and utilization of medical care and medications among HIV-infected persons. Annals of Internal Medicine 135(8 Pt 1):557–65. Kitahata MM, Koepsell TD, Deyo RA, Maxwell CL, Dodge WT, Wagner EH. 1996. Physicians’ experience with the acquired immunodeficiency syndrome as a factor in patients’ survival. New England Journal of Medicine 334(11):701–6. Landon BE, Wilson IB, Wenger NS, Cohn SE, Fichtenbaum CJ, Bozzette SA, Shapiro MF, Cleary PD. 2002. Specialty training and specialization among physicians who treat HIV/AIDS in the United States. Journal of General Internal Medicine 17(1):12–22. Landon BE, Wilson IB, Cohn SE, Fichtenbaum CJ, Wong MD, Wenger NS, Bozzette SA, Shapiro MF, Cleary PD. 2003. Physician specialization and antiretroviral therapy for HIV: Adoption and use in a national probability sample of persons infected with HIV. Journal of General Internal Medicine 18(4):233–241. McKinney M, Marconi K. 2002. Delivering HIV services to vulnerable populations: A review of CARE act-funded services. Public Health Reports 117:99–113.

OCR for page 188
Measuring What Matters: Allocation, Planning, and Quality Assessment for the Ryan White CARE Act Shapiro MF, Morton SC, McCaffrey DF, Senterfitt JW, Fleishman JA, Perlman JF, Athey LA, Keesey JW, Goldman DP, Berry SH, Bozzette SA. 1999. Variations in the care of HIV-infected adults in the United States: Results from the HIV Cost and Services Utilization Study. Journal of the American Medical Association 281(24):2305–15. Stein MD, Crystal S, Cunningham WE, Ananthanarayanan A, Andersen RM, Turner BJ, Zierler S, Morton S, Katz MH, Bozzette SA, Shapiro MF, Schuster MA. 2000. Delays in seeking HIV care due to competing caregiver responsibilities. American Journal of Public Health 90(7):1138–40. Turner B, Markson L, Cocroft J, Cosler L, Hauck WW. 1998. Clinic HIV-focused features and prevention of Pneumocystis carinii pneumonia. Journal of General Internal Medicine 13(1):16–23. UHC (University Health System Consortium). 2002. HIV Ambulatory Care Clinical Benchmarking Executive Summary. 12. U.S. Congress. 2000a. Congressional Record 146(123):S10032. U.S. Congress, House of Representatives. Committee on Commerce. 2000b. Report on Ryan White CARE Act Amendments of 2000 (106-788). 106th Cong., 2d Sess. July 25, 2000. VA (Veterans Health Administration). 2003. HIV Quality Enhancement Research Initiative (HIV-QUERI). [Online]. Available: http://www.va.gov/chrr/active_trials/queri_factsheetHIV.pdf [accessed July 31, 2003]. Wilson IB, Ding L, Hays RD, Shapiro MF, Bozzette SA, Cleary PD. 2002. HIV patients’ experiences with inpatient and outpatient care: Results of a national survey. Medical Care 40(12):1149–60. Wu A, Gifford A. 1998. Quality of Care Indicators for HIV/AIDS: A Discussion Paper for the Foundation for Accountability. [Online]. Available: http://www.facct.org/facct/doclibFiles/documentFile_302.pdf [accessed July 31, 2003]. Wu AW, Gifford A, Asch S, Cohn SE, Bozzette SA, Yurk R. 2000. Quality-of-care indicators for HIV/AIDS. Disease Management and Health Outcomes 7(6):315–30. Yeni P, Hammer SM, Carpenter CC, Cooper DA, Fischl MA, Gatell JM, Gazzard BG, Hirsch MS, Jacobsen DM, Katzenstein DA, Montaner JS, Richman DD, Saag MS, Schechter M, Schooley RT, Thompson MA, Vella S, Volberding PA. 2002. Antiretroviral treatment for adult HIV infection in 2002: Updated recommendations of the International AIDS Society-USA Panel. Journal of the American Medical Association 288(2):222–35.