Child and Adolescent Health and Health Care Quality: Measuring What Matters (2011)

D


Overview of Data Sources for Measures of Health Care Quality for Children and Adolescents

This appendix reviews sources of data on the quality of health care services for children and adolescents, including both administrative data sources (claims or claims and encounters) and population health surveys.

There are two key administrative data sources:

• the Medicaid Statistical Information System (MSIS), which contains state-level claims and encounter data; and
• the Healthcare Effectiveness Data and Information System (HEDIS^©) data collection for managed care beneficiaries.

Administrative data, primarily from claims, are an important source of information on how the system is performing. A bill is generated to obtain reimbursement whenever a service is provided that requires payment. In contrast to population health surveys, which provide household reports and snapshots of the health of the population and experiences with care, claims-based data tend to provide a more detailed picture of the services received and costs of care for given diagnoses over time (as long as the individual is enrolled in that system). Administrative data therefore serve as a fundamental tool for monitoring the adequacy of care, although they have significant limitations, as noted in an earlier chapter.

In addition to the above two administrative data sources, quality measures can be found in population health surveys, especially in the data sets compiled by the National Health Information Survey (NHIS) for the Centers for Disease Control and Prevention and three surveys conducted by the Maternal and Child Health Bureau (MCHB): the National Survey

of Children’s Health (NSCH), the National Survey of Children with Special Health Care Needs (NS-CSHCN), and the National Survey of Early Child Health (NSECH). The NSCH is the most far-reaching of the three MCHB surveys in terms of the population covered, the sample size, and the topics covered.

THE MEDICAID STATISTICAL INFORMATION SYSTEM

The MSIS is a national database of Medicaid claims and eligibility data that is maintained by the Centers for Medicare and Medicaid Services (CMS) and consists of an aggregation of individual state-level claims databases. Reporting by states to the MSIS is mandatory for state Medicaid agencies. Thus, the MSIS contains data on all Medicaid children and Children’s Health Insurance Program (CHIP) children who are part of Medicaid expansions (although not children in separate CHIP programs). The state-level data reported to CMS for the MSIS provide a base that is useful for some measures, although it has some major limitations.

On the positive side, state-level files contain data on claims and encounters for services, which include data on health insurance, diagnoses, and the services or procedures provided as core data elements. The records contain a state-assigned unique personal identifier; this identifier can be used consistently to identify a given individual across different years and different enrollment periods, making it possible to track Medicaid beneficiaries over time within that state (MacTaggart, 2010).

The major weakness of the state-level data reported to the MSIS lies in its nature as a claims-based system. In most states, claims for services rendered under Medicaid primary care case management and fee-for-service care show a complete record of the services provided and generate a reimbursement for those services. However, contractors with managed care organizations may receive a capitated payment for beneficiaries. In such cases, their claims data may not necessarily describe the actual services provided. Managed care organizations may submit encounters or “shadow claims,” which, because they do not actually generate reimbursement, may be incomplete. CMS has indicated it is working with states to improve encounter data (MacTaggart, 2010). Furthermore, the Children’s Health Insurance Program Reauthorization Act (CHIPRA) requires the Department of Health and Human Services (HHS) to collect and analyze the MSIS data from states within 6 months. MSIS data have not been used as a source of reporting in the past (MacTaggart, 2010; Simpson et al., 2009), but the new federal reporting requirements, combined with federal efforts to improve state claims/encounter databases, may lead to more usable data in those databases. In the interim, states may combine the use of their claims/encounter databases with chart audits for a sample of children to report

on quality measures, using an approach analogous to the HEDIS hybrid methodology (see below).

A second weakness in the current MSIS database is the omission of children who are enrolled in separate (non-Medicaid) CHIP programs. The MSIS also does not include privately insured or uninsured children. Since CHIPRA now requires states to compare the status of children and adolescents served by public plans with that of the general population of children and adolescents on a statewide basis, MSIS data can provide only a partial picture of the services or outcomes of those who are enrolled in Medicaid or Medicaid-expansion CHIP plans.

HEDIS

Currently, administrative data from the HEDIS collection of data from managed care plans are a primary source of information at the state level for reporting on current and new quality-of-care measures. It should be noted that in a managed care environment, the state usually provides a negotiated payment to the managed care organizations (MCOs) for services, and the MCOs pay the providers. In cases where the providers are paid on a fee-for-service basis, claims data will exist. In cases where health plans pay providers through a negotiated payment per member, there is no need for claims, and providers instead generate shadow claims for the encounter.

Developed by the National Committee for Quality Assurance (NCQA), HEDIS is a tool used by more than 90 percent of health plans to report on quality (NCQA, 2010). In its annual State of Health Care Quality report, NCQA releases detailed, plan-specific performance information for both commercial and Medicaid plans. NCQA’s 2008 report for Medicaid provided information on 52 measures of clinical quality (NCQA, 2008). States also release their own reports. For example, Michigan releases an annual report on its HEDIS results by MCO (MDCH, 2008). New York has long issued annual report cards (Quality Assurance Reporting Requirement) on health plan performance on HEDIS as well as state-level measures (NYDOH, 2010).

Many of the initial core measures published and posted for public comment by the Secretary of HHS are HEDIS measures that health plans currently use to report on quality. The measures on immunization, prenatal care, chlamydia screening, and well-care visits are examples of the HEDIS measures in the core set. This is not surprising given that the AHRQ committee recommending measures and the CHIPRA legislation placed a premium on measures that were grounded and in use. Further, because claims data form the basis for HEDIS measures, these measures generally are limited to whether a service has been delivered, rather than broader care processes across episodes of care or outcomes. For example, there is a

measure of whether chlamydia screening took place, but not whether appropriate follow-up occurred if the result was abnormal.

HEDIS protocols for assessing measures specify either methods that use administrative data alone or hybrid methods that combine the use of administrative data with chart reviews for a sample of beneficiaries. These HEDIS protocols form a strong base for CMS to use in guiding the states on reporting, but there are important cautions. First, these measures were designed to be used by managed care plans, and the protocol includes features designed to ensure that members are “continuously” enrolled in health plans long enough to benefit from their quality improvement policies (frequently 11 out of 12 months, but the “continuous enrollment” period can be longer for some measures). As a result, Medicaid children who are not enrolled in a managed care plan for the required amount of time are omitted from the measurement results, even if they have been registered in Medicaid for the designated period. As an example, HEDIS specifications for reporting immunization coverage specify that only children enrolled for 11 or more of the prior 12 months be included in the reporting denominator (NCQA, 1996). In one study, fewer than half of all enrolled Medicaid children (39 percent) were included in the health plan denominator in the 12 state studies, although most (78 percent) had been on Medicaid for the required length of time (Fairbrother et al., 2004). This problem becomes more acute as the continuous enrollment periods increase (asthma measurement, for example, requires 2 years of continuous enrollment).

A second problem is that data are not reported in a standardized manner (Partridge, 2007). Thus, although almost 90 percent of Medicaid programs and 100 percent of CHIP programs reported using HEDIS access and effectiveness measures related to child health in 2009, the data may not be comparable across states (Smith et al., 2009). Standard definitions frequently are not used, with states modifying HEDIS definitions to accommodate a Medicaid population with shorter coverage spells, as well as other local concerns (Partridge, 2007). For example, although the 1997 State Children’s Health Insurance Program (SCHIP) statute required each state to file an annual report—including the state objectives for SCHIP, the performance measures used, and progress that year toward meeting the objectives—it did not specify exactly how measures were to be reported. A review of state reports in 2005 on four HEDIS measures showed great variation in the number of states that reported on the measures, from a high of 34 to a low of 10 (Partridge, 2007). Furthermore, states modified the HEDIS specifications to accommodate their priorities, so that even though states reported on the same measures, the data were not strictly comparable. The reviewers concluded that comparable data were sufficient to build a national SCHIP database and generate national averages for two of the four measures (Partridge, 2007). This issue of the level of compara-

bility will need to be addressed in developing the reporting format required by CHIPRA.

The HEDIS protocols are an important starting point for measurement under CHIPRA. But the measures will need to be respecified to be appropriate for the entire Medicaid and CHIP population through inclusion of a denominator that addresses enrollment in these two programs. And with the emphasis in the Affordable Care Act (ACA) on all populations of children, measures may need to be respecified again to include all children, regardless of payer.

NATIONAL SURVEY OF CHILDREN’S HEALTH

The NSCH is a nationally representative household survey of children aged 0−17 that includes state-level estimates. It has been administered twice (in 2003 and 2007); a third fielding is planned for 2011 that is expected to include additional items on child well-being/thriving, health insurance and access to care, and items relevant to life-course research. The third wave of survey data may also include nearest cross-street information to enhance the geocoded linking of these data to other neighborhood-level data systems.

The NSCH represents responses of parents/guardians of a randomly selected child in each household. Survey questions encompass child health status and health conditions, health insurance and medical home, parental health, school engagement, media exposure, youth activities, and neighborhood conditions. The NSCH produces estimates for numerous demographic, socioeconomic, and health status subgroups of children, including whether their health insurance coverage is public or private, whether they have special health care needs, their race/ethnicity, their primary language, whether they are foreign born or adopted, the immigration status of their parents, their household income, and the household’s use of public assistance. NSCH national and state-level findings for numerous subgroups are posted at www.childhealthdata.org.

The NSCH includes multiple patient-centered categories of data relevant to the measurement of health care quality for children and adolescents (these data components are in addition to measures of physical and dental health, mental and emotional health, health insurance coverage, and other topics relevant to the child’s physical and social environments). The categories include preventive medical care visits, preventive dental care visits, getting needed mental health care, one or more unmet needs for care, medical home, personal doctor or nurse, usual sources for sick and well care, family-centered care, problems in obtaining needed referrals, effective care coordination, access to specialty care or services, receipt of care from specialist doctor, doctor asks about concerns, and developmental screenings.

NATIONAL HEALTH INTERVIEW SURVEY

As described in Chapter 4, the NHIS is an annual household survey conducted by the National Center for Health Statistics that collects information on all household members, including children and adolescents. NHIS data provide the basis for the AHRQ reports on health care disparities, indicating how many children and adolescents have access to health care coverage, as well as a specific source of usual health care, and how many children and adolescents rely on hospital-based services (such as outpatient or emergency departments) for usual or ongoing care. NHIS data also are used in identifying sources of health care disparities, especially in areas that involve access to care or treatment for conditions such as asthma and mental and emotional disorders.

RESOURCES FOR DATA ANALYSIS AND LINKAGE

This section describes four key resources for data analysis and linkage:

• the databases and tools that are part of the AHRQ Healthcare Cost and Utilization Project (HCUP);
• the application forms for public insurance, which contain demographic information on Medicaid and CHIP beneficiaries;
• the Physician Quality Reporting Initiative (PQRI); and
• examples of state-based data warehouse capacities that foster linkage across multiple database systems.

HCUP Databases and Tools

The HCUP databases, supported by AHRQ, represent the largest collection of multiyear, all-payer hospital and emergency room discharge data that can be applied to hospital claims to assess safety events, ambulatory care−sensitive hospitalizations, and other measures of potential interest. More than 40 states provide data as part of the project, collectively representing more than 95 percent of all discharges (AHRQ, 2010). The HCUP databases are constructed using a core set of clinical and nonclinical details found in a typical discharge claim for hospitals and emergency rooms, including data on primary and secondary diagnoses and procedures, admission source and discharge disposition, patient demographics, expected payment source, total charges, length of stay, and hospital characteristics. From this core set of discharge information, several subsets of data can be extracted to create inpatient, ambulatory care, emergency care, and child-specific databases, as shown in Table D-1. Each database in turn can be used to examine quality of care with the AHRQ quality indicators, to

TABLE D-1 HCUP Databases

aggregate data using clinical classification codes (the International Classification of Diseases [ICD]-9-CM and ICD-10 codes), and to identify and measure coexisting conditions using Comorbidity Software (see Table 2 in Fairbrother et al., 2010).

HCUP also includes software tools and indicators with which to measure quality (see Table D-2). AHRQ first developed three indicator sets: the Inpatient Quality Indicators (IQI), for the quality of care received in hospitals; the Prevention Quality Indicators (PQI), for potentially preventable hospital admissions; and the Patient Safety Indicators (PSI), for preventable complications of care. These measures were constructed based on adult health issues, complications, chronic conditions, and patterns of care and were not adequate to address the complexity of child and adolescent health care needs. Responding to this gap, AHRQ developed a fourth set of indicators focused on the safety and quality of pediatric hospital care—the Pediatric Quality Indicators (PDIs) (see Table D-3). These indicators focus on potentially preventable complications arising from inpatient care and on preventable hospitalizations for pediatric patients. This software could be used, for example, in calculating pediatric catheter-associated blood stream infection rates, one of the initial AHRQ core measures, using a state’s inpatient database.

While the HCUP tools and indicators provide important ways to

TABLE D-2 HCUP Software Tools and Indicators

TABLE D-3 Pediatric Quality Indicators (PDIs)

measure the quality of care in hospital and emergency room settings, their capacity to measure disparities is limited: more than a quarter of the claims for children do not indicate race/ethnicity (HCUP, 2006). Moreover, the nature of the disparities varies with each measure. Finally, even though the measures reflect the most prominent safety issues, the prevalence of these complications is relative low, limiting the types of analysis that can be performed. Another issue with HCUP is that income data are at the community and not the individual level.

Application Forms for Public Insurance

Application forms for public insurance (Medicaid and CHIP) are a source of demographic information because they ask parents about their child’s or adolescent’s race, ethnicity, age, gender, income, and in some cases language. A validation study conducted in New York comparing race and ethnicity information collected from applications with information collected directly from parents as part of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) surveys showed high levels of concordance between the two for all races and ethnicities (Fairbrother and Simpson, 2010).

Some states, such as New York, Kentucky, and Georgia, have the capacity to link demographic information from the application forms with claims-based data. This approach enables these states to monitor services and outcomes by selected demographic characteristics that are included on the application form and thus monitor disparities by race, ethnicity, language (if collected), and income. The federal MSIS data set also links claims/encounters data to demographic data from encounters, thus creating the potential to monitor race, ethnicity, language, and income disparities at the federal level. However, the ability to monitor disparities at the national level is restricted because the states do not collect their demographic enrollment data in a systematic manner. A review of application forms from all states (Fairbrother and Simpson, 2010) shows that only 18 states ask for “Hispanic/Latino” ethnicity as a separate category, while 19 states merge ethnicity with racial categories. Of these, 7 allow the applicant to choose more than one “race”; hence, an individual could select both “black” and “Hispanic” in these states but not in the others. Eight states have no race/ethnicity categories, but leave a blank for applicants to fill in. With respect to primary language, 14 states ask for “English,” “Spanish,” and “other” or list specific languages. However, 21 states have only a blank for applicants to fill in with their primary language. The design of application forms has been left to the states in the past; with the new emphasis on monitoring disparities at both the federal and state levels, standardization will be necessary.

Physician Quality Report Initiative

The Medicare PQRI is a quality reporting system that supports incentive payments for eligible professionals who report data on quality measures based on parameters established by CMS. The American Recovery and Reinvestment Act (ARRA) Health Information Technology for Economic and Clinical Health (HITECH) legislation significantly expanded the significance of the PQRI and the PQRI registry, which now incorporate providers who serve patients enrolled in Medicaid and CHIP plans as well as Medicare. Most of the 179 quality measures in the 2010 PQRI system are specified for adults. However, a significant number of measures are designed explicitly for children (especially those associated with the treatment of asthma, ear infections, childhood cancers, pediatric end-stage renal disease, and HIV/AIDS). Other measures include children and adolescents in the denominator, but the measurement age breaks limit the feasibility of determining how many children are included in certain data sets.

The specifications for the quality measures under PQRI provide the details for the numerator and denominator and therefore support analyses of the percentage of a defined patient population that receives a particular process of care or achieves a particular outcome. For example, PQRI measure 65 focuses on the avoidance of inappropriate use of antibiotic treatment for children with upper respiratory infections.

Examples of Data Warehouses and State-based Linkage Activities

Although states vary in their capabilities to collect, store, and analyze data, some states, such as New York, Georgia, and Kentucky, have strong warehousing capabilities, including in some cases the ability to link state databases. New York, for example, collects member-level data reported by Medicaid managed care plans (for all members) as part of annual HEDIS reporting and has created linkages of quality measurement results with eligibility files and CAHPS surveys. The resulting linked data set is organized at the person level, and includes demographic and service delivery information for Medicaid members in each measure. The resulting data warehouses can be used to monitor quality on a variety of measures and to display results by race/ethnicity, age, gender, and geography, making it possible to monitor performance for the population as a whole and for vulnerable groups.

Furthermore, some states have linked health data sets, giving them the ability to monitor over time and across settings. For example, New York has a linked data set consisting of childbirth and fetal death certificates, maternal and child hospital discharges, and Medicaid claims before and after the birth. Using this linked data set, New York can relate, for example,

aspects of prenatal care to subsequent outcomes and health behaviors. Linking data across time can also make it possible to monitor important aspects of chronic care, such as whether a child has filled all prescriptions for medications needed to treat specific conditions, whether there are duplicative or overlapping medications in a regimen, or whether a rehospitalization occurred.

REFERENCES

AHRQ (Agency for Healthcare Research and Quality). 2010. Healthcare Cost and Utilization Project (HCUP). http://hcupnet.ahrq.gov/ (accessed December 2, 2010).

Fairbrother, G., and L. Simpson. 2010 (unpublished). Measuring and reporting quality of health care for children: CHIPRA and beyond.

Fairbrother, G., A. Jain, H. L. Park, M. S. Massoudi, A. Haidery, and B. H. Gray. 2004. Churning in Medicaid managed care and its effect on accountability. Journal of Health Care for the Poor and Underserved 15(1):30-41.

Fairbrother, G., R. Sebastien, J. McAuliffe, and L. Simpson. 2010 (unpublished). Monitoring changes in health care for children and families. Child Policy Research Center.

HCUP (Healthcare Cost and Utilization Project). 2006. The KIDS’ inpatient database. http://www.hcup-us.ahrq.gov/kidoverview.jsp (accessed November 19, 2010).

MacTaggart, P. 2010 (unpublished). Overview of development & use of quality measures for children. George Washington University Medical Center.

MDCH (Michigan Department of Community Health). 2008. Michigan Medicaid HEDIS Report. Lansing, MI: MDCH.

NCQA (National Committee for Quality Assurance). 1996. HEDIS 3.0 Manual. Rockville, MD: AHRQ.

NCQA. 2008. The state of health care quality 2008. http://www.ncqa.org/portals/0/newsroom/sohc/SOHC_08.pdf (accessed March 22, 2011).

NCQA. 2010. HEDIS & quality measurement. http://www.ncqa.org/tabid/59/Default.aspx (accessed March 22, 2011).

NYDOH (New York State Department of Health). 2010. Managed care reports. http://www.health.state.ny.us/health_care/managed_care/reports/ (accessed December 3, 2010).

Partridge, L. 2007. Review of access and quality of care in SCHIP using standardized national performance measures. Washington, DC: National Health Policy Forum.

Simpson, L., G. Fairbrother, J. Touschner, and J. Guyer. 2009. Implementation choices for the Children’s Health Insurance Reauthorization Act of 2009. New York: The Commonwealth Fund.

Smith, V., J. Edwards, E. Reagan, and D. Roberts. 2009. Medicaid and CHIP strategies for improving child health. New York: The Commonwealth Fund.