B
Information for Accountability
Background
Public and private purchasers are increasingly accountable for their health care spending. Accountability involves acceptance of the responsibility to balance costs while purchasing health care with reasonable quality, as well as to determine the relative values of a variety of options within the constraints of existing resources (IOM, 1997a and b; 1998). An essential requirement for accountability is information on performance, such as whether a health plan has met conditions of participation or reached targets agreed to receive financial bonuses.
For cost-conscious purchasers and public policymakers, it is difficult to use the existing sources of public information (e.g., national surveys of households and employers and report cards from health plans) to determine the relative values of different insurance purchasing options. A better understanding of the technical limitations and strengths of the currently available data might assist decision makers as they judge performance and make informed decisions, such as whether to continue or modify current oversight processes or contractual relationships with different health care delivery organizations.
This appendix begins with a description of the available sources of public data on health insurance, with an emphasis on information about children's coverage. The appendix then discusses more generally the technical challenges in measuring performance and describes the kinds of capacities and infrastructure that need to be built to support systems of accountability, particularly in the area of health care for children. The appendix profiles sources of publicly collected data, such as federal surveys, as well as privately collected data collected by health plans and reported to consumers.
Sources of Date on Insurance Coverage and Utilization
Medicaid is the largest single insurance program for American children. One significant limitation encountered in evaluating evidence about Medicaid coverage and services has been the poor history of collecting information within the program. The U.S. Congress and the U.S. Department of Health and Human Services (DHHS) have had minimal reporting requirements, and other than financial reporting, the Health Care Financing Administration has not requested information that can be used to easily evaluate the impacts of the services that Medicaid provides.
For example, although the Omnibus Budget Reconciliation Act of 1989 mandated treatment found to be indicated through Early and Periodic Screening, Diagnosis, and Treatment (EPSDT) program screens, there was no requirement to collect information on whether the children who were referred for
treatment actually received any. The law did call for the collection of data on the number of children receiving screens, the number of children referred for treatment, and the number of children receiving dental services.
Under fee-for-service systems, screens and completed referrals could potentially be tracked by examining bills submitted for reimbursement. The EPSDT program reporting process is less reliable under managed care systems. Health plans paid through capitation may not report the encounters as EPSDT program screens if they follow other internal reporting guidelines, and capitated providers are paid whether they deliver the EPSDT program services or not. Consequently, information on the effectiveness of EPSDT as a comprehensive program of services for children is not complete.
Table B.1 summarizes the federal data sources that contain information about rates of health insurance coverage, health care utilization, and health status. The technical issues arising from these sources of data can be summarized as follows:
- Differences in definitions and data collection methods; for example, the Current Population Survey and National Health Interview Survey use different questions in collecting insurance data, making comparisons difficult.
- Limitations and discontinuities in published reports from different sources, for example, lack of statistical breakdowns for racial and ethnic groups, lack of published breakdowns for children or breakdowns for different ages, lack of breakdowns of employer data by size of employer, and lack of standardized categories for employer size.
- Differences in capacity to report current information because of the cycle of reporting requirements, the lag time in collecting large enough samples, a lack of appropriate technical resources to analyze the information, incompatibilities of computer systems, out-of-date computer software, and many other technical factors.
In addition to these technical problems, other obstacles exist. For example, some purchasers do not specify reporting requirements or standards for information collection, whereas in other circumstances different purchasers of services from the same providers may have conflicting requirements and standards (IOM, 1997a). Purchasers may make decisions without analyzing the existing data or information on effectiveness, and systematic information on implementation of new programs and strategies is not collected, so inefficiencies may inadvertently be repeated when new programs are put in place.
In the public sector, technical capacity is generally more limited than in the private sector (IOM, 1989, 1998). For example, a GAO report (GAO, IMTEC, 1993) found that the majority of government-owned software was several years old and could not keep up with the technical demands for the timely analysis of data. In recent years, with the availability of Internet access, most federal agencies have become better able to make public information accessible to researchers, state governments, and the general public.
In the private sector, a great deal of information is proprietary and is not generally available to the public, such as the scope and price of benefits for dependents, provider reimbursement levels, etc. However, the quality movement, with its emphasis on report cards, performance measures, and plan-to-plan standards for data collection and reporting, has greatly improved the level of information sharing. Information sharing is discussed in the following section.
Trends in Information Infrastructure Development
Private Sector
The de facto standard for health plan performance reporting is the Healthplan Employer Data Information System (HEDIS), now in its third version (HEDIS 3.0) (see Box B.1). This standardized set
TABLE B.1 Federal Data Sources for Information on Health Insurance, Health Care Utilization, and Health Status
Federal Data Source |
Department |
Agency |
Data Type |
Data Collection Frequency |
Description |
Current Population Survey |
DOC |
Bureau of the Census |
Survey, households |
Monthly |
Primary source of detailed information on income work experience in the U.S. for the civilian, noninstitutional population ages 15 years and older. Annual March Supplement used to generate U.S. Population Profile and labor market profile and to make employment projections. Data gathered include employment status, income, poverty status, education and sources of health insurance and noncash income (e.g., food stamps). |
Medicaid Data System |
DHHS |
HCFA |
Record |
Continuous |
The primary sources for Medicaid statistical data are the HCFA 2082 and the HCFA 64 reports. HCFA 2082, or Medicaid program statistics, is the basic source of reported data on the Medicaid population characteristics and utilization. HCFA 64, or Medicaid financial statistics, is a statement of actual expenditures for Medicaid programs that states submit to HCFA each quarter (includes payments to disproportionate share hospitals). |
Medical Expenditure Panel Survey |
DHHS |
AHCPR |
Four surveys |
Longitudinal, overlapping panel design; new panel each year. Data collected over two-year period. |
Four national probability surveys: Household Component, Insurance Component, Medical Provider Component, and the Nursing Home Component. Collects information on health care use, expenditures, sources of payment, insurance coverage, and nursing homes and residents. |
National Ambulatory Medical Care Survey |
DHHS |
CDC |
Record |
Annual |
Provides statistics on the demographic characteristics of patients and the services provided, including information on diagnostic procedures, patient management, and planned future treatment. |
Federal Data Source |
Department |
Agency |
Data Type |
Data Collection Frequency |
Description |
National Employer Health Insurance Survey |
DHHS |
CDC (cosponsored by AHCPR and HCFA) |
Survey, private and public-sector businesses |
Periodic; first collection began in April 1994 and ended in December 1994 |
Provides national and state-level analyses of private health insurance and examines characteristics of employer-sponsored health insurance. (Released December 1997) |
National Health Interview Survey |
DHHS |
CDC |
Survey, households |
Annual |
Principal source of information on the health of the civilian noninstitutional population. Basic health and demographic information, incidences of acute and chronic conditions, and injuries, the numbers of people with limited activity due to health conditions, data on respondent-assessed health status, the rate of utilization of health care services (physician care and short-stay hospitalizations), and data on other special health topics are provided. |
National Health and Nutrition Examination Survey |
DHHS |
CDC |
Survey |
Continuous |
Collects nationally representative information on the health and nutritional status of the U.S. civilian, noninstitutional population (ages 2 and older) to estimate number and percentage of people and subgroups of the population with selected diseases and risk factors. Data are used to study the relationship between diet and health, and between nutrition and health, and to explore emerging public health issues. |
National Health Provider Inventory (NHPI) |
DHHS |
CDC |
Survey |
Periodic; last conducted in 1986 (called Inventory of Long-Term Care Places) |
A comprehensive national listing of nursing homes, residential care facilities, hospices, and home health agencies. The NHPI serves as a model framework for several sample surveys. Provides national statistics on the number, type, and geographical distribution of health providers in the U.S. |
National Home and Hospice Care Survey |
DHHS |
CDC |
Survey |
Annual |
Provides data on home and health agencies and hospices and their current patients and discharges to assess the availability and utilization of these services. Data include referral and length of service, diagnoses, number of visits, patient charges, health status, reason for discharge, and types of services provided. |
National Hospital Ambulatory Medical Care Survey |
DHHS |
CDC |
Record |
Annual |
Uses visits to hospital emergency departments and outpatient departments to obtain information. Data include demographic characteristics of patients, expected sources of payment, patients' complaints, physicians' diagnoses, diagnostic/screening services, procedures, medication therapy, disposition, types of health care professionals seen, and causes of injury where applicable. |
National Hospital Discharge Survey |
DHHS |
CDC |
Record |
Annual |
Principal source of information on inpatient utilization of hospitals. It includes data on diagnoses, procedures, lengths of stay, expected sources of payment, and patterns of use of care in nonfederal hospitals. Data are used to track specific diseases, the introduction of new technologies, and the impact of changes in the financing systems. |
National Immunization Survey |
DHHS |
CDC |
Survey |
Continuous |
Collects information on the immunization coverage of children 19-35 months of age across the U.S. |
National Nursing Home Survey |
DHHS |
CDC |
Survey, providers and patients |
Periodic; most recently in 1995 |
Provides data on nursing homes and their services, staffs, and financial characteristics and data on residents' personal and health characteristics. |
National Nursing Home Survey Follow-up |
DHHS |
CDC |
Longitudinal survey |
Periodic; last survey February to April 1990 (Wave III) |
Follows the cohort of surviving current residents and discharged residents sampled in the 1985 National Nursing Home Survey. The primary purpose is to provide data on the flow of people into and out of long- term-care facilities and hospitals. Data include subjects' vital status, living arrangements, nursing home stays, hospital stays, and sources of payment for stays. |
National Survey of Ambulatory Surgery (NSAS) |
DHHS |
CDC |
Record |
Annual |
Provides information on the use of freestanding and hospital-based ambulatory surgery centers in the U.S. In addition, NSAS provides detailed data on the wide variety of surgical and diagnostic treatments now performed outside the inpatient setting. Other data include expected sources of payment, patient disposition, and region. |
of performance measures was developed by the National Committee for Quality Assurance (NCQA), an accreditation organization for health maintenance organizations (HMOs), with input from a variety of employers, federal and state agencies, providers, consumer groups, and others.
HEDIS measures cover eight domains:
- effectiveness of care,
- accessibility and availability of care,
- satisfaction with the experience of care,
- cost of care,
- stability of the health plan,
- informed health care choices,
- use of services, and
- descriptive information about health plans.
NCQA has compiled a database with HEDIS data from more than 330 health plans that collectively cover more than 37 million enrollees. Because HEDIS performance data are standardized, they allow comparisons of performance across plans; for example, the percentage of children who have been immunized can be reported for an individual plan, and average immunization rates can be compared for different regions; the national average can also be determined. NCQA has also recognized that improved performance measurement will require more detailed content as well as comparability and connectivity of health data bases.
Later revisions of HEDIS will incorporate these goals into new health plan accreditation standards for information systems capability. In addition, NCQA is in the process of developing HEDIS measures and accreditation standards for provider organizations.
The Joint Commission on Accreditation of Health Care Organizations (JCAHO) is likewise
BOX B.1 Selected Reporting Measures for HEDIS 3.0
|
making efforts to improve monitoring and information structures by integrating outcome and performance measures into its accreditation process. One initiative, entitled ORYX, will require health care providers and, eventually, plans to collect and use performance data and to communicate those data to JCAHO. As part of the ORYX initiative, JCAHO has developed the Indicator Measurement System (IMSystem). The IMSystem will function as a comparative performance measurement system on measures of patient outcomes. This system will allow the Joint Commission to evaluate performance as part of the accreditation process.
Individual purchasers have also created a demand for timely information on health plan performance beyond that of HEDIS or JCAHO. For example, in the late 1980s, Digital Equipment Corporation developed a set of performance standards for HMOs. Digital has 800 facilities, annual revenues of $14 billion, 56,000 employees, and annual health care costs of $150 million.
When negotiating contracts to purchase care for employees, Digital and other large employers using Digital's HMO performance standards began to specify that providers must meet standards in the following six areas:
- access and member services,
- information management,
- behavioral health,
- finance and contracts,
- clinical quality, and
- health education and preventive services.
Box B.2 lists Digital's HMO performance standards that are specific to children's health. The most recent edition of Digital's standards builds on HEDIS 3.0 and Foundation for Accountability measures and reporting requirements, including the format for reporting.
Purchasing coalitions such as the Pacific Business Group on Health (PBGH) have also increased the demand for standardized information from health plans. PBGH conducts a variety of studies to assess and improve the quality of health plans, hospitals, and medical groups, including regular surveys of consumers and providers. PBGH also manages the collection of HEDIS quality measures for all of the larger California health plans serving commercial enrollees. PBGH makes performance information on plans, medical groups, and hospitals available to the public through a web site and published report cards.
A majority of health plans are reporting information on plan performance, although they are meeting different sets of standards. As health plans begin to contract with public agencies to deliver health care financed by Medicaid and state insurance programs, pre-existing data used for performance measurement can be used as a platform for new applications and analyses on coverage, utilization, access, and sources and continuity of care.
Finally, each of the major benefits consulting firms maintains its own national data base on health plans. These include and extend well beyond HEDIS.
Public Sector
In 1979, DHHS initiated a process of setting national objectives and performance measures for health promotion and disease prevention. The initiative, known as Healthy People 2000, has evolved to include 300 objectives in 22 priority areas addressing health promotion, health protection, preventive services, and data systems (DHHS, 1995) (see Box B.3).
Public review and comment has involved more than 10,000 individuals from hundreds of national membership organizations, 56 state and territorial health departments, the Institute of Medicine (IOM),
and federal agencies (DHHS, 1990, 1992). Regular reports are issued to describe national progress toward meeting the objectives.
In 1995, DHHS began a process of establishing performance measures and objectives in anticipation of block grants legislation in public health, mental health, and substance abuse (NRC, 1997). A national initiative known as performance partnership grants is under way to improve the capacity of health departments and other public agencies to monitor and report on the health status of the U.S. population (NRC, 1997). DHHS also sponsored another IOM effort to examine performance measures for public health, which resulted in the development of prototypical sets of indicators for specific public health concerns that can be used by communities to monitor the performance of public agencies, private organizations delivering personal health services, and others (IOM, 1997b).
Scope of Performance Measurement for Children
The scope of performance measurement on aspects of children's health and health care spans federal, state, and local governments; independent and university-based policy research organizations; provider organizations; and many other sources.
At the federal level, the agencies that follow use, produce, and advance performance measures in their role or responsibility with regard to children's access to care, health status, and health outcomes.
- U.S. Department of Health and Human Services:
- Agency for Health Care Policy and Research,
- Centers for Disease Control and Prevention,
- Food and Drug Administration,
- Health Care Financing Administration—Medicaid,
- Health Care Financing Administration—Disabled,
- Health Resources and Services Administration,
- Indian Health Service,
- National Institutes of Health, and
- Substance Abuse and Mental Health Services Administration;
- U.S. Department of Agriculture (nutrition programs);
- U.S. Department of Commerce (Current Population Survey);
- U.S. Department of Defense (Comprehensive Health and Medical Programs of the Uniformed Services);
- U.S. Department of Labor (employer and workforce statistics);
- U.S. Department of Treasury (tax policy); and
- Social Security Administration—Social Security Income;
At the state level, the following agencies use, produce, or advance performance measures for children's health care, insurance, access to care and for quality assurance:
- Medicaid agency,
- health department (often decentralized to include local agencies),
- human services department (often decentralized to counties),
- insurance department, and
- professional licensure and credentialing boards.
Private-sector organizations with specific interest in performance measures for children's health include:
BOX B.2 Digital HMO Performance Standards Objectives:
Proximity Requirements Proximity Requirement/Two Primary Care Practitioners (PCPs)(Pediatrics): Members will have access to at least two available pediatric primary care practitioners (pediatric and/or family practice) within a 15-mile radius and/or 30 minutes in all zip codes in the HMO's service area. This standard may be adjusted in rural areas if appropriate. Provider Access The HMO will maintain the following provider-to-member ratios to ensure adequate provider access. These standards will be used as a guide and should report only those providers with practices that are open to new patients. Provider network and delivery system differences will be evaluated as needed. Ratio for Internists/Family Practitioners: The ratio of internists/family practitioners to the enrolled population will not be less than 0 = 0.50:1,000. Ratio for Pediatricians: The ratio of pediatricians to the enrolled population will not be less than 0 = .25:1,000. Appointment Availability The HMO will have a process in place to monitor PCP appointment availability for internal medicine/ family practice and pediatrics. In addition, the HMO will have a triage system which is available to members 24 hours a day. This triage system will have established guidelines for directing members to an appropriate level of care. Nonsymptomatic Office Visits: Members requesting an office visit for nonsymptomatic related conditions will be seen within 30 calendar days. Examples of nonsymptomatic visits include well/ preventive care appointments for services such as annual physical examinations, annual gynecological examinations, and pediatric/adult immunizations. Routine (Follow-up) Office Visits: Appointments will be available within 14 calendar days. Examples of routine appointments include follow-up appointments, blood pressure checks, and suture removal. Nonurgent, Symptomatic Office Visits: Members requesting an office visit for nonurgent, symptomatic conditions will receive telephone triage including guided self-care the same day and the opportu |
nity for an office visit within 48 hours. Examples of nonurgent, symptomatic office visits include appointments for the diagnosis and treatment of colds, headaches, minor injuries, joint/muscle pain, and similar nonurgent conditions. Urgent, Symptomatic Office Visits: Members requesting an office visit for urgent, symptomatic conditions will be seen by a physician within 24 hours. Examples of conditions requiring urgent care include persistent diarrhea and/or vomiting and/or high fever (>101°F). Emergency Care Services: Emergency care services will be available 24 hours per day, 7 days per week. Examples of emergency conditions include the presence of severe symptoms that require immediate medical care and for which a delay in care would be life threatening or would seriously jeopardize health or create permanent disability. Emergency Access to Medical Provider: On-call coverage or centralized triage by a medical provider will be available 24 hours per day, 7 days per week, for response to medical inquiries. Change of PCP: The HMO will permit Digital enrollees to change primary care practitioners and/or health care facility locations upon the enrollee's request. Health Education and Preventive Services The HMO will provide, upon request, targeted on-site health promotion services. These services may include:
Information Management Objectives To ensure that the data collection and reporting sources in each HMO represent an acceptable standard of reporting for internal management, including operations management, case and clinical management, financial monitoring and management, as well as external reporting to governmental agencies and employers. To ensure that each HMO demonstrates an ever improving system to record the reliability and validity of the information needed to manage patient care and the entire health plan. To ensure that each HMO displays an increasing ability to not only report but also use information to monitor and improve the clinical effectiveness and cost-efficiency of the organization. To strongly encourage each HMO to effectively use information technology in the delivery of care to enrollees. Examples include clinical workstations, Internet access, and electronic processing of eligibility, referrals, and claims. SOURCE: Digital Equipment Corporation, 1997. |
BOX B.3 Selected Healthy People 2000 Goals and Objectives Priority Areas with Objectives Relating to Children's Health Status and Health Services Nutrition 2.4 Growth retardation among low-income children ages 5 years and younger 2.10 Iron deficiency prevalence Mental Health and Mental Disorders 6.3 Mental disorders in children and adolescents ages 18 years and younger 6.14 Clinician review of children's mental functioning Unintentional Injuries 9.3a Motor vehicle crash deaths among children ages 14 years and under 9.5a Drowning deaths for children ages 4 years and under Environmental Health 11.1 Asthma hospitalizations 11.2 Mental retardation 11.4 Blood lead levels among children 11.17 Children's exposure to smoke at home Oral Health 13.1 Dental caries in children ages 6-8 years, and adolescents ages 15 years 13.2 Untreated dental caries in children ages 6-8 years and older Maternal and Infant Health 14.1 Infant mortality 14.4 Fetal alcohol syndrome |
- accreditation organizations, particularly the National Committee for Quality Assurance and the Joint Commission on Accreditation of Health care Organizations;
- Foundation for Accountability;
- ambulatory certification programs;
- insurers;
- purchasers (e.g., large employers and purchasing coalitions);
- trade unions;
- advocacy organizations; and
- private philanthropic organizations (e.g., Annie E. Casey Foundation, Robert Wood Johnson Foundation).
Given the large number of agencies and organizations with interest in and/or responsibilities for collecting and analyzing information on aspects of performance in children's health, it is not difficult to see why there is no single entity that has taken overall responsibility for assuring efficient, comprehensive performance measurement in children's health care.
14.5 Low birth weight. 14.14 Pregnant women and infants receiving risk-appropriate care. 14.15 Newborn screening and treatment. 14.16 Babies receiving primary care 14.17 Spina bifida and other neural tube defects Diabetes and Chronic Disabling Conditions 17.15 Clinician assessment of childhood development (visual acuity, hearing, speech, motor development) and treatment/referrals Immunization and Infectious Diseases 20.11 Immunization (percent immunized) 20.15 Financial barriers to immunization Clinical Preventive Services 21.2 Receipt of recommended services: immunizations, screening, counseling, chemoprophylaxis, interventions for children with special risk factors (see Box 3.2) Objectives Relating to Data Systems and Accountability Surveillance and Data Systems 22.1 Health status indicators: develop, establish use of, monitor, and provide 22.2 National data sources: state-level data for at least two thirds of state objectives 22.3 Comparable data collection procedures for federal, state, and local agencies 22.4 Identify gaps in health data 22.5 Periodic analysis and publication of data 22.6 Number of states with data transfer systems 22.7 Timely release of national data |
Assessing the Adequacy of Performance Measurement for Children's Health Care
The Balanced Budget Act of 1997 specifies that states applying for funds to establish children's health insurance programs must submit plans that describe the strategic objectives, performance goals, and performance measures that they will use to meet those goals and objectives. These plans will become legal documents, and DHHS will hold the states accountable for implementing them (IOM, 1998).
In early 1998, the Health Care Financing Administration and Health Resources and Services Administration of DHHS began to discuss the development of reporting requirements and other regulatory mechanisms to help monitor the performance of the states in meeting the objectives of the state plans.
In addition to the technical and methodological challenges described above, there are other challenges in developing performance measures and other strategies for monitoring methods of accountability in children's health care. Some of these challenges are not specific to children's health care, such as the fundamental inadequacy of the health care industry's clinical information systems or the lack of an evidence base for most current clinical services. However, special challenges relate to children. These include: (1) the inability to know the maximum potential for each child's physical, mental, and social health status, and therefore to detect and measure shortfalls; (2) ambiguity about the health care
system's scope of responsibility for ensuring optimal psychological, cognitive, and environmental inputs to children's health status compared with the responsibilities of parents, communities, and the larger environment; and (3) the embryonic state of reliable and valid outcomes measures for services to children with chronic physical and mental conditions.
Broadly viewed, large knowledge gaps remain in evidence-based medical practice, in the health care industry's information systems, measures of children's health status, as well as in the nature and extent of relationships between children's health status, health care interventions, and other causal factors.
Policy Options to Improve Accountability
In addition to adequate data sources and measures of performance, meaningful accountability—in children's health and other areas—requires an adequate means of enforcing the performance obligations assumed by each relevant party. In the SCHIP program, relevant parties extend from DHHS to states to health plans to providers to enrollees and their families (IOM, 1998). General methods of enforcement of accountability in these relationships include the following:
- setting minimum conditions for delegation of responsibility through contracting or other means, and
- setting incentives based on performance.
Factors resulting in difficulties in enforcement include the following:
- weaknesses in performance measurements,
- insufficient resources to implement enforcement methods (e.g., inadequate state agency budgets for monitoring health plan performance),
- failure to recognize enforcement methods,
- lack of resources among Medicaid beneficiaries for filing a lawsuit,
- failure to clearly specify enforcement methods in negotiations and/or contract language, and
- failure to implement enforcement methods.
A variety of general policy options for improving accountability are available. These options are not mutually exclusive and can be implemented separately or in a coordinated way.
- Continue to improve the scope and quality of performance measures and underlying data sources.
- Recommend and/or regulate a specific approach to ensuring performance at each link in the chain between public and private funding sources and patients.
As states begin to develop their children's health insurance programs under the SCHIP program, they will be making many decisions. During the decision-making and implementation phases, the accountability systems that will be developed will need to be comprehensive, and efficient, with effective placement of responsibility and authority (IOM, 1998). These systems of accountability should focus on: ( 1) developing the information infrastructure and capacity to measure coverage (eligibility and enrollment, as well as sources of financing), access (utilization), and quality (technical customer satisfaction); and (2) integrating improved performance information into an approach for ensuring maximum levels of performance by every party on whom the health of American children depends.
References
Digital Equipment Corporation. 1997. HMO Performance Standards, 3rd ed. Maynard, MA: Digital Equipment Corporation.
DHHS (U.S. Department of Health and Human Services). 1990. Healthy People 2000. Washington, D.C.: Public Health Service, U.S. Department of Health and Human Services.
DHHS. 1992. Prevention 91/92. Washington, D.C.: Public Health Service, U.S. Department of Health and Human Services.
DHHS. 1995. Healthy People 2000: Midcourse Review. Washington, D.C.: Public Health Service, U.S. Department of Health and Human Services.
GAO, IMTEC (U.S. General Accounting Office, Information Management and Technology Division). 1993. Automated Medical Records: Leadership Needed to Expedite Standards Development (GAO/IMTEC 93-17). Washington, D.C.: GAO.
IOM (Institute of Medicine). 1989. The Future of Public Health. Washington, D.C.: National Academy Press.
IOM. 1993. Employment and Health Benefits: A Connection at Risk. Washington, D.C.: National Academy Press.
IOM. 1996. Telemedicine: A Guide to Assessing Telecommunications in Health Care. Washington, D.C.: National Academy Press.
IOM. 1997a. Managing Managed Care: Quality Improvement in Behavioral Health. Washington, D.C.: National Academy Press.
IOM. 1997b. Improving Health in the Community: A Role for Performance Monitoring. Washington, D.C.: National Academy Press.
IOM. 1998. Systems of Accountability: Implementing Children's Health Insurance Programs. Washington, D.C.: National Academy Press.
NCHS (National Center for Health Statistics). 1996. Healthy People 2000 Review 1995-96. Hyattsville, MD: Public Health Service, U.S. Department of Health and Human Services.
NCQA (National Committee for Quality Assurance). 1997. HEDIS 3.0 (Health Plan Employer Data and Information Set) [www document]. URL http://www.ncqa.org/hedis.htm
NRC (National Research Council). 1997. Assessment of Performance Measures for Public Health, Substance Abuse, and Mental Health. Washington, D.C.: National Academy Press.