Providing National Statistics on Health and Social Welfare Programs in an Era of Change: Summary of a Workshop (1998)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

OVERVIEW 1 Overview For decades the federal government administered or set standards for state administration of the major health and social welfare programs—Aid to Families with Dependent Children (AFDC), Supplemental Security Income (SSI), the Food Stamp Program, Medicaid, and Medicare. The federal government also invested in national household surveys and other data systems with which to monitor trends in health and social welfare program participation, assess the effects of programs on a broad range of indicators of societal well-being, and estimate the likely effects of proposed modifications on program costs, caseloads, and outcomes using microsimulation modeling techniques. Such surveys as the March Income Supplement to the Current Population Survey, the National Health Interview Survey, and the Survey of Income and Program Participation were developed and refined over many years to serve these purposes. Now, major changes to health and social welfare programs raise concerns that existing national surveys are no longer adequate for monitoring trends and informing policy debate. In this area, perhaps the most important change is the transfer of substantial responsibility for many programs from federal agencies to state and local governments. Devolution of program responsibility is occurring through legislation—most notably, the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 (PRWORA), which replaced AFDC with a block grant to states, Temporary Assistance to Needy Families (TANF)—and also by encouragement of state experimentation through a process of granting waivers to federal program rules. Program devolution, which gives states considerable leeway in the kinds of benefits and kinds of people they can serve, poses substantial new challenges for data collection and analysis. In addition, legislated changes in social welfare program goals will necessitate new kinds of data for program monitoring and assessment. Notably, while the old AFDC program existed primarily to provide income support, PRWORA sets goals for substantially reducing welfare dependency, including time limits on benefit receipt. Finally, sweeping changes in the health care industry, such as the spread of managed care, as well as other legislative and regulatory changes, are affecting the operation of Medicaid and Medicare in ways that raise questions about the adequacy of existing data for program analysis. In this context the Committee on National Statistics held a workshop in December 1996 to evaluate existing national statistics on health and social welfare programs in light of major new legislation and other changes that are substantially altering the philosophy and operation of the 

OVERVIEW 2 nation's safety net. The workshop was funded by the Office of the Assistant Secretary for Planning and Evaluation in the U.S. Department of Health and Human Services. The workshop brought policy analysts together with statistical agency staff to consider the implications of major changes in health and social welfare programs for national household surveys. The workshop was intended to foster a cooperative effort among agencies to identify data needs for program monitoring and assessment in the new program environment and ways in which to adapt household surveys so that they can continue to provide useful data for program purposes. Through a series of background papers, presentations, and discussions, the workshop participants addressed four broad topics. (The workshop agenda and list of attendees are in Appendix A.) This report summarizes the discussions in each area: • What is changing in health and social welfare programs? • What information is needed? • What are the implications for data collectors? • What statistical and research issues must be addressed? A number of common themes emerged from the workshop and are highlighted in this report. Improved coordination between federal program and statistical agencies is essential to ensure the relevance of national household surveys for monitoring and analysis of health and social welfare programs. To this end, Katherine Wallman, the chief statistician of the United States in the U.S. Office of Management and Budget (OMB), made a commitment to establish an interagency group under OMB to further examine the adequacy of existing surveys and begin to implement necessary changes. Shortly after the conference, OMB did set up a survey coordination mechanism. The changes that are occurring in health and social welfare programs require new or modified survey questions on a wide range of topics. Information is needed to track program participation and benefits, estimate program eligibility, and assess program outcomes. In turn, content changes have implications for questionnaire design and testing, training of interviewers, data editing, and other survey procedures. A comprehensive, regularly updated, accessible database that provides detailed information about program features for states (and localities, where applicable) is essential. Without accurate information on program rules, it will not be possible to use household survey data to accurately estimate program eligibility and participation rates, to develop questions that appropriately reflect program variations, or to assess the quality of survey reports of participation and benefits received. Validating the accuracy of responses is critical to ensure the continued relevance of survey data. For programs established under PRWORA, even more so than for previous programs, participants may not know or understand what benefits they receive. It will be important to develop approaches for validating survey responses, not only to establish the quality of the survey data for program analysis purposes, but also to suggest ways to improve survey reporting. Flexibility in developing surveys is also needed to ensure their continued relevance. Features of health and social welfare programs will likely differ across states, and even across localities within a state, much more than they have in the past. Also program features will likely change over 

OVERVIEW 3 time as states experiment with different approaches. It will be important to develop ways for national household surveys to keep up to date with changes in programs. Modifying the sampling schemes of existing surveys is important to meet data needs for program analysis and monitoring. The devolution of responsibility for social welfare programs from the federal to state governments implies a greater need for state-level estimates. Currently, none of the national surveys provides complete and reliable estimates at the state level. Similarly, since program eligibility criteria may differ for population groups (e.g., immigrants), the existing samples may need modification to provide reliable estimates for particular groups. Even with improved survey data, researchers face methodological challenges in assessing the effects on program participation and other behaviors of major changes in programs. Altered survey content that affects the comparability of measurements is one problem that may be hard to avoid; there is also the problem of estimating how respondents would have behaved if the programs had not changed. The workshop was organized to address two general issues—how to keep national household surveys relevant for program monitoring and analysis and how federal program and statistical agencies can work together more effectively to this end—but workshop participants also acknowledged that other agencies and other data sources are important in an era of program devolution. Thus, partnerships of federal and state agencies will be vitally important, as will communication channels among public agencies, academic researchers, and private organizations, to ensure adequate data collection and analysis of the nation's health and social welfare programs in an era of change. The workshop participants recognized that providing adequate national data for health and social welfare programs in the new environment will involve tradeoffs. For example, data needs may suggest lengthening survey questionnaires, but longer questionnaires may impair public cooperation. Also, limited resources may force difficult choices between expenditures for validation of new questions and expenditures for increased sample sizes, or between expenditures for new questions on programs and expenditures for existing questions on other topics. These and other tradeoffs will require careful consideration by statistical and program agencies. The need to make choices and to develop innovative ways to maximize the use of scarce resources underscores the important role of the newly established interagency coordination mechanism for improving national surveys for health and social welfare programs.