National Academies Press: OpenBook

Food Insecurity and Hunger in the United States: An Assessment of the Measure (2006)

Chapter: 7 Applicability of Food Insecurity Outcomes for Assessment of Program Performance

« Previous: 6 Survey Vehicles to Measure Food Insecurity and Hunger
Suggested Citation:"7 Applicability of Food Insecurity Outcomes for Assessment of Program Performance." National Research Council. 2006. Food Insecurity and Hunger in the United States: An Assessment of the Measure. Washington, DC: The National Academies Press. doi: 10.17226/11578.
×

7
Applicability of Food Insecurity Outcomes for Assessment of Program Performance

This chapter examines the use of the annual prevalence estimates of food insecurity in the United States for assessing the performance of the U.S. Department of Agriculture’s (USDA) food assistance programs in accordance with the Government Performance and Results Act of 1993. Since specific recommendations and suggestions discussed earlier in the report may lead to changes in the current system, the objective in this chapter is to consider this issue from a broad perspective.

FOOD SECURITY AS A MEASURE OF PROGRAM PERFORMANCE

The Government Performance and Results Act of 1993 (GPRA) requires government agencies to account for the intended results of their activities. Although it provided for strategic planning and managerial accountability, the primary goal of the law was to force a shift in government programs away from process goals (like the number of grants made) to measuring outcomes—whether the intended results of the program were achieved. Federal government agencies were required to “improve program effectiveness and public accountability by promoting a new focus on results, service quality, and customer satisfaction.” The law requires that specific performance goals be established and that annual measurement of these output goals be undertaken to determine the success or failure of the program (Government Performance and Results Act, 1993, Section 2. Findings and Purposes, no. 2).

USDA’s Food and Nutrition Service (FNS) is the agency responsible for the major food assistance programs, including the Food Stamp Program,

Suggested Citation:"7 Applicability of Food Insecurity Outcomes for Assessment of Program Performance." National Research Council. 2006. Food Insecurity and Hunger in the United States: An Assessment of the Measure. Washington, DC: The National Academies Press. doi: 10.17226/11578.
×

the Special Supplemental Nutrition Program for Women, Infants, and Children, and the National School Lunch program. The 2000–2005 strategic plan for FNS states as a goal for the agency, in delivering the food assistance programs, to reduce the prevalence of “food insecurity with hunger” among households with incomes under 130 percent of the federal poverty standard.1 Currently, FNS uses trends in the prevalence of food insecurity with hunger based on the food insecurity module included in the annual Food Security Supplement (FSS) to the Current Population Survey (CPS) as a measure of its annual performance. The panel was asked to comment on the applicability of these data for this purpose.

The mission of FNS is “to increase food security and reduce hunger in partnership with cooperating organizations by providing children and low-income people access to food, a healthful diet, and nutrition education in a manner that supports American agriculture and inspires public confidence.” Legislative language authorizing the Food Stamp Program in the 1977 Food Stamp Act explicitly mentions the alleviation of hunger as a program goal. Therefore, it would seem reasonable to use trends in the prevalence of food insecurity and hunger in low-income populations as performance indicators to assess whether FNS is fulfilling its mission.

Estimates of the prevalence of food insecurity or hunger—when properly designed and measured—can be helpful in determining whether a population is experiencing more or less food insecurity over time. In principle such estimates could also serve as an important tool for identifying trends and levels in food insecurity for specific subgroups of households, such as the elderly, and for different geographic areas, such as rural areas and specific regions or states. Such monitoring efforts are important because they can help to identify where additional assistance may be needed, or where it can be reduced.

The methodological and conceptual issues regarding the measurement of food insecurity and hunger have already been reviewed in this report. There are many actual and possible limitations of the CPS and all other national household surveys in sampling the lowest income households in American society. As explained in Chapter 1, as with most national household population surveys, the CPS routinely excludes people who are institutionalized and those homeless people who cannot be found in households or other living quarters visited during household surveys. The panel recognizes the likelihood of relatively high rates of food insecurity among homeless, and the resulting negative bias resulting from their exclusion. At the

1  

See Wilde (2004) for a discussion of the use of the food insecurity measure for other performance assessment tools—for example, use with the Healthy People 2010 goals.

Suggested Citation:"7 Applicability of Food Insecurity Outcomes for Assessment of Program Performance." National Research Council. 2006. Food Insecurity and Hunger in the United States: An Assessment of the Measure. Washington, DC: The National Academies Press. doi: 10.17226/11578.
×

same time, it has serious questions about the operational and methodological issues. The panel concludes that until better methods to survey the homeless are developed, continuing to limit the target population to the household population seems appropriate. The panel, however, urges USDA to undertake research as part of its long-term agenda, leading to obtaining estimates of food insecurity in periodic or a one-time survey to get a sense of the negative bias of excluding this population in the household survey.

However, even an appropriate measure of food insecurity or hunger using appropriate samples would not be a definitive performance indicator of food assistance programs because their performance is only one of many factors that result in food insecurity. Consequently, changes in food insecurity could be due to many factors other than the performance of the food safety net. For example, if food insecurity declined because the price of food declined, the decline in food insecurity may not indicate better performance of the food safety net, because these programs would not have been responsible for all or part of the change. Conversely, if the reverse were to occur—that is, if food prices were to rise steeply or household income were to fall—the result might be an increase in the number of food-insecure households. But this, too, would not be the result of a decline in the performance of the food safety net. Developments like these could result in errors in assessing the performance of such programs as food stamps that are intended to reduce food insecurity and hunger.

USDA staff and colleagues have studied issues of this kind in a number of ways. They have compared the food security data with information collected in other surveys, such as the Survey of Income and Program Participation (SIPP) and the National Health and Nutrition Examination Survey (NHANES). They are currently using the panel feature of the CPS to look at the food insecurity of households as they approach the beginning of a food stamp spell—a period of one or more months during which a household received food stamps every month. Analysis is still in progress. A recent paper by Wilde and Nord (2005) used the food security data collected in 2002 and 2003 to estimate the effect of Food Stamp Program participation on food security. They used CPS Food Security Supplement data for December 2001 and December 2002 to determine the change in food insecurity with hunger status for food stamp participants who were in the survey both in December 2001 and in December 2002. They found that only 41 percent of the food-insecure households with hunger in 2001 had become food secure in 2002. They concluded that “it appears that unobserved hardships strike from time to time, with large effects on both program participation and food security. These hardships are sufficiently severe to swamp the presumably beneficial direct effect of food stamps on food security” (Wilde and Nord, 2005, pp. 430–431).

To better attribute changes in food insecurity or hunger to the food

Suggested Citation:"7 Applicability of Food Insecurity Outcomes for Assessment of Program Performance." National Research Council. 2006. Food Insecurity and Hunger in the United States: An Assessment of the Measure. Washington, DC: The National Academies Press. doi: 10.17226/11578.
×

safety net would require that trends in these outcomes be supplemented with estimates of changes in need. FNS could annually analyze trends in the prevalence of food insecurity, taking into account economic and other changes that might affect that trend. Ideally, the food safety net would be able to quickly respond to changes in need so as to prevent an increase in food insecurity when need changes. Realistically the food safety net probably cannot fully do this. For example, food stamps cannot help people when food prices increase unless their income declines to make them eligible for food stamps.

The GPRA intended that agencies develop assessment measures that would allow them to assess the efficiency of their programs as well as their effectiveness. Trends in food insecurity or hunger tell us little about changes in the efficiency of FNS. Imagine that changes in food prices or other factors reduce the need for the food safety net, so that food insecurity declines among the poor. The trend in the prevalence of food insecurity would suggest that the food safety net was doing a better job at meeting the need for food assistance. But it is conceivable that prices could drop so much that the poor actually may need less than the safety net offered.

The GPRA emphasis on performance indicators requires agencies to define their goals in quantitative standards related to the purposes of the programs. In this respect, it is important to distinguish performance measurement from program evaluation. Definitions of program measurement and program evaluation promulgated by the U.S. General Accounting Office (1998) identify performance measurement as the ongoing monitoring and reporting of program progress toward preestablished goals. Program evaluations, by contrast, are individual systematic studies conducted periodically or on an ad hoc basis to assess how well a program is working. Program evaluation examines achievement of program objectives in the context of other factors in the program performance that may impede or contribute to its success.

The difficulty facing FNS (and other government agencies)—once having found a way to quantify a program outcome—is that they often must take account of other factors that might affect the need for the program. This can be extremely difficult. For the food safety net programs, this suggests that simple annual trends in food insecurity and hunger would have to be supplemented with estimates of changes in need as well as other possible economic changes that could affect the program participants. It may be difficult, sometimes impossible, to come up with reliable assessment indicators that fulfill the broad intent of GPRA. For example, to assess the effectiveness of the Food Stamp Program, FNS each year would have to analyze trends in the prevalence of food insecurity and hunger among the poor, taking account also of changes in income, food prices and scarcities, and other relevant matters that might affect that trend. The staff of FNS has

Suggested Citation:"7 Applicability of Food Insecurity Outcomes for Assessment of Program Performance." National Research Council. 2006. Food Insecurity and Hunger in the United States: An Assessment of the Measure. Washington, DC: The National Academies Press. doi: 10.17226/11578.
×

already done a considerable amount of micro-level research as they have worked to understand the benefits achieved by the Food Stamp Program. They should continue this research.

The panel concludes that relying on trends in prevalence estimates of food insecurity as a sole indicator of program result is inappropriate. To assess programmatic results, better understanding is needed of the transitions into and out of poverty made by low-income households and the kind of unexpected changes that frequently bring about alterations—for good or bad—in households participating in food assistance programs.

Suggested Citation:"7 Applicability of Food Insecurity Outcomes for Assessment of Program Performance." National Research Council. 2006. Food Insecurity and Hunger in the United States: An Assessment of the Measure. Washington, DC: The National Academies Press. doi: 10.17226/11578.
×
Page 108
Suggested Citation:"7 Applicability of Food Insecurity Outcomes for Assessment of Program Performance." National Research Council. 2006. Food Insecurity and Hunger in the United States: An Assessment of the Measure. Washington, DC: The National Academies Press. doi: 10.17226/11578.
×
Page 109
Suggested Citation:"7 Applicability of Food Insecurity Outcomes for Assessment of Program Performance." National Research Council. 2006. Food Insecurity and Hunger in the United States: An Assessment of the Measure. Washington, DC: The National Academies Press. doi: 10.17226/11578.
×
Page 110
Suggested Citation:"7 Applicability of Food Insecurity Outcomes for Assessment of Program Performance." National Research Council. 2006. Food Insecurity and Hunger in the United States: An Assessment of the Measure. Washington, DC: The National Academies Press. doi: 10.17226/11578.
×
Page 111
Suggested Citation:"7 Applicability of Food Insecurity Outcomes for Assessment of Program Performance." National Research Council. 2006. Food Insecurity and Hunger in the United States: An Assessment of the Measure. Washington, DC: The National Academies Press. doi: 10.17226/11578.
×
Page 112
Next: 8 Closing Remarks »
Food Insecurity and Hunger in the United States: An Assessment of the Measure Get This Book
×
Buy Paperback | $51.00 Buy Ebook | $40.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The United States is viewed by the world as a country with plenty of food, yet not all households in America are food secure, meaning access at all times to enough food for an active, healthy life. A proportion of the population experiences food insecurity at some time in a given year because of food deprivation and lack of access to food due to economic resource constraints. Still, food insecurity in the United States is not of the same intensity as in some developing countries. Since 1995 the U.S. Department of Agriculture (USDA) has annually published statistics on the extent of food insecurity and food insecurity with hunger in U.S. households. These estimates are based on a survey measure developed by the U.S. Food Security Measurement Project, an ongoing collaboration among federal agencies, academic researchers, and private organizations.

USDA requested the Committee on National Statistics of the National Academies to convene a panel of experts to undertake a two-year study in two phases to review at this 10-year mark the concepts and methodology for measuring food insecurity and hunger and the uses of the measure. In Phase 2 of the study the panel was to consider in more depth the issues raised in Phase 1 relating to the concepts and methods used to measure food security and make recommendations as appropriate.

The Committee on National Statistics appointed a panel of 10 experts to examine the above issues. In order to provide timely guidance to USDA, the panel issued an interim Phase 1 report, Measuring Food Insecurity and Hunger: Phase 1 Report. That report presented the panel's preliminary assessments of the food security concepts and definitions; the appropriateness of identifying hunger as a severe range of food insecurity in such a survey-based measurement method; questions for measuring these concepts; and the appropriateness of a household survey for regularly monitoring food security in the U.S. population. It provided interim guidance for the continued production of the food security estimates. This final report primarily focuses on the Phase 2 charge. The major findings and conclusions based on the panel's review and deliberations are summarized.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!