4
Survey Measurement of Food Insecurity and Hunger

This chapter reviews the current measurement of food insecurity, examines the questions used to measure food insecurity and food insecurity with hunger, and identifies several problems in the design of the Household Food Security Survey Module (HFSSM) that should be addressed.

CURRENT APPROACH TO MEASUREMENT

Design of the Current Population Survey

As summarized in Chapter 2, the Food Security Supplement (FSS) is conducted as a supplement to the Current Population Survey (CPS). The FSS was first included in the CPS in April 1995 and has since been administered every year—September 1996, April 1997, August 1998, April 1999, September 2000, April 2001, December 2001, December 2002, December 2003, and December 2004.

The CPS is a representative national sample survey of the civilian, noninstitutional population, conducted monthly by the U.S. Bureau of the Census for the U.S. Department of Labor. Approximately 60,000 households are interviewed each month with data collected on the labor force participation status of approximately 119,000 individuals. The CPS uses a rotating panel of households that are interviewed for four consecutive months, not interviewed for the next eight months, and then interviewed for four additional months, for a total of eight interviews. For any given month, the data represent interviews collected from eight rotation groups, that is, one-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 55
Food Insecurity and Hunger in the United States: An Assessment of the Measure 4 Survey Measurement of Food Insecurity and Hunger This chapter reviews the current measurement of food insecurity, examines the questions used to measure food insecurity and food insecurity with hunger, and identifies several problems in the design of the Household Food Security Survey Module (HFSSM) that should be addressed. CURRENT APPROACH TO MEASUREMENT Design of the Current Population Survey As summarized in Chapter 2, the Food Security Supplement (FSS) is conducted as a supplement to the Current Population Survey (CPS). The FSS was first included in the CPS in April 1995 and has since been administered every year—September 1996, April 1997, August 1998, April 1999, September 2000, April 2001, December 2001, December 2002, December 2003, and December 2004. The CPS is a representative national sample survey of the civilian, noninstitutional population, conducted monthly by the U.S. Bureau of the Census for the U.S. Department of Labor. Approximately 60,000 households are interviewed each month with data collected on the labor force participation status of approximately 119,000 individuals. The CPS uses a rotating panel of households that are interviewed for four consecutive months, not interviewed for the next eight months, and then interviewed for four additional months, for a total of eight interviews. For any given month, the data represent interviews collected from eight rotation groups, that is, one-

OCR for page 55
Food Insecurity and Hunger in the United States: An Assessment of the Measure eighth of the sample is included for the first time, one-eighth is in the sample for the second time, etc. The sample unit is the housing unit; if residents move over the course of the 15 months, the remaining CPS interviews are conducted with the new members of the housing unit. For this reason, the information collected may not represent panel data per se. The initial interview and the fifth interview are face-to-face interviews; the remaining interviews are conducted, if possible, by telephone. The design of the CPS implies that the FSS data are collected across mixed modes; depending on the month-in-sample, the interview may be conducted as a face-to-face or telephone interview. The respondent selected for the CPS is the person who is identified as most knowledgeable concerning the labor force status of the members of the household. Supplements to the CPS labor force questions are included in various months; for example, in March of every year, the supplement focuses on the collection of detailed data on household income, employment, and social assistance program participation. In other months, the supplements may focus on child support payments, ownership of home computers and use of the Internet, or health-related behaviors. As noted above, the FSS is currently collected as a supplement to the CPS in December of each year. Prior to the administration of the FSS, the interviewer determines the most knowledgeable member of the household concerning food that is purchased and eaten by the household and interviews that person. Since moving to the December field date, the rotating panel design of the CPS implies that the FSS is administered to each housing unit twice, one year apart. One caution is that, as stated above, the CPS is a housing unit–based sample; individuals and families are not followed if they move from the CPS selected housing unit. For example, a sampled housing unit for which December 2002 represented the first month-in-sample will be interviewed again in December 2003 (fifth month-in-sample). Hence, changes over a one-year interval can be examined for up to 50 percent of the sampled households that have had no change in their composition. The reinterview of part of the sampled cases allows estimation of within-household variance for the food security measures.1 Researchers in the Economic Research Service (ERS) of USDA are currently using the panel feature of the CPS to look at food insecurity of households as they approach the beginning of a food stamp spell, a period of one or more months during which a household receives food stamps every month. Wilde and 1   By applying appropriate statistical methods, one can account for the intrahousehold variations in food security in the estimated distribution of food insecurity in the population although the sampled household may have been replaced by another household at the same address.

OCR for page 55
Food Insecurity and Hunger in the United States: An Assessment of the Measure Nord (2005) used the 2002–2003 CPS Food Security Supplement panel to estimate the effect of food stamp program participation on food security status. That paper points to some of the inherent problems and suggests some directions for future research. The Food Security Supplement The Food Security Supplement to the CPS includes many questions in addition to the core HFSSM. As described in Chapter 2, it contains a battery of more than 70 questions regarding various aspects of household food use and experiences during the 30 days and 12 months prior to the interview. Appendix A contains the FSS questions (note: not all the follow-up questions are numbered in the set of questions shown in Appendix A). The FSS includes five major sections: Food expenditures. Minimum food spending needed. Food assistance program participation. Food sufficiency and food security (this section includes the 18 food security and hunger questions that are used to calculate the household food security scale). Ways of coping with not having enough food. The content of the FSS varied somewhat in 1996 and 1997, and more substantial revisions in screening and format were introduced in 1998. As described in Chapter 2, households with incomes higher than 185 percent of the federal poverty line and who give no indication of problems with food access or adequacy on either of two preliminary screening questions are deemed to be food secure and are not asked subsequent questions. The two screening questions are: People do different things when they are running out of money for food in order to make their food or their food money go further. In the last 12 months, since December of last year, did you ever run short of money and try to make your food or your food money go further? (This is the first question in the food assistance program participation section in the FSS and Q. 12 in Appendix A.) Which of these statements best describes the food eaten in your household—enough of the kinds of food we want to eat, enough but not always the kinds of food we want to eat, sometimes not enough to eat, or often not enough to eat? (This is the first ques-

OCR for page 55
Food Insecurity and Hunger in the United States: An Assessment of the Measure tion in the food sufficiency and food security section in the FSS and Q. 22 in Appendix A.) As stated in Chapter 2, the HFSSM that is the basis of the food security scale has remained constant in all years. The module consists of 10 questions for households with no children and 18 questions for households with children. These questions inquire about experiences and behaviors of households having difficulty meeting their food needs. These questions focus on: whether the household experienced anxiety over the lack of resources to meet basic food needs, insufficiency in quality of food, and reduced food intake or the feeling of hunger. The questions range in severity of the food security experience from the least severe, of worrying about being able to afford food, to the most severe, of skipping or cutting meals or losing weight because of lack of food. Each question asks about a specific time frame of either the past 12 months or the past 30 days. Separate scales are developed for the different reference periods, although only the 12-month scale is commonly used in household food security analyses. The questions that comprise the household food security scale are shown in Box 2-1. Most of these questions have follow-up questions that are not included as part of the 18 questions used to assess food insecurity. Among households responding to the items shown in Box 2-1, the classification as to food security depends on whether the household includes children. The classification categories and the number of affirmed items necessary to be in a given category are shown in Box 4-1. A primary purpose of the food security measures is to estimate the prevalence of food insecurity in the United States. The U.S. Department of Agriculture (USDA) publishes a report each year summarizing the results of the latest round of the Food Security Supplement.2 Table 4-1 provides estimates of the percentage of households and individuals who are food secure, food insecure without hunger, and food insecure with hunger for the years 1998–2004 based on the CPS survey. 2   The latest in the series is Nord, Andrews, and Carlson (2005b).

OCR for page 55
Food Insecurity and Hunger in the United States: An Assessment of the Measure Box 4-1 Categorization of Food Security Status of Households According to the Number of Affirmed Items on the Food Security Scale Households without children (based on responses to the 10 adult and household items): Food secure = households that denied all items or affirmed 1 or 2 items Food insecure without hunger = households that affirmed 3, 4, or 5 items Food insecure with hunger = households that affirmed 6 or more items Households with children (based on responses to all 18 items): Food secure = households that denied all items or affirmed 1 or 2 items Food insecure without hunger = households that affirmed 3 to 7 items Food insecure with hunger = households that affirmed 8 or more items RELATIONSHIP BETWEEN CONCEPTS AND QUESTIONS The validity of the questions used can be assessed in many ways. Content validity refers to the degree to which the items currently included in a scale represent the various facets of the concept to be measured (see Bohrnstedt, 1983)—in this case, the components of food security, food insecurity, and food insecurity with hunger. This section considers the relationships among the three major categories of food insecurity and hunger (whether the household experienced uncertainty, insufficiency in quality of food, or reduced food intake or the feeling of hunger). 1. Household experience of uncertainty and food depletion (Questions 1 and 2, Box 2-1). The first area of inquiry concerns whether or not the respondent has experienced an “anxiety or perception that the household food budget or food supply was inadequate” (Bickel, Nord, Price, Hamilton, and Cook, 2000). The questions ask about worrying if the food would run out before they got money to buy more, and whether what they bought just

OCR for page 55
Food Insecurity and Hunger in the United States: An Assessment of the Measure TABLE 4-1 Prevalence of Food Security, Food Insecurity Without Hunger, and Food Insecurity with Hunger, by Year (Percentage) Unit Food Secure Food Insecure Without Hunger With Hunger Households   1998 88.2 8.1 3.7 1999 89.9 7.1 3.0 2000 89.5 7.3 3.1 2001 89.3 7.4 3.3 2002 88.9 7.6 3.5 2003 88.8 7.7 3.5 2004 88.1 8.0 3.9 All individuals (by food security status of household)a   1998 86.5 9.8 3.7 1999 88.5 8.6 2.9 2000 87.9 9.0 3.1 2001 87.8 8.9 3.3 2002 87.5 9.1 3.4 2003 87.3 9.3 3.4 2004 86.8 9.5 3.7 Adults (by food security status of household)a   1998 88.8 7.9 3.3 1999 90.5 7.0 2.5 2000 89.9 7.3 2.8 2001 89.8 7.3 3.0 2002 89.5 7.5 3.0 2003 89.2 7.7 3.1 2004 88.7 7.9 3.4 did not last and they did not have money to get more. For both these questions, the respondent was asked if that was often, sometimes, or never true in the last 12 months. 2. Insufficiency in quality or quantity of diet (Questions 3, 11, and 12, Box 2-1). With respect to nutritional adequacy, the HFSSM includes items concerning not being able to afford to eat “balanced meals” for the household and the children. The HFSSM also assesses whether the household relied on “a few kinds of low-cost food” for the children; a parallel item is not included for adults in the household. 3. Reduced food intake or the feeling of hunger (Questions 4, 5, 6, 7, 8, 9, 10, 13–18, Box 2-1). Questions on reduced food intake, skipping meals, and going a whole day without food are asked of adults and children sepa-

OCR for page 55
Food Insecurity and Hunger in the United States: An Assessment of the Measure Unit Food Secure Food Insecure Without Hunger Among Children With Hunger Among Children Households with children   1998 82.4 16.7 0.9 1999 85.2 14.2 0.6 2000 83.8 15.5 0.7 2001 83.9 15.6 0.6 2002 83.5 15.8 0.7 2003 83.3 16.1 0.5 2004 82.4 16.9 0.7 Children (by food security status of household)a   1998 80.3 18.7 1.0 1999 83.1 16.2 0.7 2000 82.0 17.2 0.8 2001 82.4 16.9 0.6 2002 81.9 17.3 0.8 2003 81.8 17.6 0.6 2004 81.0 18.2 0.7 NOTE: Percentages calculated by the Economic Research Service using data from the August 1998, April 1999, September 2000, December 2001, December 2002, and December 2003 Current Population Survey Food Security Supplements. aThe food security survey measures food security status at the household level. Not all individuals residing in food-insecure households are appropriately characterized as food insecure. Similarly, not all individuals in households classified as food insecure with hunger, nor all children in households classified as food insecure with hunger among children, were subject to reductions in food intake or experienced resource-constrained hunger. SOURCE: Nord, Andrews, and Carlson (2005b). rately. The adult questions also ask whether the individual has lost weight. As discussed in Chapter 3, food insecurity is a concept that refers to the social and economic problem of lack of food due to economic deprivation. As such, the concept implies decision making and allocation of resources at the consumer unit, most often the family or the household. The concept implies that the measurement or operationalization of the concept is appropriately applied at the household or family level. Hunger, in contrast, is a distinct concept that is a possible consequence of food insecurity, but it is experienced at the individual level (e.g., painful sensation). Thus, the concept is distinct from food insecurity and the unit of observation is different. Although questions concerning the sensation of hunger are included in the HFSSM, the only adult for whom the information is collected is the respon-

OCR for page 55
Food Insecurity and Hunger in the United States: An Assessment of the Measure dent, and, with respect to children, it is collected by proxy for all children in the household as a group. The Concept of Balanced Meal The concept of “balanced meal” is included in the measurement of the concept of food insecurity based on earlier ethnographic work by Radimer, who describes interviewees expressing concerns about not having specific foods or food groups (as cited in Derrickson, Sakai, and Anderson, 2001). However, evidence from telephone interviews of charitable food recipients in Hawaii as well as previous qualitative work in initial cognitive testing with low-income food “gatekeepers” (who purchased and/or prepared food) shows that the interpretation of the concept of “balanced meal” is neither valid nor reliable as a measure of food insecurity (Derrickson et al., 2001). Frequency and Duration of Food Insecurity As stated in Chapter 3, frequency and duration are important elements for USDA to consider in the concept and operational definition of household food insecurity. USDA’s food security scale measures the severity of food insecurity in surveyed households and classifies their food security status during the previous year. The frequency of food insecurity and the duration of spells of insecurity are not assessed directly in the HFSSM questions used to classify households by food security status. Although some of the response options do offer choices of “often, sometimes, or never,” these response options are not sufficient measures of frequency. In addition, the FSS also includes questions about duration. The questions ask the respondent to estimate the number of days (within the past 30) the phenomenon or behavior was experienced (e.g., in the last 30 days, how many days were you hungry but didn’t eat because you couldn’t afford enough food?). These questions are not used in the 18-item HFSSM, although they have been used in research to estimate the percentage of the population that is food insecure on a given day in a given month. A recent research study undertaken by researchers at the Economic Research Service examined the extent to which food insecurity and hunger are occasional, recurring, or frequent in U.S. households that experience them (Nord, Andrews, and Winicki, 2002). The study analyzes the supplementary data along with the scale and its constituent items using data from the August 1998 CPS supplement. The study found that, on the basis of reported frequency of occurrence of individual items, about two-thirds of the food insecurity and hunger conditions measured by the 12-month scale occurred in 3 or more months of the year. Furthermore, for about one-fifth of the households that experience conditions indicating food insecurity and

OCR for page 55
Food Insecurity and Hunger in the United States: An Assessment of the Measure one-quarter of those reporting hunger-related conditions, occurrences of the reported experiences and behaviors were frequent or chronic, that is, they were reported to have occurred “often” or “in almost every month.” The monthly prevalence of resource-constrained hunger as currently measured by USDA is estimated to have been about 60 percent of the annual prevalence. Among households that reported food insecurity with hunger during a month, around 70 percent of the respondents experienced these conditions in 7 days or fewer; 10 to 20 percent experienced the conditions in 15 days or more. On a typical day in 1998, the prevalence of food insecurity with hunger is estimated to have been 13 to 18 percent of the annual prevalence. The researchers note that although “the analyses add considerably to the understanding of the frequency and duration of food insecurity and hunger, they should be used with some caution. The proportions of food insecurity and hunger that are recurring and frequent or chronic almost certainly vary across subpopulations…. Precise estimation of frequency or chronic incidence would require collection of frequency-of-occurrence information for all items in the scale and creation of a separate scale of frequent or chronic food insecurity and hunger” (Nord et al., 2002, p. 200). Recommendation 4-1: USDA should determine the best way to measure frequency and duration of household food insecurity. Any revised or additional measures should be appropriately tested before implementing in the Household Food Security Survey Module. QUESTION DESIGN ISSUES The focus of this section is on the validity and reliability of the individual questions that comprise the HFSSM. The specific issues are summarized from Dykema and Schaeffer (2005).3 Although the focus is on improving specific items in the HFSSM, the issues outlined in the following discussion are also applicable to the full FSS. Measurement and Item Construction Defining Constrained Economic Resources The concept of constrained economic resources is not consistently referenced in the current set of questions. The focus of the questions is on limited intake and availability of food due to constrained economic re- 3   This information is drawn from the background paper by Dykema and Schaeffer (2005).

OCR for page 55
Food Insecurity and Hunger in the United States: An Assessment of the Measure sources. It is therefore critical for respondents to understand that each of the questions is limited by the constrained economic resource condition. The questionnaire uses several different terms to describe these constraints, including “before we got money to buy more,” “didn’t have/wasn’t enough money,” “couldn’t afford,” and “running out of money.” Providing reference to the condition in a consistent manner throughout the supplement, in contrast, would reduce the burden on the respondent. In addition, the concept should be introduced to the respondent prior to the questions so as to frame the full set of questions. Specification of the Reference Person(s) Cognitive research indicates that questions should be grouped by the topic and reference person of interest. Throughout the FSS, including the HFSSM, the reference unit shifts among the household, the children in the household, the adults in the household, and the reference person. In addition, for many of the questions, the reference person or persons is ambiguous. Both conceptually and analytically, these shifts in the reference person present problems for the respondent as well as in the interpretation of responses. Since food security or insecurity is a household-level phenomenon, the operationalization of the concept should be reflected in household-level questions. Similarly, if the concept of insufficient intake or hunger is an individually experienced phenomenon, questions should address the individual.4 In addition to the conceptual and analytic issues noted above, the specification of the reference person in the Food Security Supplement poses several challenges for respondents: The unit referred to by the question changes across questions. The unit is variously the household (e.g., “Was that often true, sometimes true, or never true for you/your household?”), adults in the household (e.g., “In the last 12 months, did (you/you or other adults in your household) ever cut the size of your meals or skip meals because there wasn’t enough money for food?”), the respondent (e.g., “In the last 12 months, since December of last year, did you ever eat less than you felt you should because there wasn’t enough money for food?” and “In the last 12 4   Although a number of questions ask specifically about the experience of the respondent, the data cannot be used to produce unbiased estimates because the respondent to the supplement is not randomly selected.

OCR for page 55
Food Insecurity and Hunger in the United States: An Assessment of the Measure months, since December of last year, did you ever cut the size of (your child’s/any of the children’s) meals because there wasn’t enough money for food?”), and children within the household (e.g., “In the last 12 months, did (your child/any of the children) ever skip a meal because there wasn’t enough money for food?”). Moreover, the questions (as administered) are not grouped by reference unit but move among the household, the children, adults in the household, and the respondent. The unit referred to changes within question. For example, one of the questions presents the following to the respondent: “The food that (I/we) bought just didn’t last, and (I/we) didn’t have money to get more. Was that often, sometimes, or never true for you in the last 12 months?” In households with more than one adult, the statement uses the plural “we,” but the target question asks about “you.” Although some respondents may infer that “you” refers to “your household,” others may simply shift focus to the easier unit to report about, the respondent himself or herself. The unit is an aggregate unit that implicitly requires that the respondent aggregate or summarize in order to describe the unit. In responding to the statement “(I/We) worried whether (my/our) food would run out before (I/we) got money to buy more. Was that often true, sometimes true, or never true for (you/your household) in the last 12 months?” the respondent must summarize the experience of multiple adults. These experiences may be variable within household—one person may worry sometimes, another may never worry. It is not clear whether the respondent should report that “your household” worries if at least one adult worries or only if all adults worry. Similarly, the question assumes that there is some economic sharing among the adults in the household, so that it makes sense to say, “before we got money to buy more.” It is not clear how many adults would need to “get money to buy more” and which adults they must be willing to share the food they buy with. Many questions ask for proxy reports. The respondent may not know how often all the adults in the household cut the size of meals or skip meals. If the respondent is not the person who feeds the children, she or he may not know how often the size of the children’s meals was cut. Several of the questions in the HFSSM and the FSS appear to use “you” as singular to refer to the respondent themselves (e.g., “did you ever eat less than you

OCR for page 55
Food Insecurity and Hunger in the United States: An Assessment of the Measure felt you should because there wasn’t enough money for food,” “were you ever hungry but didn’t eat because you couldn’t afford enough food?”) This seems appropriate, because the experiences asked about seem unsuited to proxy reports; if “you” in these questions refers to the respondent, then it is inconsistent with the use of “you” as (probably) plural in other FSS questions. The experience of “worry” (“We worried whether our food would run out”) also seems unsuited to proxy reports, but it is asked about as though it is a household-level concept. Reference Periods A large body of empirical literature exists that examines the relationship between length of the reference period and the level of measurement error (see for example, Bound, Brown, and Mathiowetz, 2001). As noted by Schaeffer and Presser (2003), “The choice of reference period is usually determined by the periodicity of the target event, how memorable or patterned the events are likely to be, and the analytic goals of the survey.” In principle, the 12-month reference period currently employed in the FSS allows researchers to estimate the proportion of households that experienced food insecurity in a year, a concept that encompasses seasonal variation. While the motivation to use a 12-month reference period is clear, evidence indicates that the reference period may be too difficult for respondents to implement accurately. For example, analysis of prevalence rates of food insecurity and hunger suggested a seasonality effect such that rates differed depending on whether the survey was fielded in the spring (April) or the fall (September) (Cohen et al., 2002). Although one might expect that episodes of severe food shortage or hunger would be salient and therefore memorable, it is possible that the occurrence of such events is salient, but their frequency, duration, or timing are not reported accurately. This could occur if episodes of hunger occur in stressful contexts that are not conducive to encoding these experiential features. Furthermore, episodes of severe food shortage or hunger (or of constrained resources) may have vague boundaries: it may be easy to say that a time of constrained resources has now become a time of hardship, but it may be difficult to pinpoint exactly when the character of the event changed. Such ambiguities may make it difficult to encode or enumerate events like times of hunger. Two important issues regarding the reference periods included in the questions need to be addressed: (1) the appropriateness of using (for the most part) an annual reference period to evaluate food insecurity and (2) the best way to specify the reference period within the wording of the individual questions. Note that the FSS as administered to the respondent includes not only the 12-month reference period but also 30-day follow-up

OCR for page 55
Food Insecurity and Hunger in the United States: An Assessment of the Measure questions (e.g., did this ever happen in the last 30 days). Therefore, in the administration of the questionnaire, the respondent is constantly being asked to shift between a 12-month and a 30-day reference period. Such shifts are cognitively difficult for respondents and should be avoided if possible. The 12-month reporting period used in the HFSSM questions could reduce the reliability of responses. Findings from an evaluation study conducted after the 1995 survey pretest (Hess and Singer, 1995) indicated that nearly 25 percent of the respondents “failed to understand correctly the time period referred to by the question asking about ‘the last 12 months’.” Moving to a 30-day reference period may reduce the cognitive burden of recalling phenomena for a 12-month period, but, as expected, it could also reduce the percentage of households classified as food insecure. A 30-day scale was originally developed for use in the analysis of the 1995 CPS data, but it has not been used much. Researchers at ERS have revised the scale to make it more consistent with the standard 12-month U.S. food security scale commonly used in food security household analyses. A nonlinear (Rasch model–based) scaling method was used to statistically assess both the original and revised scales (Nord, 2002b). The report of this work examines the feasibility of a 30-day food security scale, and it specifies procedures for calculating the revised 30-day scale from the FSS data and classifying households as to 30-day food security status. It also compares prevalence rates of food insecurity with hunger based on the 30-day scale with those based on the 12-month scale for the years 1998–2000. Nord found that of the 3 percent of households classified as food insecure with hunger based on a 12-month scale, 74 percent (or 2 percent of the households) were similarly classified for the 30-day period prior to the survey.5 The impact of the length of the reference period and confusion about its boundaries on the quality of the resulting data depends partly on the actual organization of the episodes of food insecurity in respondents’ lives. Both the difficulty of the reporting task and the organization of the respondents’ experiences will influence which heuristics respondents use to supplement their memory in constructing answers. For example, a respondent who experiences food shortages monthly because of the timing of income may take a monthly value and multiply by 12 to produce an answer. Another example is suggested by Bickel and colleagues (2000, p. 16), “The U.S. standard food security measure reflects the household’s situation over the 12 months before the interview…. A household that experienced food insecu- 5   In addition, given the current design in which the FSS is asked only as a supplement to the December CPS, a 30-day reference period would limit analysts to examining the period between mid-November and mid-December with respect to food insecurity.

OCR for page 55
Food Insecurity and Hunger in the United States: An Assessment of the Measure rity at some time during the past year (or other period), and therefore is considered food insecure, may in fact be food secure at the time of the interview.” In such a case, the respondent’s beliefs about the stability or level of change in periods of food insecurity or hunger may supplement the respondent’s memory as the answer is constructed (see, for example, Ross and Conway, 1986). Other heuristics that supplement or replace memory when answers are constructed are described in Schwarz (1994) and Tourangeau, Rips, and Rasinski (2000). The items currently included in the HFSSM vary in how the reference period is specified and where in the question the reference is located. In addition, the reference period confusion is exacerbated by the manner in which the questions are actually administered. Response Scales Items in the HFSSM use several formats to record the frequency of the various psychological states or behaviors. Six of the questions in the HFSSM require respondents to rate how often (i.e., often, sometimes, or never) the behavior or psychological state in question was true for them (or in some sort of aggregation across adults in the household). Three follow-up questions ask respondents to assess whether the behavior occurred “almost every month, some months but not every month, or in only 1 or 2 months.” Several of the questions in the FSS use the response options “often true, sometimes true, never true.” For the remaining items, respondents answer using a dichotomous yes/no format. When responses to scale items are converted into numerical scores, both “often” and “sometimes” responses are collapsed. The problems in deciding how often a description of an event is “true” are exacerbated when it is a complex event (so that part of the event described may have happened and part not) and a compound event (which is aggregated over multiple actors, who may have experienced the event with different frequencies). The use of vague quantifiers (often, sometimes) further confounds the interpretation of the response options. Interpretation of Question Wording One of the critical concerns in designing questions is ensuring that respondents interpret terms consistently. Cognitive testing on low-income respondents in upstate New York who were mostly white and black found that most respondents understood the terms of “hungry” or “not eating enough” as intended (“hunger” as a severe problem of decreased food quantity and “not eating enough” as less severe), however, some respondents also associated reduced quality with “not eating enough” (Alaimo, Olson, and Frongillo, 1999). As noted above, respondents may not understand the

OCR for page 55
Food Insecurity and Hunger in the United States: An Assessment of the Measure concept of “balanced meal” as applied, and therefore the question may not be understood in terms of “insufficient quality.” Conclusion Many of the issues outlined in the preceding discussion can be classified under an overarching principle of questionnaire design, that is, the reduction of cognitive burden for the respondent. Consistent use of terminology, clustering questions so as to focus on a specific reference person or reference group (e.g., the respondent, all adults) and on a specific reference period, and developing response options that most closely map to the respondent’s representation of the behavior (or attitude) are all means by which questions can be designed to reduce cognitive burden and, as a result, improve the validity and reliability of the measures. Inevitably, questionnaire design requires balancing multiple intents and principles, and there is no perfect questionnaire design. Nevertheless, the panel concludes that the questions in the HFSSM in particular and the FSS in general can be improved by attending to these design principles as well as possible. Finally, the panel notes that many of the questions included in the FSS are not incorporated in the classification of households as food secure or insecure. However, they are used for research by USDA and other researchers. When developing the food security scale, Hamilton and colleagues (1997a, 1997b) tested some of these questions for inclusion in the scale but decided against using them after testing the scale. If any of the questions are not important for research purposes, they should be deleted from the FSS. However, any changes made to the supplement should be mindful of potential context effects. In reviewing the research, the panel was impressed with the unusually comprehensive program of methodological research conducted in the mid-1990s. That series of studies provides, in many respects, a model on which to ground future research. At the time those studies were conducted, cognitive assessment of questions was undertaken prior to the launching of the 1995 CPS supplement, but the field was not as advanced as it is now (see e.g., Willis, 2004; Presser et al., 2004). USDA should consider cognitive interviews to explore who in the household is the most appropriate person to answer the questions and what topics are appropriate for proxy responding. Following substantial cognitive testing, if a major revision of these items is undertaken, it is then appropriate to focus on improvements to the reliability of the items by simplifying them and the cognitive burden they impose. In addition to cognitive assessment of the individual items, the use of computer-assisted interviewing (either for in-person or telephone interviews)

OCR for page 55
Food Insecurity and Hunger in the United States: An Assessment of the Measure offers a means by which to consolidate questions by topic, reference person, and reference period without overburdening the interviewer. For example, should USDA wish to maintain questions concerning both 12-month and 30-day reference periods, all the 12-month questions could be grouped together. Items that were affirmed for the 12-month reference period could be followed up, focusing exclusively on a 30-day reference period. Recommendation 4-2: USDA should revise the wording and ordering of the questions in the Household Food Security Survey Module. Examples of possible revisions that should be considered include improvements in the consistent treatment of reference periods, reference units, and response options across questions. The revised questions should reflect modern cognitive questionnaire design principles and new data collection technology and should be tested prior to implementation.