Uses of the CE in Administering Federal Programs: Debriefing of Program Staff
The Panel on Redesigning the BLS Consumer Expenditure Surveys was particularly interested in understanding the use of the CE in administering federal programs to ensure that a redesign would continue to meet those needs. The panel spoke directly with several staff economists whose agencies use the CE data in administering their programs. The purpose of these conversations was to better understand the agency’s use of the CE data, including which data were most important and how they were used. Officials were sent a letter requesting a telephone conversation.
A summary of discussions with staff at the Federal Reserve Board, Centers for Medicare & Medicaid Services of the U.S. Department of Health and Human Services, Economic Research Service of the U.S. Department of Agriculture, and Bureau of Economic Analysis appears below. These staff members spoke about their own experiences and use of the CE and were not providing an official response for their respective agencies.
Mary Kate Catlin, Economist, Centers for Medicare & Medicaid Services
Cathy A. Cowan, Economist, Office of the Actuary/National Health Statistics Group
Kenneth Hanson, Economist, Economic Research Service
Arnold Katz, Bureau of Economic Analysis
Geng Li, Economist, Federal Reserve Board
Marshall Reinsdorf, Bureau of Economic Analysis
Question 1: Do you need total expenditures for each household, as opposed to some expenditure categories collected from one set of households, and other categories collected from another set of households?
Federal Reserve: Dr. Li’s use of the data would be severely impacted by this suggestion. He needs the big picture, how expenditures are related to a balance sheet. He does not know how to most efficiently use the data in this scenario. However, if the CE kept all sections for all households, but did global questions for some sections and detailed questions for others, that might work.
Centers for Medicare & Medicaid Services: Primarily use the health care section as well as income, with possible use of taxes for analysis.
Economic Research Service: Collecting some sections from some households and other sections from other households would make Dr. Hanson a little nervous about being able to do analysis of household food expenditures in the context of expenditures on other categories of goods and services (complete demand system). This survey strategy could work if enough households have at least one question per category, and then go into detail for a subset of sections for different households. He urged trying to get a large enough sample of detailed questions by category so that the analyst can distinguish different types of households, perhaps by household size and income.
Bureau of Economic Analysis: BEA worries that such a plan would eliminate the possibility of looking at substitution effects. However, the problem of underreporting of certain kinds of expenditures is so severe that a strategy of asking each respondent about only a subset of expenditures should be considered. By having sets of overlapping modules in which different respondents report on different combinations of expenditure categories, researchers who are willing to make some assumptions should still be able to study substitution effects.
Question 2: Is the panel nature of the survey important, i.e., do you need to follow households quarterly or is an annual number workable? Related, would fewer collections per household work (e.g., three times per year or twice a year)?
Federal Reserve: The strong preference is for a panel-nature survey, and even a longer panel would be extremely useful for Dr. Li’s work. A modified design where the visits are every six months over two years (same number of collections) has some appeal for Dr. Li.
Centers for Medicare & Medicaid Services: The panel nature is not important, as they deal with annual numbers.
Economic Research Service: Dr. Hanson’s work needs primarily an annual number. The quarterly aspect is not that important.
Bureau of Economic Analysis: Less frequent collection would end up with more boundary problems, mainly because respondents would have trouble recalling expenditures over a longer period. BEA uses quarterly data when they can get it, annual when they cannot.
Question 3: Do you need a complete financial profile for each household, e.g., income, assets, etc., in addition to expenditures?
Federal Reserve: Yes, in fact, Dr. Li and his colleagues are working with BLS to enhance the collection of assets and liabilities. This may mean a reduction in the number of questions on this, but also more accuracy and precision. Dr. Li believes that some of the questions currently being asked should be revised to match the state of household finance. He also needs income to get the complete picture.
Centers for Medicare & Medicaid Services: They do not need great financial details, but staff did express that assets are equally important to income because older people tend to have less income and more assets.
Economic Research Service: Asset data are not used, but better income data would be nice. For example, low-income households often spend more than their income, so something is missed (e.g., alimony, informal income).
Bureau of Economic Analysis: In the final survey for a household, it would be nice to have the complete picture. But the big need for BEA is for interest rates paid on big loans. Assets and liabilities would be nice.
Question 4: How detailed do you need expenses? Would global expenditures work, or if not, what level of detail do you need?
Federal Reserve: The need is more for high-level aggregate data rather than a lot of detail.
Centers for Medicare & Medicaid Services: Staff listed nine major services areas for which they need data. This is quite detailed, but not as detailed as this section gets, e.g., they don’t need data on Blue Cross
Blue Shield (hospital stays, physicians, dentists, drugs, durables, health insurance, etc.).
Economic Research Service: How BLS summarizes the food data can be frustrating, and Dr. Hanson has particular concerns about the consistency of how food data are aggregated over time. He believes there is more detail in the diary than is being summarized, e.g., whole grains versus processed grains. He wants the nutritional aspects of the food to come through.
Bureau of Economic Analysis: For the most part, global expenditures would work, as long as they are accurate. For some modules, BEA would like detail in order to fill in data gaps from other sources. BEA needs to estimate a savings rate.
Question 5: Would other sources of data work for you if the CE data were not available?
Federal Reserve: The CE is unique and cannot be replaced. The PSID does not have the wealth of expenditure data and certainly not the frequency needed by this work.
Centers for Medicare & Medicaid Services: CMS uses MEPS, MCBS, and the Census Service Annual Survey.
Economic Research Service: Dr. Hanson mentioned the USDA National Household Food Acquisition and Purchase Survey (FOODAPS), conducted by Mathematica Policy Research. This survey gives good information, but it has only been funded for its current cycle, and in any best case may be a survey conducted once every five years. The annual nature of the CE is important to Dr. Hanson.
Bureau of Economic Analysis: BEA uses hundreds of sources of data. They use the CE to fill in data gaps. For example, a rental survey conducted by the Census was discontinued in 2001, and the BEA uses CE data to try to substitute for it. The need is, for example, for rent paid for single-family homes. They can get some indication of this from the CE microdata, but it’s not perfect.
Dr. Katz took great pains to explain the BEA’s use of many sources of data. He noted that the BEA is responsible for estimating macro-national totals such as Gross Domestic Product. To do this, they take data from where they can get it.
BEA uses CE data when the agency cannot get better data from other sources, in other words, to fill in the gaps.
BEA goes to more accurate sources of data when it can. Data on big-ticket items are more accurate from the producers, e.g., car sales best come from the auto manufacturers. But for services, these data are more likely to come from household surveys. Often, a survey such as CE is the only source of these kinds of data.
Question 6: Do you have issues with data quality of the current collection methods?
Federal Reserve: For a new user of the data, it is very hard to understand the imputations and allocations. This needs to be better explained and illustrated. There are a number of issues with internal consistency in the units from quarter to quarter. Dr. Li cited the example of a 39-year-old person who, because of an imputation in a subsequent quarter, is all of a sudden 49 years old. Dr. Li also believes that certain sections have more issues than others. For example, mortgage data have greater deficiencies. He would like internal consistency to be improved.
Centers for Medicare & Medicaid Services: Sample size is an issue since CMS produces state-level data and creates estimates for year-to-year changes. There are some level differences when compared to other data sources, but this did not seem to be a big concern.
Economic Research Service: As mentioned above, Dr. Hanson has concerns about the processing of the food data when BLS summarizes the data. He also mentioned a slide from his presentation at the 2010 Data Users Forum on the jumpiness of the year-to-year change in average food at home consumption; the jumpiness exceeds the variance.
Bureau of Economic Analysis: BEA is concerned when the data bounce around. They are concerned that this happens due to the change in panel composition every quarter, and not due to changes in consumer behavior. There are also large discrepancies between the breakdown of total expenditures derived from responses to the CE and expenditure patterns derived from sales reported by retailers and other businesses. BEA staff suspect that the main reason for these discrepancies is underreporting of expenditures by CE respondents. They believe that the sales data obtained from businesses are generally pretty accurate (but they do NOT provide as high a level of detail on the composition of
consumer expenditures as the CE). They also are concerned about the representativeness of the CE sample; in particular, higher income households account for a substantial share of total spending but getting them to answer a complete CE survey is difficult. The once-a-year release of data is a problem; BEA would like it more timely. This point, about increased need for timeliness, was made several times.
Question 7: Do you use microdata or are aggregate data acceptable?
Federal Reserve: The use of the data is primarily at the household (micro) level, but Dr. Li also uses some tabular aggregations.
Centers for Medicare & Medicaid Services: CMS uses both aggregate summaries and microdata.
Economic Research Service: Both are used.
Bureau of Economic Analysis: BEA uses microdata only when they have to in order to build up their own aggregates. They want to stay at the aggregate level. BEA does very little modeling with the CE data. They try to use the microdata as little as possible, but sometimes they don’t get the cross tabulations they need, so they do some aggregations on their own from the microdata.
Question 8: Do you have any suggestions for improvement to the CE data collection?
Federal Reserve: The one strong suggestion is that when someone leaves the panel, Dr. Li would like to know why. Did the household move? Or is it a refusal?
Centers for Medicare & Medicaid Services: Timeliness is an issue. For example, they are working on 2010 estimates and the CE 2010 data won’t be available until October. They would also like more information on household composition, especially older and younger than 65 years old.
Economic Research Service: Dr. Hanson believes that the diary has improved over the years, especially with recording food obtained through SNAP or other benefits. He believes that food-scanning technology is or would be a boon to the quality of the data, especially allowing nutritional content to be recorded alongside the price.
Bureau of Economic Analysis: There is great need for faster turnaround. This is true not only for collected data, but in general, in keeping up with an innovative economy. It can take years to get a new item onto a questionnaire, such as consumer electronics items, and by the time it gets on the questionnaire, it is out-of-date.
Strategies should be developed to reduce underreporting of irregular or infrequent expenditures. Devices to help people remember, such as seeing whether the reported expenditures plus saving add up to after-tax income, should be considered. Also, respondent fatigue due to the length of the survey contributes to underreporting, so breaking the survey into overlapping modules and having a respondent just report on two or three of the modules should be considered.
BEA uses the Census Bureau Economic Censuses, but these are done every five years. In between these collections, they need data to fill in the gaps. For example, the mix of sales at retail establishments can change greatly over five years. BEA might try to use CE data to estimate these mix changes indirectly. Sometimes they are dealing with seven-year-old data.
BEA identified several data gaps they feel that the CE could help with:
- Financial services; the specific services used and the fees paid.
- Data on interest rates of consumer loans, including length of loan. This is especially true of big items such as mortgages and car loans.
- Data on home maintenance and repairs. This is specifically not data on additions and alterations. BEA would like to know how much is spent just on keeping a home running.
- Online purchases, including automated bill payments.
- Keeping up with changes in the economy, including changes in consumer electronics.
Question 9: Are your needs ongoing, or more of an ad hoc use?
Federal Reserve: Dr. Li’s uses of the data are ongoing. There is a long list of papers that could be written, and if the data could be improved (more of a panel nature, longer panel, less inconsistency), there would be even more to do with these data.
Centers for Medicare & Medicaid Services: Uses are ongoing, but there are some special studies.