National Academies Press: OpenBook
« Previous: 3 American Community Survey Data Products, Data Uses, and Data Needs
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

4

Sampling Frame Development and Maintenance

THE MASTER ADDRESS FILE

The sampling frame for the American Community Survey (ACS) is based on the Master Address File (MAF), which is the Census Bureau’s inventory of known housing units, group quarters (GQ), transitory locations, and selected nonresidential units in the United States, along with associated information, such as address, location, and additional attributes. The Census Bureau developed the MAF in preparation for the 2000 census, with the intention of keeping it continuously updated during the years between censuses.

The quality of the list is perhaps the single most important aspect of any list-based data collection approach, because it serves as the foundation on which all other elements of the survey depend, from sample selection to the development of controls used to produce the final estimates. In the case of the ACS, maintaining an up-to-date inventory of GQ facilities has proven to be a major challenge.

Over the past decade, the primary sources of MAF updates have been regular “refreshes” from the U.S. Postal Service Delivery Sequence File, which is the Postal Service’s inventory of mail delivery points. The Demographic Area Address Listing Files, a system that coordinates various operations related to the review and automated update of the geographic content of the database, is another main source of updates. Some updates are generated by clerical operations, such as the Master Address File Geocoding Office Resolutions, and some by field observations, such as the Community Address Updating System in primarily rural areas.

Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

The Census Bureau’s current Geographic Support System initiative focuses on improving address coverage to facilitate a transition from a complete to a targeted address canvassing operation in preparation for the 2020 census. As part of this initiative, the Census Bureau established an Address Coverage and Sources Working Group that focuses on three priority areas: (1) reviewing the current MAF update processes and recommending areas for improvement; (2) recommending methods and requirements for maintaining GQ addresses in the MAF; and (3) working closely with the Partnerships Working Group to recommend methods to improve MAF coverage using nontraditional address sources, such as partner-supplied or commercial address data (U.S. Census Bureau, 2011f).

The fact that GQ addresses will receive special attention as part of the Geographic Support System initiative is a promising development because, to date, procedures developed to maintain the MAF have been focused almost exclusively on keeping the inventory of housing units current. Strategies for updating the inventory of GQ facilities are less comprehensive, and the sources used for updates related to them are less than adequate. As a result, the relatively poor quality of GQ records is one of the shortcomings of the MAF.

SAMPLING FRAME DEVELOPMENT FOR THE AMERICAN COMMUNITY SURVEY

The ACS Office receives the MAF data used to develop a sampling frame for the American Community Survey in the form of data extracts based on ACS-specific filtering rules developed to minimize both undercoverage and overcoverage in the survey. The filtering rules specify what types of addresses should be included in the extracts, which are delivered to the ACS Office twice a year. The main sampling operation occurs around August and September of the year preceding the sample year, and the MAF extract delivered for this is based on the records available as of July of that year. A second extract is drawn for the January and February supplemental sampling operation, and this is designed to update the sample with addresses that have been added to the MAF since the main extract was drawn.

The GQ sample is selected during the main sampling operation, occurring during the year preceding the sample year. Because the updating of the GQ addresses between censuses is operationally difficult and lags behind procedures for updating housing unit addresses, the inventory of group quarters in the MAF extract contains information primarily from the most recent census, including the identification of group quarters closed on Census Day, which is checked again during the ACS fieldwork.

The ACS Office creates its own sampling frame for the GQ population. The initial GQ sampling frame consisted only of the GQs that were listed in the MAF extracts. The current procedure used to create the next year’s sam-

Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

pling frame is to take the current year’s sampling frame and apply updates from several different sources. Updates include the most recent MAF extracts. The Federal Bureau of Prisons provides a yearly listing to the ACS Office of all federally regulated prisons and detention centers. The ACS Office also uses its own Internet queries to update a variety of group quarters, such as state prisons and migrant worker camps. As feasible, it also relies on military liaisons to update the list of military GQ facilities.

Information about the group quarters selected into the sample is further updated with data obtained from field representatives, who complete a Group Quarters Facility Questionnaire during their initial visit to a facility. Problems encountered during field visits are researched by headquarters staff, and this research often provides new information about the status or location of a facility. The updates resulting from the fieldwork are used to update the ACS GQ universe for future samples.

Overall, the number of cases that are added to the GQ sampling frame, after the MAF extract is produced, is relatively small, but this can vary from year to year. For example, in 2007 there were 3,060 cases added to the sampling frame, approximately half of which were based on a file that provided updates on migrant worker camps (Hefter, 2010). In 2008, the number of updates was 339 (Hefter, 2010).

OPPORTUNITIES FOR INCREASED COLLABORATION TO IMPROVE THE GQ INVENTORY

The accuracy of the MAF and the sampling frame is a crucial element in the accuracy of the American Community Survey estimates; however, despite a variety of updating operations, the current procedures are still inadequate in terms of maintaining the sampling frame for the GQ population. Given that the MAF relies heavily on the decennial census, the problem is cyclical to some extent. The inventory of group quarters is most accurate following the decennial census and becomes less representative of reality later in the decade.

A major reason for the inefficiency in the address update operations seems to be the inadequate collaboration among the Census Bureau divisions to integrate address updates and corrections resulting from work related to individual programs carried out by the bureau. Although the MAF was envisioned as a resource not only for the decennial censuses but also for the Census Bureau’s other major survey programs, to date, the integration of the MAF with programs beyond the census has been limited, despite the obvious potential benefits of integration. As a consequence, the ACS Office essentially has to maintain its own GQ sampling frame for the American Community Survey in an effort to make up for deficiencies in the MAF. Information flowing in the opposite direction—from the ACS to the MAF—is minimal at present, although plans are under way to develop mechanisms for updating the MAF based on some

Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

of the updates performed by the ACS Office as part of its efforts to maintain the ACS GQ sampling frame.

As described, the Census Bureau has been working on researching and establishing procedures that will allow for more thorough, continuous updating of the MAF and TIGER (Topologically Integrated Geographic Encoding and Referencing), which is the Census Bureau’s digital system that automates mapping and related geographic activities. This is the right time to consider a more integrated, agency-wide approach toward the MAF as a complete inventory of living quarters, both housing units and group quarters. Due to the large number and complexity of updating operations that have been developed over the years for the census, the ACS, the Census Bureau’s Population Estimates Program (PEP), and other Census Bureau surveys, it may be tempting to focus on incremental improvements to further fine-tune the existing procedures. However, given the central role played by the MAF in so many of the Census Bureau’s programs, it is imperative to conduct a top-to-bottom assessment of the MAF and articulate a vision that integrates the work of different Census Bureau divisions that have an interest in and benefit from the MAF.

Coordination and integration are especially important in the case of the GQ population because the resources available for updating and maintaining GQ addresses are more limited than the resources available for housing units. The status of at least some types of group quarters that tend to change frequently (for example, small group homes) can also be particularly difficult to track, which is another argument for greater coordination of efforts among the Census Bureau divisions.

Some units within the Census Bureau have long-standing partnerships with states and localities and rely on these for local information. However, these partnerships are often established on the basis of the needs of a specific program, without maximizing coordination with other Census Bureau units that may have similar needs. For example, the Federal-State Cooperative Program for Population Estimates involves states in assisting the PEP to produce subnational population estimates. The State Data Center Program is another partnership between the Census Bureau and the states, which facilitates the dissemination of data and other assistance to meet local needs. Many state partners currently provide information related to group quarters to the Census Bureau. Although this remains rather ad hoc, and while the information is often provided in a variety of nonuniform formats, the program has the potential for doing more to meet ACS needs if efforts were better coordinated and incentivized across the different Census Bureau units.

Working more closely with a large number of states and localities will present challenges. Establishing formal agreements with the approximately 39,000 functioning local governments, or even a subset of them, would be a major undertaking. Data availability varies greatly across local sources, and processing and standardizing these data may involve substantial resources. A perhaps more

Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

practical alternative to consider would be heavier reliance on state demographic offices that maintain their own inventory of group quarters, some using them to generate their own estimates for state and local geographic areas. These offices could supply lists of facilities or estimates of the GQ populations as part of a formal program. These types of agreements may be particularly useful in large urban areas, where there are more GQ facilities with complex living arrangements (Goldenkoff, 2010).

ACS efforts to collect detailed information from GQ populations are not unique in the federal statistical system. Several national surveys, including the Current Population Survey (CPS) conducted by the Census Bureau for the Bureau of Labor Statistics, include residents of at least some noninstitutional group quarters in the sample, although it is important to note that the residence rules used by such surveys as the CPS often differ from the ACS residence rules.

Targeted surveys of specific GQ types exist as well. The National Center for Health Statistics (NCHS) conducts several surveys of some types of GQ facilities and their residents. Although resident-level data are collected and available from these surveys, NCHS typically collects resident data from facility managers and staff, rather than by directly interviewing residents. The National Survey of Residential Care Facilities is a new survey conducted for the first time in 2010 by NCHS. The survey collected data on 8,094 persons residing in 2,302 residential care facilities. NCHS developed the sampling frame for this survey by relying on lists of licensed residential care facilities. Every few years, NCHS also conducts a survey of nursing home residents. The most recent (2004) National Nursing Home Survey collected data on 1,174 facilities and 13,670 residents. The sampling frame for this survey was developed based on a service provider file from the Centers for Medicare & Medicaid Services and state licensing lists compiled by a private organization. Another NCHS survey, the National Home and Hospice Care Survey, involves home health and hospice care agencies. The last survey, conducted in 2007, included 1,036 facilities and 9,416 patients and hospice discharges. The sampling frame was developed based on service provider information from the Centers for Medicare & Medicaid Services, state licensing lists, and the National Hospice and Palliative Care Organization file of hospices.

The Bureau of Justice Statistics (BJS) regularly sponsors administrative censuses of correctional facilities of various types and administers surveys to their occupants, and the Office of Juvenile Justice and Delinquency Prevention does the same for facilities for juveniles. For example, the Census of Jails and the Census of Jail Inmates, conducted every 5 years, are focused on locally administered facilities and their residents. The Census of State and Federal Adult Correctional Facilities is conducted every 5 to 7 years, and so are the Survey of Inmates in Federal Correctional Facilities and a Survey of Inmates in State Correctional Facilities. The Annual Survey of Jails collects data from a nationally representative sample of jails and inmates. The sample development

Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

and data collection for many of the BJS surveys are performed by the Census Bureau. BJS also has an administrative records program, which involves working with states to collect individual-level records of a cross-section of those in prisons, as well as prisoners admitted to and leaving prison. Although currently not all states are participating, BJS is working on expanding this program.

Other data collections involving group quarters include the National Center for Education Statistics surveys of students, including residents of college dorms, and regular surveys of military personnel conducted by the Defense Manpower Data Center in the U.S. Department of Defense. The data collected by these agencies, and possibly others, ought to complement the Census Bureau’s efforts to maintain and update the sampling frames for certain types of group quarters, especially given that the Census Bureau often serves as the data collection contractor for other agencies sponsoring studies of these populations. Closer collaboration with the Census Bureau would also be beneficial to other agencies that have to invest significant resources in maintaining the sampling frames for their surveys.

Given that many group quarters operate as licensed establishments, often with a maximum number of beds approved, increased collaboration with the Census Bureau’s economic statistics directorate and the possible use of the North American Industry Classification System (which classifies business establishments) could be another avenue for improving the GQ inventory. The panel is aware that efforts to explore these opportunities by the Census Bureau are already under way.

Recommendation 4-1: The Census Bureau should give high priority to developing a detailed and systematic operational plan, with clear timelines and evaluation benchmarks, for a group quarters (GQ) address updating system. This should include a plan for greater information sharing and more efficient information flow between different Census Bureau divisions and programs to improve the inventory of group quarters in the Master Address File (MAF). The updating process for the MAF should include not only the additional information that is acquired by the American Community Survey Office on some types of group quarters but also information that is potentially available from other sources, including:

1. the Census Bureau’s Population Estimates Program (PEP), which obtains updated information on group quarters from state demographic offices, with varied success—PEP staff should follow up with every state to obtain information on changes to their GQ inventories, and the Census Bureau should develop procedures to ensure that the information is incorporated into the MAF updating process;

2. Census Bureau divisions that develop frames for sampling particular GQ types for other federal agencies; and

Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

3. other federal agencies that may have information on particular types of group quarters.

SCOPE OF COVERAGE

Because of the difficulties associated with maintaining the sampling frame, the GQ sample contains a relatively high percentage of ineligible cases, which are identified only during the facility-level data collection phase. This includes cases that are determined to be housing units instead of group quarters and group quarters that no longer exist—for example, because the facility has been closed. As discussed, for a variety of reasons the ACS also classifies some GQ types (for example, domestic violence shelters, soup kitchens, regularly scheduled mobile food vans, targeted nonsheltered outdoor locations, crews of commercial maritime vessels, natural disaster shelters, and dangerous encampments) as permanently out of scope and excludes them from the sampling universe.

An additional challenge is introduced by a combination of the data collection methodology and the seasonal nature of some group quarters. Examples are college dormitories or military facilities, which are valid GQs but may have no residents during some of the data collection months in which they are included in the sample. As discussed later, most sampled GQ facilities are randomly assigned to an interview month throughout the course of the year, and some facilities—especially large ones—may be in the sample during more than one month.

Table 4-1 summarizes the distribution of the main GQ-level outcome codes for those included in the 2008 ACS sample based on an internal evaluation of the sampling frame conducted by the Census Bureau (Williams, 2010). A facility case is considered completed at the GQ level if a field representative collects basic information and a resident roster from the facility. Once the names of the residents are collected, the actual respondents can be sampled and approached for an interview.

Although inability to locate a sampled facility or refusal to participate do not seem to represent serious problems for the ACS GQ operation, eligible facilities that are unoccupied at the time of the survey and ineligible cases make up close to one-quarter of the sample. Tables 4-2 and 4-3, also based on the Census Bureau’s internal research, show that the rates of cases that fall into one of these two categories differ considerably by GQ type and size. The Census Bureau stratifies facilities by size: the small stratum includes group quarters with 15 or fewer residents, as shown on the frame, and the large stratum includes those with more than 15 residents, as shown on the frame.

Table 4-2 shows that the GQ types with the highest rates of noninterviews (resulting from the facility being unoccupied at the time of the survey) are college housing, military facilities, “other institutional facilities,” homeless shel-

Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

TABLE 4-1 Distribution of GQ-Level Outcomes for Facilities Sampled in 2008


GQ-Level Outcomes Number in Sample Percentage (unweighted)

Completed 13,610 76.4
Refusals, unable to locate, and other nonresponse 37 0.1
Eligible but unoccupied at the time of survey 1,694 9.5
Ineligible 2,482 13.9
No longer exists 928 5.2
Converted to housing unit 1,049 5.9
Domestic violence shelter 17 0.1
Other out of scope 488 2.7

SOURCE: Williams (2010).

ters, and “other noninstitutional facilities.” In the case of college dorms, one out of four of the large dorms sampled is not interviewed because the facility is unoccupied at the time of the survey. Table 4-3 illustrates that the sample includes many ineligible cases as well, especially among small group quarters. For example, approximately half of the small military facilities, homeless shelters, and nursing homes are found to be ineligible.

As discussed in Chapter 2, small and large group quarters are not sampled in the same way, which indicates that it is worthwhile to continue research on whether a cutpoint other than 15 or fewer for the expected number of residents would be more efficient for defining small facilities. A model relating an indicator of being in scope to the measure of size might be helpful in this regard.

The relatively high rates of ineligible and eligible but unoccupied cases raise concerns about overcoverage, in addition to the previously discussed undercoverage issues, such as those resulting from deficiencies in updating the GQ inventory. The Census Bureau uses population estimates obtained from the PEP (which we refer to as control totals throughout the report) as auxiliary estimates to adjust the ACS sample estimates. However, similar to the MAF, the population estimates are also less accurate for group quarters than for housing units, because the PEP procedures for group quarters are less thorough than those developed for housing units. The PEP controls tend to be more accurate immediately following a decennial census, and they experience a decline in quality toward the end of the decade.1 As a result, the controls based on the PEP estimates are not always able to compensate for the problems discussed above and may in fact lead to biased estimates.

The significant effort spent in the field pursuing GQ facilities that do not lead to interviews warrants a closer look from a cost-benefit perspective. For

1For more information on how the 2010 PEP estimates compare with the census 2010 estimates for the total population, see http://www.census.gov/newsroom/releases/pdf/acs_2010_population_controls.pdf

Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

TABLE 4-2 Distribution of Eligible but Unoccupied Rates by GQ Size for Facilities Sampled in 2008


      Number of Sample Cases Percentage of Cases Unoccupied but Eligible
GQ Type Number in Sample Percentage of Sample Large (>15) Small (<15) Large (>15) Small (<15)

Correctional facilities for adults 3,482 19.4 3,373 109 1.1 5.5
Juvenile facilities 330 1.8 241 89 7.5 6.7
Nursing facilities/skilled nursing facilities 4,256 23.8 4,075 181 0.5 1.7
Other institutional facilities 474 2.6 348 126 9.5 22.2
College/university student housing 4,872 27.2 4,672 200 25.2 17.0
Military group quarters 790 4.4 697 93 16.9 15.1
Emergency and transitional shelters 557 3.1 390 167 4.1 13.8
Group homes intended for adults 2,279 12.7 955 1,324 1.9 2.6
Other noninstitutional facilities 872 4.9 356 516 14.3 11.8
Total 17,912 100.0 15,107 2,805 9.8 7.5

SOURCE: Based on tabulations provided by the Census Bureau, August 11, 2010.
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

TABLE 4-3 Distribution of Ineligible Rates by GQ Size for Facilities Sampled in 2008



      Number of Sample Cases Percentage of Cases Ineligible
GQ Type Number in Sample Percentage of Sample Large
(>15)
Small
(<15)
Large
(>15)
Small
(<15)

Correctional facilities for adults 3,482 19.4 3,373 109 2.8 13.8
Juvenile facilities 330 1.8 241 89 14.5 22.5
Nursing facilities/skilled nursing facilities 4,256 23.8 4,075 181 10.3 48.6
Other institutional facilities 474 2.6 348 126 20.1 28.6
College/university student housing 4,872 27.2 4,672 200 5.1 30.5
Military group quarters 790 4.4 697 93 16.5 55.9
Emergency and transitional shelters 557 3.1 390 167 28.2 52.1
Group homes intended for adults 2,279 12.7 955 1,324 34.3 35.3
Other noninstitutional facilities 872 4.9 356 516 20.5 39.1
Total 17,912 100.0 15,107 2,805 8.3 36.6

SOURCE: Based on tabulations provided by the Census Bureau, August 11, 2010.
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

some GQ types, procedures may exist that could be implemented to improve the frame before sampling, or at least before cases are assigned to field representatives. This should be evaluated in the context of the relative costs of the additional research that would have to be conducted in-house compared with the costs associated with closing out the cases in the field. It is also important to consider how the two methods compare in terms of the quality of the information available to make a determination about a facility’s status—in other words, whether one method or the other is less prone to error.

In some cases there is a significant lag between the time when information is received by the Census Bureau that can be used to update the MAF and the time the sample is generated (not to mention the time when the fieldwork is carried out). To reduce the percentages of group quarters that no longer exist or have been converted to housing units in the sample, this lag time should be examined to identify possible opportunities for increased efficiency.

One GQ facility type with relatively high rates of no residents is military facilities. When address updates are received by the Census Bureau, the challenge often becomes matching information from the different sources and identifying potential duplicates. More information about the quality of the updates received from such sources as the Defense Manpower Data Center in the U.S. Department of Defense is needed to assess whether replacing outdated lists of military facilities with updates from these sources, without spending additional resources on matching and reconciliation, may be justified. Naturally, increased reliance on alternative sources for updates would require that the updates are performed with adequate frequency.

Recommendation 4-2: The Census Bureau should evaluate, by comparison with the 2010 census and other data sources, the reasons for the relatively high rates of ineligible and eligible but unoccupied group quarters (GQ) facilities in the American Community Survey sample and determine whether there are practical ways to reduce these rates for all or some GQ types. The evaluation should take into account the costs associated with determining that a facility is ineligible or unoccupied and how these costs would change if, for some GQ types, additional in-house research is performed before a case is sent to the field.

SAMPLE REDESIGN OPTIONS

The label “group quarters” encompasses a wide variety of populations, ranging from inmates in maximum security prisons to people in small group homes in residential settings. In some sense, the main characteristic that GQ populations have in common is that they are not part of households (although some GQ facilities are more comparable to households than others). From a survey design perspective, the fact that group quarters are not households

Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

makes all aspects of the data collection involving these special populations more resource intensive. Practical considerations—ranging from the updating of the sampling frame to interviewer training—typically lead survey designers to develop different procedures for nonhouseholds, and in the case of most surveys this usually means excluding them from the population universe. The ACS includes nearly all group quarters, but the sampling and data collection procedures are separate for households and nonhouseholds, for the same practical reasons.

The processes currently in place for updating the GQ inventory are more likely to identify and remove out-of-business or out-of-scope records from the ACS sampling frame than to locate and add new records, creating the impression that the GQ population is shrinking between updates from each decennial census. Some of the additions are found to be ineligible after they are added to the sampling frame to “update” it, and sometimes the reason is that the address is in fact a housing unit.

Anecdotal evidence suggests that it is not unusual for housing units to be converted to certain types of group quarters, such as group homes for adults, and then back to housing units over relatively short periods of time. Small group quarters are disproportionately more likely to be converted to housing units by the time a field representative visits the premises than larger facilities, and some GQ types, such as homeless shelters and juvenile detention centers, are also more susceptible to this type of change (Williams, 2010). In urban areas, additional challenges are posed by complex housing arrangements, such as apartment buildings of conventional housing mixed with small group quarters—for example, for populations with special needs.

Duplication between the two samples is also a concern, particularly in the case of some GQ types. An evaluation of the 2000 census found that duplication between the GQ and housing unit samples represents a problem for the enumeration conducted as part of the decennial census as well, especially in the case of small GQs that are often similar in appearance to households (U.S. Census Bureau, 2003). Some of this problem is likely to be carried over to the ACS sampling frames.

Based on current ACS field procedures, data collection for the two samples is also carried out independently. If, for example, a GQ facility has been converted to a housing unit or if a sampled housing unit turns out to be a GQ facility, the case is closed out and deleted from the sample as “not a group quarters facility” or “not a housing unit,” respectively. A recently implemented change enables field representatives assigned to the housing unit sample to administer a brief questionnaire if they encounter a GQ facility, with the goal of determining the GQ type and size (maximum number of people who can stay at the facility at a particular time). This information enables the ACS Office to improve the GQ sampling frame for the next round of data collection, which will increase the efficiency of the updating operations and should also reduce

Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

costs. Enabling field representatives to collect information about a housing unit that was included in the GQ sample has also been considered. However, cases that are encountered in the “wrong” sample are still closed out as ineligible in both samples, without respondent-level data being collected.

A state-of-the-art MAF, which would be a truly comprehensive and up-to-date inventory of all living quarters in the United States, would allow the Census Bureau to step back and consider what a survey of the U.S. population would look like if the difficulties associated with keeping the sampling frames current were not one of the primary considerations in the sample design. If an overarching sampling frame could be developed and maintained (perhaps treating group quarters as a stratum), then residence in a GQ facility could be treated similarly to any other population characteristic. The emphasis could be placed on the real differences associated with GQ type rather than on an either/or, household/nonhousehold designation.

INTEGRATING THE SAMPLE FOR SOME GQ FACILITIES WITH THE HOUSING UNIT SAMPLE

Although a major reconceptualization of the GQ classifications may not be feasible at this time, it is still important to consider the question of whether the sampling design—which relies on two separate samples, one for housing units and one for group quarters—is equally efficient for every GQ type. As discussed, the sampling frame performs particularly poorly for some GQ types, and keeping the list current will always be more challenging for some types of group quarters, which tend to go in and out of business or change profile frequently.

The panel thinks that strictly separating the entire GQ facility sample from the housing unit sample could be reconsidered. Feedback from data users about the importance of total population data underscores the benefits of continuing to include the group quarters in the ACS. However, some GQ types, or group quarters of a certain size, might sensibly be dropped from the GQ sample, and instead the data collection for these facilities could be performed as part of the housing unit data collection. This could be accomplished without affecting the population universe or modifying the specific GQ categories that are covered by the ACS. In other words, group quarters that are currently part of the ACS could continue to be included as group quarters. This approach would require a closer integration between the two data collections, including the development of procedures to enable field representatives to collect data from GQ residents in what are believed to be housing units (beyond what is currently collected to ascertain GQ type). However, once the integration is accomplished, the problem of GQ cases being deemed out of scope—at the cost of substantial fieldwork—because they are in the wrong sample should be reduced.

Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

Considering that the current approach to maintaining the GQ sampling frame is inadequate and improving the quality of the sampling frame for some types of group quarters is likely to be particularly challenging and resource intensive, dropping some of the GQ types from the GQ sampling frame could present a cost-effective compromise. This alternative would reduce the need to keep the inventory up to date for the GQ types that are integrated with the housing unit sample.

Research will be needed to determine which group quarters are most suitable to be moved to the housing unit sample. Some factors to consider include GQ type, typical size, the extent to which the structure of the facility resembles a residential housing unit, and the number of ineligible cases encountered in the category. Within these categories, if there are large, stable subcategories, those could be kept in the original GQ sample frame. The recent change implemented that allows field representatives working the housing unit sample to collect basic information about group quarters encountered should be especially useful as part of this research. Another possible source of data to inform this research is the Group Quarters Validation process that is part of the decennial census operations; it should provide insight into what types of living quarters are especially difficult to classify.

The goal is to identify the categories of group quarters that are most difficult to keep up to date and are most likely to turn up in the housing unit sample. Generally, small noninstitutional group quarters are most likely to fit this category. Given that the size of a GQ facility is likely to change more frequently than the type, a design based on GQ type may be most practical. Specific possibilities could include moving “other noninstitutional facilities,” “other institutional facilities,” and “juvenile facilities” to the household sample.

One way to approach this question is to focus on the institutional versus noninstitutional aspect of the GQ facilities and on comparability with other national household surveys, such as the Current Population Survey. This would mean moving only noninstitutional group quarters to the housing unit sample. “Other noninstitutional facilities” may be an obvious category. The CPS limits its data collection from group quarters to the noninstitutional civilian population and, as such, it excludes most military housing. Not combining “military quarters” with the household population makes sense for the purposes of the ACS as well, because military quarters are more similar to institutional group quarters in terms of the operational considerations applicable to the data collection. A third category of noninstitutional group quarters that should be considered is “college/university student housing.” The CPS collects data about college students at their parents’ address as part of a household interview. Because the goals of the ACS are to provide small-area information that follows decennial census concepts as much as possible, the ACS follows decennial census rules and surveys residents of student housing where the facility is located (see Box 4-1). Options could be explored for treating residents of

Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

BOX 4-1
Residence Rules for College Students

Since 1950, the decennial census has counted college students at their college location, whether in on-campus or off-campus housing. The Census Bureau’s authority to make residence rule determinations in the case of college students and institutional populations was upheld by the U.S. Supreme Court in Borough of Bethel Park v. Stans (1971). The American Community Survey (ACS) follows census concepts except that it delimits the census “usual residence” rule to a 2-month (or more) window of time to accord with its continuous month-by-month data collection.

The ACS 2-month residence rule means that students who reside in on-campus facilities (or off-campus housing) may or may not be counted at that location depending on what time of year the facility (or off-campus housing) falls into the sample. Also, students may be double counted, not only because of misinterpretation of the rules by respondents, but also because, say, their parents’ (or other) residence falls into the sample in the summer and their college residence falls into the sample at another time during the same year, although it is important to remember that the sample of GQ residents in the ACS is small overall. According to the ACS residence rules, it is appropriate to count them at both locations. The panel’s charge did not include a revisiting of the census or ACS residence rules, which were the subject of a previous panel study (see National Research Council, 2006).

student housing as single-person households. In addition, in the case of large dormitories that may be closed in the summer, the efficiency of the data collection could be increased by determining the schedule of dormitories based on information that may be available online or via phone instead of through a visit by a field representative.

Regardless of which group quarters are moved to the housing unit sample, the ACS will still be based on two sampling frames after the redesign: (1) a sampling frame of housing units and “housing-unit like” group quarters (for simplicity’s sake, referred to here as the housing unit frame) and (2) a sampling frame of mostly institutional group quarters (referred to as the GQ sampling frame). The procedures for the latter can remain the same as they are now. The housing unit sample should be stratified by whether a case is expected to be a GQ facility or not based on the information on the sampling frame. This will allow for a desired sampling rate to be set for GQ facilities. The actual status of a case as a housing unit or group quarters will be determined after the data are collected, based on the information in the questionnaire (subsampling procedures would likely have to be applied on the fly while the interviews are in progress).

It is important to note that the housing unit data collection procedures are

Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

different from the GQ data collection procedures. Sampled housing units are first mailed a questionnaire, with a request to complete it and return it by mail. If a telephone number is available, nonresponding housing units receive a telephone follow-up. Finally, a subset of those who did not respond either by mail or telephone is followed up by an in-person visit from a field representative.

The data collection procedures would have to be adjusted to more closely integrate the operations for the two samples and to accommodate the group quarters included in the housing unit sample. The changes would have to be carefully considered to ensure that they do not have an adverse effect on the outcome of the housing unit data collection. The panel acknowledges that these changes are not trivial, but we think that, once implemented, the data collection process can be seamless. Some of the specific changes required are discussed below.

First, the mail data collection procedures would have to be modified to accommodate data collection from GQ residents. One option is to slightly modify the current housing unit questionnaire and accompanying instructions to enable their use for GQ residents as well. An alternative would be to use the information on the sampling frame about whether a unit is a group quarters or a housing unit (outdated as it may be) to customize the data collection for the initial mail contact with the sample members. In other words, living quarters that are expected to be housing units will receive the current housing unit questionnaire and those that are expected to be group quarters will be mailed the GQ questionnaire. This would reduce concerns that changes to the housing unit questionnaire to accommodate a small number of group quarters could increase the burden on household respondents and adversely affect housing unit response rates. In this case, the difference from the current procedures would be that the identification of sampling frames for the strata of group quarters that are moved to the housing unit sample will be based on the MAF extracts or ACS records from the previous data collection and on a more integrated, real-time updating system as part of the ongoing ACS data collection. No additional resources would be invested into keeping the sampling frames for these categories of group quarters up to date between data collections.

The procedures for housing units and the group quarters moved to the housing unit sample could be fully integrated for the follow-up stages, regardless of the approach chosen for the mail stage of the data collection. It is expected that the mail stage nonresponse will be higher among the group quarters included in the sample than among households because of the higher questionnaire burden associated with a larger number of residents. It may also be desirable to follow up with all the group quarters instead of just a subset, as it is done with the household cases. However, GQ facilities already require more resource-intensive follow-up than housing units, and the number of cases requiring follow-up based on the revised procedures should be lower. The costs

Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

per interview for at least a subset of the group quarters that do respond by mail will also be lower.

For the telephone and in-person follow-ups, the current housing unit and GQ data collections will ideally be more closely integrated to allow interviewers to switch between housing unit and group quarters cases, completing interviews for both. This has implications ranging from interviewer training to the functionality of the software for both computer-assisted telephone and computer-assisted personal interviewing, but an integrated operation is a worthwhile investment. An alternative would be to instruct interviewers to refer GQ facilities encountered during the follow-up to the central office for interviewing at a later time. However, this would probably mean a more cumbersome and less cost-effective process.

The sample design changes essentially mean that the status of some of the living quarters would be determined based on the data collected, as opposed to a priori, as is done now. The group quarters that end up in the housing unit sample through this method would still be represented in estimates of the total population, but the sample sizes for some of the GQ types from the housing unit sample are likely to be too small to produce reliable results for these GQ populations separately. However, this is already the case with the current method as well, and it is typically a problem for very small populations in other surveys as well. Moreover, as discussed, a small number of GQ types are already excluded from the ACS data collection for a variety of reasons, and ACS estimates are controlled to be consistent with the PEP estimates for all group quarters (U.S. Census Bureau, 2009).

To produce state-level estimates (and possibly estimates for large metropolitan areas and congressional districts), data from the group quarters interviewed as part of the housing unit sample could be combined with all of the other group quarters, and weights could be applied by GQ type using the state-level GQ population controls. The Census Bureau could publish characteristics for the total population, the noninstitutional population, and the GQ population by type.

To produce estimates below the state level, data from the group quarters interviewed as part of the housing unit sample could be combined with the household sample for weighting and tabulation purposes, with the appropriate PEP controls at the county level. Depending on which group quarters are moved to the housing unit sample, the Census Bureau could publish total population numbers for three groups––noninstitutional group quarters, institutional group quarters, and housing units––and provide characteristics for the total noninstitutional population, which will make the ACS more comparable to other major household surveys. Alternatively, characteristics could continue to be provided for the combined total GQ and household populations, as is currently done.

With the redesigned sample, the ACS could retain the goal of covering

Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

virtually the entire U.S. population, but without having to actively maintain a sampling frame for some of the group quarters that are particularly difficult to update. The new design and the closer integration between the procedures for the two samples could improve the coverage rate for both households and group quarters, because cases that are in the wrong sample based on the information on the sampling frame will no longer have to be removed as ineligible. Once the logistical details are worked out and a few years’ worth of data collection are completed, the more integrated approach should start showing real benefits that could ultimately be the key to a better inventory of all GQ types, which is also necessary for the population controls for small geographic areas.

Recommendation 4-3: To increase effective sample size by more efficiently targeting resources, the Census Bureau should consider combining the American Community Survey (ACS) sampling frame for some types of group quarters (GQ) with the housing unit sampling frame and, in tandem, modifying its data collection procedures to enable field representatives to collect data from all cases—housing unit and group quarters—in the combined sample. Additional research will be needed to determine which GQ types are best suited for integration with the housing unit sample, but the GQ types that are especially difficult to update and that are most similar to housing units may be the best candidates. These group quarters could continue to be included in the ACS GQ universe for purposes of weighting and estimation.

Recommendation 4-4: For group quarters (GQ) types that are not integrated into the housing unit sampling frame, the Census Bureau should develop improved and expanded procedures that enable more efficient, real-time use of status updates received from field representatives. An operations plan needs to be constructed that allows new GQ facilities to be added to the Master Address File and changes in the status of existing addresses to be reported. The Census Bureau should also continue to pursue the development of procedures that will allow for more efficient updating of the housing unit sample with cases that have been converted from group quarters to housing units.

Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 45
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 46
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 47
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 48
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 49
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 50
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 51
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 52
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 53
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 54
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 55
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 56
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 57
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 58
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 59
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 60
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 61
Suggested Citation:"4 Sampling Frame Development and Maintenance." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 62
Next: 5 Sample Allocation and Selection »
Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey Get This Book
×
Buy Paperback | $44.00 Buy Ebook | $35.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In the early 1990s, the Census Bureau proposed a program of continuous measurement as a possible alternative to the gathering of detailed social, economic, and housing data from a sample of the U.S. population as part of the decennial census. The American Community Survey (ACS) became a reality in 2005, and has included group quarters (GQ)-such places as correctional facilities for adults, student housing, nursing facilities, inpatient hospice facilities, and military barracks-since 2006, primarily to more closely replicate the design and data products of the census long-form sample.

The decision to include group quarters in the ACS enables the Census Bureau to provide a comprehensive benchmark of the total U.S. population (not just those living in households). However, the fact that the ACS must rely on a sample of what is a small and very diverse population, combined with limited funding available for survey operations, makes the ACS GQ sampling, data collection, weighting, and estimation procedures more complex and the estimates more susceptible to problems stemming from these limitations. The concerns are magnified in small areas, particularly in terms of detrimental effects on the total population estimates produced for small areas.

Small Populations, Large Effects provides an in-depth review of the statistical methodology for measuring the GQ population in the ACS. This report addresses difficulties associated with measuring the GQ population and the rationale for including GQs in the ACS. Considering user needs for ACS data and of operational feasibility and compatibility with the treatment of the household population in the ACS, the report recommends alternatives to the survey design and other methodological features that can make the ACS more useful for users of small-area data.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!