National Academies Press: OpenBook
« Previous: 4 Sampling Frame Development and Maintenance
Suggested Citation:"5 Sample Allocation and Selection." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

5

Sample Allocation and Selection

The goal of the American Community Survey (ACS) is to provide estimates of detailed characteristics of the total population of the United States at levels of geography as small as census block groups, replacing data that were previously obtained through the census long form. The current ACS sample design is optimized to produce substate-level estimates of characteristics of the household population. However, for the group quarters (GQ) population, the design accommodates only state-level estimates of the overall GQ population. The sample design is not adequate for substate GQ estimates, and this also affects the usability of the total population estimates for smaller geographic areas.

As discussed in Chapter 2, the GQ sample consists of two strata, small and large. The small stratum includes facilities with 15 or fewer residents, as shown on the frame, and ones for which the expected number of residents is unknown because the facility was closed on Census Day or because it was recently added to the sampling frame without information about the expected population count. The large stratum includes group quarters with expected populations of more than 15 residents.

The approach to sampling the small stratum is similar to the household sampling method (U.S. Census Bureau, 2009). First, each small facility is randomly assigned to one of five subframes, and sample is selected from each subframe once every 5 years. The facilities in a state are then sorted by small versus closed on Census Day, GQ type, and geographical order, and a systematic sample is selected. In most states, the overall facility sampling rate is approximately 1 in 40, or 2.5 percent in a given year. Some of the less populated states have higher target sampling rates to boost the precision of the state-level estimates.

Suggested Citation:"5 Sample Allocation and Selection." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

For example, the target sampling rate is 7.11 percent for Wyoming and 4.95 percent for Vermont (see Table 5-1). All residents of the selected small facilities are eligible to be interviewed, unless the actual number of residents exceeds 15. In such instances, a subsample of 10 residents is selected when the field representative visits the facility (a process similar to that for the large facilities).

TABLE 5-1 GQ Facility Annual Sampling Rates and Sample Sizes by State, 2009


State Facility Sampling Rate (percentage) Facility Sample Size

Alabama 2.50 2,688
Alaska 5.53 1,033
Arizona 2.50 3,024
Arkansas 2.50 1,861
California 2.50 20,205
Colorado 2.50 2,591
Connecticut 2.50 2,805
Delaware 4.86 1,036
District of Columbia 3.08 1,019
Florida 2.50 9,649
Georgia 2.50 5,626
Hawaii 3.24 983
Idaho 3.48 945
Illinois 2.50 7,432
Indiana 2.50 4,439
Iowa 2.50 2,468
Kansas 2.50 2,002
Kentucky 2.50 3,049
Louisiana 2.50 3,400
Maine 3.14 1,159
Maryland 2.50 3,376
Massachusetts 2.50 5,494
Michigan 2.50 6,796
Minnesota 2.50 3,591
Mississippi 2.50 2,309
Missouri 2.50 4,093
Montana 4.38 1,100
Nebraska 2.50 1,334
Nevada 3.36 1,148
New Hampshire 3.17 1,176
New Jersey 2.50 5,184
New Mexico 3.06 1,033
New York 2.50 13,948
North Carolina 2.50 6,363
North Dakota 4.59 1,103
Ohio 2.50 7,678
Suggested Citation:"5 Sample Allocation and Selection." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

Oklahoma 2.50 2,687
Oregon 2.50 2,193
Pennsylvania 2.50 11,073
Rhode Island 2.75 1,095
South Carolina 2.50 3,488
South Dakota 3.91 1,128
Tennessee 2.50 3,615
Texas 2.50 13,001
Utah 2.79 1,061
Vermont 4.95 1,054
Virginia 2.50 5,853
Washington 2.50 3,287
West Virginia 2.50 1,174
Wisconsin 2.50 3,958
Wyoming 7.11 1,001
Puerto Rico 2.50 960

SOURCE: Based on tabulations provided by the Census Bureau on July 30, 2010.

The sampling units for the large facilities are clusters of GQ residents, who are selected in groups of 10. This means that a large GQ facility is indirectly sampled with probability proportional to size (PPS) measured by its number of anticipated groups of 10 residents. Larger facilities can have several groups of 10 residents represented in the sample. Specifically, groups of residents of large group quarters are sorted by GQ type and geographical order and the groups of residents are then systematically sampled at a rate of approximately 1 in 40 (again, with some exceptions). This means that only group quarters with 40 or more groups of 10 residents are guaranteed to have at least 1 group of residents represented in a particular sample. As described above, the list of residents eligible to be interviewed is determined during the field representative’s visit to the facility. During the visit, an algorithm with a random start is applied to the actual roster of residents. If multiple groups of 10 are selected, the groups are assigned to be interviewed during different months (with some exceptions, in the case of GQ types in which the data collection is concentrated in a shorter period of time for logistical reasons).

When the ACS sample design was first developed, the sampling rate for group quarters was 3 percent of addresses annually, translating into 15 percent over 5 years, but budget constraints resulted in lower sampling rates over the years. This means that a careful look at the sample design is warranted to identify possible opportunities for increased efficiency.

Suggested Citation:"5 Sample Allocation and Selection." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

STATE-LEVEL ALLOCATION

As discussed above, the sample size of small group quarters in a state is proportional to the number of small group quarters on the frame for that state. The sample size of large group quarter residents is proportional to the expected number of residents in large group quarters in the state.

Because the GQ sample is not currently controlled at substate geographies, substate estimates of the combined household and GQ population may be highly variable, a problem that is discussed in more detail in the next section. To address this, the sample design could be modified to better manage (control) the sample allocation rates at the substate level and over time. For 3- and 5-year estimates, the sample could be required to have a minimum number of group quarters in each county over the course of the 5-year period. Some states have a large number of small counties, a situation that represents a challenge, but this change could improve the quality of the data available for small areas.

Another approach would be to individualize the sample further, depending on the characteristics of the small jurisdictions. For example, the lack of control over the allocation rates for smaller geographies may have a large effect on the estimates produced for a community that has 1,000 persons living in households and a correctional facility with 100 residents. According to counts from the 2000 census, places that have 10 percent or more of their population residing in group quarters represent less than 5 percent of all places in the United States. These may be the cases that would need individualized attention.

Additional control over the allocation to substate areas may be facilitated by switching from a PPS design for large group quarters to one in which strata are created on the basis of size and substate area and an equal probability sample is selected within strata. This would permit the allocation to substate areas to be better controlled over time. This type of design would also simplify variance estimation, which appears to be a problem with the current design (Keathley, Navarro, and Asiala, 2010). To determine whether any efficiency would be lost by such a design, the Census Bureau could undertake a study of the effectiveness of the current PPS methods. The expected population numbers in the frame are often incorrect, which reduces the efficiency of PPS sampling. Consequently, the loss in precision from moving from PPS to stratified, equal probability sampling within strata may not be serious.

Recommendation 5-1: The Census Bureau should conduct a formal evaluation of sample redesign strategies that would make it possible to control the American Community Survey group quarters sample allocation at the substate level. The evaluation should focus on identifying options that can improve the precision of the estimates at the state and substate levels, without substantially increasing the costs of the data collection.

Suggested Citation:"5 Sample Allocation and Selection." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

As discussed, there are concerns that the sampling frame is outdated for many of the GQ types, and this includes the number of expected residents in a GQ facility—information that is used in the PPS sample selection. Table 5-2 shows the differences between the observed GQ population based on the 2008 data collection and the GQ population numbers expected from information on the sampling frame, by survey month. Discussions with the Census Bureau indicated that there are variations in the quality of the sampling frame by GQ type. Seasonality could also play a role in the discrepancies in the case of some GQ types. The Census Bureau has been researching the discrepancies between the expected and actual GQ sizes, and this research should continue to better understand the causes of the discrepancies, how they differ by GQ type, and how they affect the PPS sample design.

Recommendation 5-2: The Census Bureau should monitor the accuracy of the measures of size used in the probability proportional to size group quarters (GQ) sample design in the American Community Survey and should assess the resources allocated for updating the GQ sampling frame in the context of how the measures-of-size information available from the sampling frame affects the effectiveness of the sample design.

TABLE 5-2 Differences in 2008 Expected Population and Observed Population


Month Sum of Expected Population Sum of Observed Population Difference in Sum of Expected and Observed Population Percentage Difference in Sum of Expected and Observed Population

January 287,797 270,204 17,593 6.1
February 288,046 278,728 9,318 3.2
March 270,439 252,692 17,747 6.6
April 301,963 287,009 14,954 5.0
May 294,135 265,257 28,878 10.0
June 275,424 235,003 40,421 14.7
July 298,281 248,065 50,216 16.8
August 288,238 250,172 38,066 13.2
September 438,497 266,501 171,996 39.2
October 287,672 276,236 11,436 4.0
November 279,735 264,367 15,368 5.5
December 301,329 282,236 19,093 6.0
Total 3,611,556 3,176,470 435,086 12.0

SOURCE: U.S. Census Bureau (2010).
Suggested Citation:"5 Sample Allocation and Selection." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

Given that some GQ facilities can be very large relative to the size of the household population in a geographic area, capturing them in the sample with certainty may be important. In some respects, data collection from GQ facilities resembles surveys of business populations, which often include a stratum of “must-take” units in the sample. For example, statistical strategies developed for business surveys—including methods to identify units to be in the sample with certainty—may be useful to consider for the ACS.

Clearly, if small-area estimates are important, then, in principle, a sampling design more suitable to address that need would be ideal. However, a must-take approach is often justified when, for example, local experts can identify (domain) estimates that are not reasonable, or there are reasonably foreseeable uses that the survey design either did not or could not account for.

Typically, the development of a must-take stratum is guided by data use considerations, including the data needs of subject-matter specialists who can provide input on whether specific locally significant units must be included in the sample. However, it will be important to carefully monitor the impact of the inclusion of the must-take units on the sample design to ensure that a high number of must-take cases does not excessively distort the design from a more optimal use of resources.

Recommendation 5-3: The Census Bureau should assess whether useful strategies could be learned from other surveys that incorporate a must-take stratum of large units in the sample design and evaluate these strategies for possible use in the sample design for group quarters in the American Community Survey.

SUBSAMPLING WITHIN LARGE GQ FACILITIES

The residents of large group quarters are subsampled in groups of 10, and some group quarters can have multiple groups of 10 in the sample. Given that group quarters provide housing and services to people with similar needs and circumstances, the intraclass correlations within group quarters are naturally high for many variables. Thus, while cost-effective, subsampling a large number of residents in a facility may be statistically inefficient. Reducing the number of persons subsampled in a facility and increasing the number of sample group quarters could improve the reliability of the estimates. This would also mean increased field costs if the number of sampled group quarters has to be increased to achieve the same level of precision of estimates. However, it is also possible that the subsample sizes could be reduced without a substantial loss in precision. If so, there may be no need to increase the number of sampled group quarters. The ideal balance between data quality considerations and cost would have to be evaluated to determine the optimal subsample size.

A recent Census Bureau project calculated the optimal subsample size

Suggested Citation:"5 Sample Allocation and Selection." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

for residents of GQ facilities to be around four after averaging the results of calculations based on two different sets of assumptions about travel costs (Sommers and Hefter, 2010). The question can be approached in a variety of ways, particularly in terms of calculating cost savings. This is one reason why pursuing this research further is important. Future research could also take into consideration possible differences among the intraclass correlations that characterize different GQ types, given that the correlations are presumably not equally high among all of them.

Recommendation 5-4: The Census Bureau should expand on the research it initiated to determine the optimal cluster size for subsampling residents in large group quarters (GQ) in the American Community Survey, estimating intraclass correlations for different variables, and factoring in facility-level and person-level costs using a variety of approaches. The analysis should address whether the same subsample size is efficient for each GQ type and whether the size of the subsample per facility should be reduced.

Suggested Citation:"5 Sample Allocation and Selection." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×

This page intentionally left blank.

Suggested Citation:"5 Sample Allocation and Selection." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 63
Suggested Citation:"5 Sample Allocation and Selection." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 64
Suggested Citation:"5 Sample Allocation and Selection." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 65
Suggested Citation:"5 Sample Allocation and Selection." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 66
Suggested Citation:"5 Sample Allocation and Selection." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 67
Suggested Citation:"5 Sample Allocation and Selection." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 68
Suggested Citation:"5 Sample Allocation and Selection." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 69
Suggested Citation:"5 Sample Allocation and Selection." National Research Council. 2012. Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey. Washington, DC: The National Academies Press. doi: 10.17226/13387.
×
Page 70
Next: 6 Weighting and Estimation »
Small Populations, Large Effects: Improving the Measurement of the Group Quarters Population in the American Community Survey Get This Book
×
Buy Paperback | $44.00 Buy Ebook | $35.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In the early 1990s, the Census Bureau proposed a program of continuous measurement as a possible alternative to the gathering of detailed social, economic, and housing data from a sample of the U.S. population as part of the decennial census. The American Community Survey (ACS) became a reality in 2005, and has included group quarters (GQ)-such places as correctional facilities for adults, student housing, nursing facilities, inpatient hospice facilities, and military barracks-since 2006, primarily to more closely replicate the design and data products of the census long-form sample.

The decision to include group quarters in the ACS enables the Census Bureau to provide a comprehensive benchmark of the total U.S. population (not just those living in households). However, the fact that the ACS must rely on a sample of what is a small and very diverse population, combined with limited funding available for survey operations, makes the ACS GQ sampling, data collection, weighting, and estimation procedures more complex and the estimates more susceptible to problems stemming from these limitations. The concerns are magnified in small areas, particularly in terms of detrimental effects on the total population estimates produced for small areas.

Small Populations, Large Effects provides an in-depth review of the statistical methodology for measuring the GQ population in the ACS. This report addresses difficulties associated with measuring the GQ population and the rationale for including GQs in the ACS. Considering user needs for ACS data and of operational feasibility and compatibility with the treatment of the household population in the ACS, the report recommends alternatives to the survey design and other methodological features that can make the ACS more useful for users of small-area data.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!