Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 45
4
Sampling Frame Development
and Maintenance
THE MASTER ADDRESS FILE
The sampling frame for the American Community Survey (ACS) is based
on the Master Address File (MAF), which is the Census Bureau’s inventory of
known housing units, group quarters (GQ), transitory locations, and selected
nonresidential units in the United States, along with associated information,
such as address, location, and additional attributes. The Census Bureau devel -
oped the MAF in preparation for the 2000 census, with the intention of keeping
it continuously updated during the years between censuses.
The quality of the list is perhaps the single most important aspect of any
list-based data collection approach, because it serves as the foundation on
which all other elements of the survey depend, from sample selection to the
development of controls used to produce the final estimates. In the case of the
ACS, maintaining an up-to-date inventory of GQ facilities has proven to be a
major challenge.
Over the past decade, the primary sources of MAF updates have been
regular “refreshes” from the U.S. Postal Service Delivery Sequence File, which
is the Postal Service’s inventory of mail delivery points. The Demographic Area
Address Listing Files, a system that coordinates various operations related to
the review and automated update of the geographic content of the database,
is another main source of updates. Some updates are generated by clerical
operations, such as the Master Address File Geocoding Office Resolutions, and
some by field observations, such as the Community Address Updating System
in primarily rural areas.
45
OCR for page 46
46 SMALL POPULATIONS, LARGE EFFECTS
The Census Bureau’s current Geographic Support System initiative focuses
on improving address coverage to facilitate a transition from a complete to a
targeted address canvassing operation in preparation for the 2020 census. As
part of this initiative, the Census Bureau established an Address Coverage and
Sources Working Group that focuses on three priority areas: (1) reviewing the
current MAF update processes and recommending areas for improvement; (2)
recommending methods and requirements for maintaining GQ addresses in
the MAF; and (3) working closely with the Partnerships Working Group to
recommend methods to improve MAF coverage using nontraditional address
sources, such as partner-supplied or commercial address data (U.S. Census
Bureau, 2011f).
The fact that GQ addresses will receive special attention as part of the Geo-
graphic Support System initiative is a promising development because, to date,
procedures developed to maintain the MAF have been focused almost exclu -
sively on keeping the inventory of housing units current. Strategies for updating
the inventory of GQ facilities are less comprehensive, and the sources used for
updates related to them are less than adequate. As a result, the relatively poor
quality of GQ records is one of the shortcomings of the MAF.
SAMPLING FRAME DEVELOPMENT FOR
THE AMERICAN COMMUNITY SURVEY
The ACS Office receives the MAF data used to develop a sampling frame
for the American Community Survey in the form of data extracts based on
ACS-specific filtering rules developed to minimize both undercoverage and
overcoverage in the survey. The filtering rules specify what types of addresses
should be included in the extracts, which are delivered to the ACS Office twice
a year. The main sampling operation occurs around August and September of
the year preceding the sample year, and the MAF extract delivered for this is
based on the records available as of July of that year. A second extract is drawn
for the January and February supplemental sampling operation, and this is
designed to update the sample with addresses that have been added to the MAF
since the main extract was drawn.
The GQ sample is selected during the main sampling operation, occurring
during the year preceding the sample year. Because the updating of the GQ
addresses between censuses is operationally difficult and lags behind proce -
dures for updating housing unit addresses, the inventory of group quarters in
the MAF extract contains information primarily from the most recent census,
including the identification of group quarters closed on Census Day, which is
checked again during the ACS fieldwork.
The ACS Office creates its own sampling frame for the GQ population.
The initial GQ sampling frame consisted only of the GQs that were listed in
the MAF extracts. The current procedure used to create the next year’s sam -
OCR for page 47
47
SAMPLING FRAME DEVELOPMENT AND MAINTENANCE
pling frame is to take the current year’s sampling frame and apply updates from
several different sources. Updates include the most recent MAF extracts. The
Federal Bureau of Prisons provides a yearly listing to the ACS Office of all fed -
erally regulated prisons and detention centers. The ACS Office also uses its own
Internet queries to update a variety of group quarters, such as state prisons and
migrant worker camps. As feasible, it also relies on military liaisons to update
the list of military GQ facilities.
Information about the group quarters selected into the sample is further
updated with data obtained from field representatives, who complete a Group
Quarters Facility Questionnaire during their initial visit to a facility. Problems
encountered during field visits are researched by headquarters staff, and this
research often provides new information about the status or location of a facil -
ity. The updates resulting from the fieldwork are used to update the ACS GQ
universe for future samples.
Overall, the number of cases that are added to the GQ sampling frame,
after the MAF extract is produced, is relatively small, but this can vary from
year to year. For example, in 2007 there were 3,060 cases added to the sampling
frame, approximately half of which were based on a file that provided updates
on migrant worker camps (Hefter, 2010). In 2008, the number of updates was
339 (Hefter, 2010).
OPPORTUNITIES FOR INCREASED COLLABORATION
TO IMPROVE THE GQ INVENTORY
The accuracy of the MAF and the sampling frame is a crucial element in
the accuracy of the American Community Survey estimates; however, despite
a variety of updating operations, the current procedures are still inadequate in
terms of maintaining the sampling frame for the GQ population. Given that
the MAF relies heavily on the decennial census, the problem is cyclical to some
extent. The inventory of group quarters is most accurate following the decen -
nial census and becomes less representative of reality later in the decade.
A major reason for the inefficiency in the address update operations seems
to be the inadequate collaboration among the Census Bureau divisions to inte -
grate address updates and corrections resulting from work related to individual
programs carried out by the bureau. Although the MAF was envisioned as a
resource not only for the decennial censuses but also for the Census Bureau’s
other major survey programs, to date, the integration of the MAF with pro -
grams beyond the census has been limited, despite the obvious potential bene -
fits of integration. As a consequence, the ACS Office essentially has to maintain
its own GQ sampling frame for the American Community Survey in an effort
to make up for deficiencies in the MAF. Information flowing in the opposite
direction—from the ACS to the MAF—is minimal at present, although plans
are under way to develop mechanisms for updating the MAF based on some
OCR for page 48
48 SMALL POPULATIONS, LARGE EFFECTS
of the updates performed by the ACS Office as part of its efforts to maintain
the ACS GQ sampling frame.
As described, the Census Bureau has been working on researching and
establishing procedures that will allow for more thorough, continuous updat -
ing of the MAF and TIGER (Topologically Integrated Geographic Encoding
and Referencing), which is the Census Bureau’s digital system that automates
mapping and related geographic activities. This is the right time to consider a
more integrated, agency-wide approach toward the MAF as a complete inven -
tory of living quarters, both housing units and group quarters. Due to the large
number and complexity of updating operations that have been developed over
the years for the census, the ACS, the Census Bureau’s Population Estimates
Program (PEP), and other Census Bureau surveys, it may be tempting to focus
on incremental improvements to further fine-tune the existing procedures.
However, given the central role played by the MAF in so many of the Census
Bureau’s programs, it is imperative to conduct a top-to-bottom assessment of
the MAF and articulate a vision that integrates the work of different Census
Bureau divisions that have an interest in and benefit from the MAF.
Coordination and integration are especially important in the case of the
GQ population because the resources available for updating and maintain -
ing GQ addresses are more limited than the resources available for housing
units. The status of at least some types of group quarters that tend to change
frequently (for example, small group homes) can also be particularly difficult
to track, which is another argument for greater coordination of efforts among
the Census Bureau divisions.
Some units within the Census Bureau have long-standing partnerships
with states and localities and rely on these for local information. However,
these partnerships are often established on the basis of the needs of a specific
program, without maximizing coordination with other Census Bureau units
that may have similar needs. For example, the Federal-State Cooperative Pro -
gram for Population Estimates involves states in assisting the PEP to produce
subnational population estimates. The State Data Center Program is another
partnership between the Census Bureau and the states, which facilitates the
dissemination of data and other assistance to meet local needs. Many state
partners currently provide information related to group quarters to the Census
Bureau. Although this remains rather ad hoc, and while the information is often
provided in a variety of nonuniform formats, the program has the potential for
doing more to meet ACS needs if efforts were better coordinated and incentiv -
ized across the different Census Bureau units.
Working more closely with a large number of states and localities will pres-
ent challenges. Establishing formal agreements with the approximately 39,000
functioning local governments, or even a subset of them, would be a major
undertaking. Data availability varies greatly across local sources, and processing
and standardizing these data may involve substantial resources. A perhaps more
OCR for page 49
49
SAMPLING FRAME DEVELOPMENT AND MAINTENANCE
practical alternative to consider would be heavier reliance on state demographic
offices that maintain their own inventory of group quarters, some using them to
generate their own estimates for state and local geographic areas. These offices
could supply lists of facilities or estimates of the GQ populations as part of a
formal program. These types of agreements may be particularly useful in large
urban areas, where there are more GQ facilities with complex living arrange -
ments (Goldenkoff, 2010).
ACS efforts to collect detailed information from GQ populations are not
unique in the federal statistical system. Several national surveys, including the
Current Population Survey (CPS) conducted by the Census Bureau for the
Bureau of Labor Statistics, include residents of at least some noninstitutional
group quarters in the sample, although it is important to note that the residence
rules used by such surveys as the CPS often differ from the ACS residence rules.
Targeted surveys of specific GQ types exist as well. The National Center
for Health Statistics (NCHS) conducts several surveys of some types of GQ
facilities and their residents. Although resident-level data are collected and
available from these surveys, NCHS typically collects resident data from facility
managers and staff, rather than by directly interviewing residents. The National
Survey of Residential Care Facilities is a new survey conducted for the first
time in 2010 by NCHS. The survey collected data on 8,094 persons residing in
2,302 residential care facilities. NCHS developed the sampling frame for this
survey by relying on lists of licensed residential care facilities. Every few years,
NCHS also conducts a survey of nursing home residents. The most recent
(2004) National Nursing Home Survey collected data on 1,174 facilities and
13,670 residents. The sampling frame for this survey was developed based on
a service provider file from the Centers for Medicare & Medicaid Services and
state licensing lists compiled by a private organization. Another NCHS survey,
the National Home and Hospice Care Survey, involves home health and hos -
pice care agencies. The last survey, conducted in 2007, included 1,036 facilities
and 9,416 patients and hospice discharges. The sampling frame was developed
based on service provider information from the Centers for Medicare & Med -
icaid Services, state licensing lists, and the National Hospice and Palliative Care
Organization file of hospices.
The Bureau of Justice Statistics (BJS) regularly sponsors administrative
censuses of correctional facilities of various types and administers surveys to
their occupants, and the Office of Juvenile Justice and Delinquency Preven -
tion does the same for facilities for juveniles. For example, the Census of Jails
and the Census of Jail Inmates, conducted every 5 years, are focused on locally
administered facilities and their residents. The Census of State and Federal
Adult Correctional Facilities is conducted every 5 to 7 years, and so are the
Survey of Inmates in Federal Correctional Facilities and a Survey of Inmates
in State Correctional Facilities. The Annual Survey of Jails collects data from a
nationally representative sample of jails and inmates. The sample development
OCR for page 50
50 SMALL POPULATIONS, LARGE EFFECTS
and data collection for many of the BJS surveys are performed by the Census
Bureau. BJS also has an administrative records program, which involves work -
ing with states to collect individual-level records of a cross-section of those in
prisons, as well as prisoners admitted to and leaving prison. Although currently
not all states are participating, BJS is working on expanding this program.
Other data collections involving group quarters include the National Cen -
ter for Education Statistics surveys of students, including residents of college
dorms, and regular surveys of military personnel conducted by the Defense
Manpower Data Center in the U.S. Department of Defense. The data col -
lected by these agencies, and possibly others, ought to complement the Census
Bureau’s efforts to maintain and update the sampling frames for certain types of
group quarters, especially given that the Census Bureau often serves as the data
collection contractor for other agencies sponsoring studies of these populations.
Closer collaboration with the Census Bureau would also be beneficial to other
agencies that have to invest significant resources in maintaining the sampling
frames for their surveys.
Given that many group quarters operate as licensed establishments, often
with a maximum number of beds approved, increased collaboration with the
Census Bureau’s economic statistics directorate and the possible use of the
North American Industry Classification System (which classifies business estab-
lishments) could be another avenue for improving the GQ inventory. The panel
is aware that efforts to explore these opportunities by the Census Bureau are
already under way.
Recommendation 4-1: The Census Bureau should give high priority to
developing a detailed and systematic operational plan, with clear timelines
and evaluation benchmarks, for a group quarters (GQ) address updating
system. This should include a plan for greater information sharing and
more efficient information flow between different Census Bureau divisions
and programs to improve the inventory of group quarters in the Master
Address File (MAF). The updating process for the MAF should include
not only the additional information that is acquired by the American Com -
munity Survey Office on some types of group quarters but also information
that is potentially available from other sources, including:
1. the Census Bureau’s Population Estimates Program (PEP), which
obtains updated information on group quarters from state demo-
graphic offices, with varied success—PEP staff should follow up with
every state to obtain information on changes to their GQ inventories,
and the Census Bureau should develop procedures to ensure that the
information is incorporated into the MAF updating process;
2. Census Bureau divisions that develop frames for sampling particular
GQ types for other federal agencies; and
OCR for page 51
51
SAMPLING FRAME DEVELOPMENT AND MAINTENANCE
3. other federal agencies that may have information on particular types
of group quarters.
SCOPE OF COVERAGE
Because of the difficulties associated with maintaining the sampling frame,
the GQ sample contains a relatively high percentage of ineligible cases, which
are identified only during the facility-level data collection phase. This includes
cases that are determined to be housing units instead of group quarters and
group quarters that no longer exist—for example, because the facility has
been closed. As discussed, for a variety of reasons the ACS also classifies some
GQ types (for example, domestic violence shelters, soup kitchens, regularly
scheduled mobile food vans, targeted nonsheltered outdoor locations, crews of
commercial maritime vessels, natural disaster shelters, and dangerous encamp -
ments) as permanently out of scope and excludes them from the sampling
universe.
An additional challenge is introduced by a combination of the data collec -
tion methodology and the seasonal nature of some group quarters. Examples
are college dormitories or military facilities, which are valid GQs but may
have no residents during some of the data collection months in which they
are included in the sample. As discussed later, most sampled GQ facilities are
randomly assigned to an interview month throughout the course of the year,
and some facilities—especially large ones—may be in the sample during more
than one month.
Table 4-1 summarizes the distribution of the main GQ-level outcome codes
for those included in the 2008 ACS sample based on an internal evaluation of
the sampling frame conducted by the Census Bureau (Williams, 2010). A facil -
ity case is considered completed at the GQ level if a field representative collects
basic information and a resident roster from the facility. Once the names of the
residents are collected, the actual respondents can be sampled and approached
for an interview.
Although inability to locate a sampled facility or refusal to participate do
not seem to represent serious problems for the ACS GQ operation, eligible
facilities that are unoccupied at the time of the survey and ineligible cases
make up close to one-quarter of the sample. Tables 4-2 and 4-3, also based
on the Census Bureau’s internal research, show that the rates of cases that fall
into one of these two categories differ considerably by GQ type and size. The
Census Bureau stratifies facilities by size: the small stratum includes group quar-
ters with 15 or fewer residents, as shown on the frame, and the large stratum
includes those with more than 15 residents, as shown on the frame.
Table 4-2 shows that the GQ types with the highest rates of noninterviews
(resulting from the facility being unoccupied at the time of the survey) are
college housing, military facilities, “other institutional facilities,” homeless shel-
OCR for page 52
52 SMALL POPULATIONS, LARGE EFFECTS
TABLE 4-1 Distribution of GQ-Level Outcomes for Facilities Sampled in
2008
Number in Percentage
GQ-Level Outcomes Sample (unweighted)
Completed 13,610 76.4
Refusals, unable to locate, and other nonresponse 37 0.1
Eligible but unoccupied at the time of survey 1,694 9.5
Ineligible 2,482 13.9
No longer exists 928 5.2
Converted to housing unit 1,049 5.9
Domestic violence shelter 17 0.1
Other out of scope 488 2.7
SOURCE: Williams (2010).
ters, and “other noninstitutional facilities.” In the case of college dorms, one
out of four of the large dorms sampled is not interviewed because the facility
is unoccupied at the time of the survey. Table 4-3 illustrates that the sample
includes many ineligible cases as well, especially among small group quarters.
For example, approximately half of the small military facilities, homeless shel -
ters, and nursing homes are found to be ineligible.
As discussed in Chapter 2, small and large group quarters are not sampled
in the same way, which indicates that it is worthwhile to continue research on
whether a cutpoint other than 15 or fewer for the expected number of residents
would be more efficient for defining small facilities. A model relating an indi -
cator of being in scope to the measure of size might be helpful in this regard.
The relatively high rates of ineligible and eligible but unoccupied cases
raise concerns about overcoverage, in addition to the previously discussed
undercoverage issues, such as those resulting from deficiencies in updating the
GQ inventory. The Census Bureau uses population estimates obtained from
the PEP (which we refer to as control totals throughout the report) as auxiliary
estimates to adjust the ACS sample estimates. However, similar to the MAF, the
population estimates are also less accurate for group quarters than for housing
units, because the PEP procedures for group quarters are less thorough than
those developed for housing units. The PEP controls tend to be more accurate
immediately following a decennial census, and they experience a decline in
quality toward the end of the decade.1 As a result, the controls based on the
PEP estimates are not always able to compensate for the problems discussed
above and may in fact lead to biased estimates.
The significant effort spent in the field pursuing GQ facilities that do not
lead to interviews warrants a closer look from a cost-benefit perspective. For
1For more information on how the 2010 PEP estimates compare with the census 2010 estimates
for the total population, see http://www.census.gov/newsroom/releases/pdf/acs_2010_population_
controls.pdf.
OCR for page 53
TABLE 4-2 Distribution of Eligible but Unoccupied Rates by GQ Size for Facilities Sampled in 2008
Percentage of Cases
Unoccupied but
Number of Sample Cases Eligible
Number in Percentage Large Small Large Small
GQ Type Sample of Sample (>15) (>15)
(≤15) (≤15)
Correctional facilities for adults 3,482 19.4 3,373 109 1.1 5.5
Juvenile facilities 330 1.8 241 89 7.5 6.7
Nursing facilities/skilled nursing facilities 4,256 23.8 4,075 181 0.5 1.7
Other institutional facilities 474 2.6 348 126 9.5 22.2
College/university student housing 4,872 27.2 4,672 200 25.2 17.0
Military group quarters 790 4.4 697 93 16.9 15.1
Emergency and transitional shelters 557 3.1 390 167 4.1 13.8
Group homes intended for adults 2,279 12.7 955 1,324 1.9 2.6
Other noninstitutional facilities 872 4.9 356 516 14.3 11.8
Total 17,912 100.0 15,107 2,805 9.8 7.5
SOURCE: Based on tabulations provided by the Census Bureau, August 11, 2010.
53
OCR for page 54
54
TABLE 4-3 Distribution of Ineligible Rates by GQ Size for Facilities Sampled in 2008
Percentage of Cases
Number of Sample Cases Ineligible
Number in Percentage Large Small Large Small
GQ Type Sample of Sample (>15) (>15)
(≤15) (≤15)
Correctional facilities for adults 3,482 19.4 3,373 109 2.8 13.8
Juvenile facilities 330 1.8 241 89 14.5 22.5
Nursing facilities/skilled nursing facilities 4,256 23.8 4,075 181 10.3 48.6
Other institutional facilities 474 2.6 348 126 20.1 28.6
College/university student housing 4,872 27.2 4,672 200 5.1 30.5
Military group quarters 790 4.4 697 93 16.5 55.9
Emergency and transitional shelters 557 3.1 390 167 28.2 52.1
Group homes intended for adults 2,279 12.7 955 1,324 34.3 35.3
Other noninstitutional facilities 872 4.9 356 516 20.5 39.1
Total 17,912 100.0 15,107 2,805 8.3 36.6
SOURCE: Based on tabulations provided by the Census Bureau, August 11, 2010.
OCR for page 55
55
SAMPLING FRAME DEVELOPMENT AND MAINTENANCE
some GQ types, procedures may exist that could be implemented to improve
the frame before sampling, or at least before cases are assigned to field repre -
sentatives. This should be evaluated in the context of the relative costs of the
additional research that would have to be conducted in-house compared with
the costs associated with closing out the cases in the field. It is also important to
consider how the two methods compare in terms of the quality of the informa -
tion available to make a determination about a facility’s status—in other words,
whether one method or the other is less prone to error.
In some cases there is a significant lag between the time when information
is received by the Census Bureau that can be used to update the MAF and the
time the sample is generated (not to mention the time when the fieldwork is
carried out). To reduce the percentages of group quarters that no longer exist
or have been converted to housing units in the sample, this lag time should be
examined to identify possible opportunities for increased efficiency.
One GQ facility type with relatively high rates of no residents is military
facilities. When address updates are received by the Census Bureau, the chal -
lenge often becomes matching information from the different sources and iden-
tifying potential duplicates. More information about the quality of the updates
received from such sources as the Defense Manpower Data Center in the U.S.
Department of Defense is needed to assess whether replacing outdated lists of
military facilities with updates from these sources, without spending additional
resources on matching and reconciliation, may be justified. Naturally, increased
reliance on alternative sources for updates would require that the updates are
performed with adequate frequency.
Recommendation 4-2: The Census Bureau should evaluate, by comparison
with the 2010 census and other data sources, the reasons for the rela -
tively high rates of ineligible and eligible but unoccupied group quarters
(GQ) facilities in the American Community Survey sample and determine
whether there are practical ways to reduce these rates for all or some GQ
types. The evaluation should take into account the costs associated with
determining that a facility is ineligible or unoccupied and how these costs
would change if, for some GQ types, additional in-house research is per-
formed before a case is sent to the field.
SAMPLE REDESIGN OPTIONS
The label “group quarters” encompasses a wide variety of populations,
ranging from inmates in maximum security prisons to people in small group
homes in residential settings. In some sense, the main characteristic that GQ
populations have in common is that they are not part of households (although
some GQ facilities are more comparable to households than others). From
a survey design perspective, the fact that group quarters are not households
OCR for page 56
56 SMALL POPULATIONS, LARGE EFFECTS
makes all aspects of the data collection involving these special populations
more resource intensive. Practical considerations—ranging from the updating
of the sampling frame to interviewer training—typically lead survey designers
to develop different procedures for nonhouseholds, and in the case of most
surveys this usually means excluding them from the population universe. The
ACS includes nearly all group quarters, but the sampling and data collection
procedures are separate for households and nonhouseholds, for the same prac -
tical reasons.
The processes currently in place for updating the GQ inventory are more
likely to identify and remove out-of-business or out-of-scope records from the
ACS sampling frame than to locate and add new records, creating the impres -
sion that the GQ population is shrinking between updates from each decennial
census. Some of the additions are found to be ineligible after they are added to
the sampling frame to “update” it, and sometimes the reason is that the address
is in fact a housing unit.
Anecdotal evidence suggests that it is not unusual for housing units to be
converted to certain types of group quarters, such as group homes for adults,
and then back to housing units over relatively short periods of time. Small
group quarters are disproportionately more likely to be converted to housing
units by the time a field representative visits the premises than larger facilities,
and some GQ types, such as homeless shelters and juvenile detention centers,
are also more susceptible to this type of change (Williams, 2010). In urban
areas, additional challenges are posed by complex housing arrangements, such
as apartment buildings of conventional housing mixed with small group quar-
ters—for example, for populations with special needs.
Duplication between the two samples is also a concern, particularly in the
case of some GQ types. An evaluation of the 2000 census found that duplica -
tion between the GQ and housing unit samples represents a problem for the
enumeration conducted as part of the decennial census as well, especially in
the case of small GQs that are often similar in appearance to households (U.S.
Census Bureau, 2003). Some of this problem is likely to be carried over to the
ACS sampling frames.
Based on current ACS field procedures, data collection for the two samples
is also carried out independently. If, for example, a GQ facility has been con -
verted to a housing unit or if a sampled housing unit turns out to be a GQ
facility, the case is closed out and deleted from the sample as “not a group
quarters facility” or “not a housing unit,” respectively. A recently implemented
change enables field representatives assigned to the housing unit sample to
administer a brief questionnaire if they encounter a GQ facility, with the goal of
determining the GQ type and size (maximum number of people who can stay
at the facility at a particular time). This information enables the ACS Office to
improve the GQ sampling frame for the next round of data collection, which
will increase the efficiency of the updating operations and should also reduce
OCR for page 57
57
SAMPLING FRAME DEVELOPMENT AND MAINTENANCE
costs. Enabling field representatives to collect information about a housing unit
that was included in the GQ sample has also been considered. However, cases
that are encountered in the “wrong” sample are still closed out as ineligible in
both samples, without respondent-level data being collected.
A state-of-the-art MAF, which would be a truly comprehensive and up-
to-date inventory of all living quarters in the United States, would allow the
Census Bureau to step back and consider what a survey of the U.S. population
would look like if the difficulties associated with keeping the sampling frames
current were not one of the primary considerations in the sample design. If
an overarching sampling frame could be developed and maintained (perhaps
treating group quarters as a stratum), then residence in a GQ facility could be
treated similarly to any other population characteristic. The emphasis could
be placed on the real differences associated with GQ type rather than on an
either/or, household/nonhousehold designation.
INTEGRATING THE SAMPLE FOR SOME GQ FACILITIES
WITH THE HOUSING UNIT SAMPLE
Although a major reconceptualization of the GQ classifications may not
be feasible at this time, it is still important to consider the question of whether
the sampling design—which relies on two separate samples, one for housing
units and one for group quarters—is equally efficient for every GQ type. As
discussed, the sampling frame performs particularly poorly for some GQ types,
and keeping the list current will always be more challenging for some types
of group quarters, which tend to go in and out of business or change profile
frequently.
The panel thinks that strictly separating the entire GQ facility sample from
the housing unit sample could be reconsidered. Feedback from data users
about the importance of total population data underscores the benefits of con-
tinuing to include the group quarters in the ACS. However, some GQ types,
or group quarters of a certain size, might sensibly be dropped from the GQ
sample, and instead the data collection for these facilities could be performed
as part of the housing unit data collection. This could be accomplished without
affecting the population universe or modifying the specific GQ categories that
are covered by the ACS. In other words, group quarters that are currently part
of the ACS could continue to be included as group quarters. This approach
would require a closer integration between the two data collections, including
the development of procedures to enable field representatives to collect data
from GQ residents in what are believed to be housing units (beyond what is
currently collected to ascertain GQ type). However, once the integration is
accomplished, the problem of GQ cases being deemed out of scope—at the
cost of substantial fieldwork—because they are in the wrong sample should
be reduced.
OCR for page 58
58 SMALL POPULATIONS, LARGE EFFECTS
Considering that the current approach to maintaining the GQ sampling
frame is inadequate and improving the quality of the sampling frame for some
types of group quarters is likely to be particularly challenging and resource
intensive, dropping some of the GQ types from the GQ sampling frame could
present a cost-effective compromise. This alternative would reduce the need
to keep the inventory up to date for the GQ types that are integrated with the
housing unit sample.
Research will be needed to determine which group quarters are most suit -
able to be moved to the housing unit sample. Some factors to consider include
GQ type, typical size, the extent to which the structure of the facility resembles
a residential housing unit, and the number of ineligible cases encountered in
the category. Within these categories, if there are large, stable subcategories,
those could be kept in the original GQ sample frame. The recent change imple-
mented that allows field representatives working the housing unit sample to col-
lect basic information about group quarters encountered should be especially
useful as part of this research. Another possible source of data to inform this
research is the Group Quarters Validation process that is part of the decennial
census operations; it should provide insight into what types of living quarters
are especially difficult to classify.
The goal is to identify the categories of group quarters that are most dif -
ficult to keep up to date and are most likely to turn up in the housing unit
sample. Generally, small noninstitutional group quarters are most likely to
fit this category. Given that the size of a GQ facility is likely to change more
frequently than the type, a design based on GQ type may be most practical.
Specific possibilities could include moving “other noninstitutional facilities,”
“other institutional facilities,” and “juvenile facilities” to the household sample.
One way to approach this question is to focus on the institutional versus
noninstitutional aspect of the GQ facilities and on comparability with other
national household surveys, such as the Current Population Survey. This would
mean moving only noninstitutional group quarters to the housing unit sample.
“Other noninstitutional facilities” may be an obvious category. The CPS limits
its data collection from group quarters to the noninstitutional civilian popula -
tion and, as such, it excludes most military housing. Not combining “military
quarters” with the household population makes sense for the purposes of
the ACS as well, because military quarters are more similar to institutional
group quarters in terms of the operational considerations applicable to the
data collection. A third category of noninstitutional group quarters that should
be considered is “college/university student housing.” The CPS collects data
about college students at their parents’ address as part of a household inter-
view. Because the goals of the ACS are to provide small-area information that
follows decennial census concepts as much as possible, the ACS follows decen -
nial census rules and surveys residents of student housing where the facility
is located (see Box 4-1). Options could be explored for treating residents of
OCR for page 59
59
SAMPLING FRAME DEVELOPMENT AND MAINTENANCE
BOX 4-1
Residence Rules for College Students
Since 1950, the decennial census has counted college students at their college
location, whether in on-campus or off-campus housing. The Census Bureau’s
authority to make residence rule determinations in the case of college students
and institutional populations was upheld by the U.S. Supreme Court in Borough
of Bethel Park v. Stans (1971). The American Community Survey (ACS) follows
census concepts except that it delimits the census “usual residence” rule to a
2-month (or more) window of time to accord with its continuous month-by-month
data collection.
The ACS 2-month residence rule means that students who reside in on-campus
facilities (or off-campus housing) may or may not be counted at that location
depending on what time of year the facility (or off-campus housing) falls into the
sample. Also, students may be double counted, not only because of misinterpreta-
tion of the rules by respondents, but also because, say, their parents’ (or other)
residence falls into the sample in the summer and their college residence falls into
the sample at another time during the same year, although it is important to remem-
ber that the sample of GQ residents in the ACS is small overall. According to the
ACS residence rules, it is appropriate to count them at both locations. The panel’s
charge did not include a revisiting of the census or ACS residence rules, which
were the subject of a previous panel study (see National Research Council, 2006).
student housing as single-person households. In addition, in the case of large
dormitories that may be closed in the summer, the efficiency of the data col -
lection could be increased by determining the schedule of dormitories based
on information that may be available online or via phone instead of through a
visit by a field representative.
Regardless of which group quarters are moved to the housing unit sam -
ple, the ACS will still be based on two sampling frames after the redesign:
(1) a sampling frame of housing units and “housing-unit like” group quarters
(for simplicity’s sake, referred to here as the housing unit frame) and (2)
a sampling frame of mostly institutional group quarters (referred to as the
GQ sampling frame). The procedures for the latter can remain the same
as they are now. The housing unit sample should be stratified by whether a
case is expected to be a GQ facility or not based on the information on the
sampling frame. This will allow for a desired sampling rate to be set for GQ
facilities. The actual status of a case as a housing unit or group quarters will
be determined after the data are collected, based on the information in the
questionnaire (subsampling procedures would likely have to be applied on
the fly while the interviews are in progress).
It is important to note that the housing unit data collection procedures are
OCR for page 60
60 SMALL POPULATIONS, LARGE EFFECTS
different from the GQ data collection procedures. Sampled housing units are
first mailed a questionnaire, with a request to complete it and return it by mail.
If a telephone number is available, nonresponding housing units receive a tele -
phone follow-up. Finally, a subset of those who did not respond either by mail
or telephone is followed up by an in-person visit from a field representative.
The data collection procedures would have to be adjusted to more closely
integrate the operations for the two samples and to accommodate the group
quarters included in the housing unit sample. The changes would have to be
carefully considered to ensure that they do not have an adverse effect on the
outcome of the housing unit data collection. The panel acknowledges that
these changes are not trivial, but we think that, once implemented, the data
collection process can be seamless. Some of the specific changes required are
discussed below.
First, the mail data collection procedures would have to be modified to
accommodate data collection from GQ residents. One option is to slightly
modify the current housing unit questionnaire and accompanying instruc -
tions to enable their use for GQ residents as well. An alternative would be to
use the information on the sampling frame about whether a unit is a group
quarters or a housing unit (outdated as it may be) to customize the data col -
lection for the initial mail contact with the sample members. In other words,
living quarters that are expected to be housing units will receive the current
housing unit questionnaire and those that are expected to be group quarters
will be mailed the GQ questionnaire. This would reduce concerns that changes
to the housing unit questionnaire to accommodate a small number of group
quarters could increase the burden on household respondents and adversely
affect housing unit response rates. In this case, the difference from the current
procedures would be that the identification of sampling frames for the strata
of group quarters that are moved to the housing unit sample will be based
on the MAF extracts or ACS records from the previous data collection and
on a more integrated, real-time updating system as part of the ongoing ACS
data collection. No additional resources would be invested into keeping the
sampling frames for these categories of group quarters up to date between
data collections.
The procedures for housing units and the group quarters moved to the
housing unit sample could be fully integrated for the follow-up stages, regard -
less of the approach chosen for the mail stage of the data collection. It is
expected that the mail stage nonresponse will be higher among the group
quarters included in the sample than among households because of the higher
questionnaire burden associated with a larger number of residents. It may also
be desirable to follow up with all the group quarters instead of just a subset,
as it is done with the household cases. However, GQ facilities already require
more resource-intensive follow-up than housing units, and the number of cases
requiring follow-up based on the revised procedures should be lower. The costs
OCR for page 61
61
SAMPLING FRAME DEVELOPMENT AND MAINTENANCE
per interview for at least a subset of the group quarters that do respond by mail
will also be lower.
For the telephone and in-person follow-ups, the current housing unit
and GQ data collections will ideally be more closely integrated to allow inter-
viewers to switch between housing unit and group quarters cases, completing
interviews for both. This has implications ranging from interviewer training
to the functionality of the software for both computer-assisted telephone and
computer-assisted personal interviewing, but an integrated operation is a worth-
while investment. An alternative would be to instruct interviewers to refer GQ
facilities encountered during the follow-up to the central office for interviewing
at a later time. However, this would probably mean a more cumbersome and
less cost-effective process.
The sample design changes essentially mean that the status of some of the
living quarters would be determined based on the data collected, as opposed
to a priori, as is done now. The group quarters that end up in the housing
unit sample through this method would still be represented in estimates of
the total population, but the sample sizes for some of the GQ types from the
housing unit sample are likely to be too small to produce reliable results for
these GQ populations separately. However, this is already the case with the
current method as well, and it is typically a problem for very small populations
in other surveys as well. Moreover, as discussed, a small number of GQ types
are already excluded from the ACS data collection for a variety of reasons, and
ACS estimates are controlled to be consistent with the PEP estimates for all
group quarters (U.S. Census Bureau, 2009).
To produce state-level estimates (and possibly estimates for large metropol-
itan areas and congressional districts), data from the group quarters interviewed
as part of the housing unit sample could be combined with all of the other
group quarters, and weights could be applied by GQ type using the state-level
GQ population controls. The Census Bureau could publish characteristics for
the total population, the noninstitutional population, and the GQ population
by type.
To produce estimates below the state level, data from the group quarters
interviewed as part of the housing unit sample could be combined with the
household sample for weighting and tabulation purposes, with the appropri -
ate PEP controls at the county level. Depending on which group quarters are
moved to the housing unit sample, the Census Bureau could publish total
population numbers for three groups––noninstitutional group quarters, insti -
tutional group quarters, and housing units––and provide characteristics for the
total noninstitutional population, which will make the ACS more comparable
to other major household surveys. Alternatively, characteristics could continue
to be provided for the combined total GQ and household populations, as is
currently done.
With the redesigned sample, the ACS could retain the goal of covering
OCR for page 62
62 SMALL POPULATIONS, LARGE EFFECTS
virtually the entire U.S. population, but without having to actively maintain
a sampling frame for some of the group quarters that are particularly difficult
to update. The new design and the closer integration between the procedures
for the two samples could improve the coverage rate for both households and
group quarters, because cases that are in the wrong sample based on the infor-
mation on the sampling frame will no longer have to be removed as ineligible.
Once the logistical details are worked out and a few years’ worth of data col-
lection are completed, the more integrated approach should start showing real
benefits that could ultimately be the key to a better inventory of all GQ types,
which is also necessary for the population controls for small geographic areas.
Recommendation 4-3: To increase effective sample size by more efficiently
targeting resources, the Census Bureau should consider combining the
American Community Survey (ACS) sampling frame for some types of
group quarters (GQ) with the housing unit sampling frame and, in tandem,
modifying its data collection procedures to enable field representatives to
collect data from all cases—housing unit and group quarters—in the com -
bined sample. Additional research will be needed to determine which GQ
types are best suited for integration with the housing unit sample, but the
GQ types that are especially difficult to update and that are most similar
to housing units may be the best candidates. These group quarters could
continue to be included in the ACS GQ universe for purposes of weighting
and estimation.
Recommendation 4-4: For group quarters (GQ) types that are not inte-
grated into the housing unit sampling frame, the Census Bureau should
develop improved and expanded procedures that enable more efficient,
real-time use of status updates received from field representatives. An
operations plan needs to be constructed that allows new GQ facilities to
be added to the Master Address File and changes in the status of exist -
ing addresses to be reported. The Census Bureau should also continue to
pursue the development of procedures that will allow for more efficient
updating of the housing unit sample with cases that have been converted
from group quarters to housing units.
