5
CONTINUOUS MEASUREMENT

Following the early work of Kish (e.g., 1981), there have been several proposals in recent years to extend the collection of census data throughout the decade, rather than restricting it to once every 10 years. Herriott et al. (1989), Horvitz (1986), and Kish (1990) have proposed various data collection schemes that involve this key concept of extending the collection of census data in a more or less continuous fashion. More recently, Alexander (1993) has put forth a relatively detailed proposal for incorporating continuous measurement as part of the 2000 census, and we discuss this proposal below. There are two essential features of the continuous measurement proposals for census data that have been made to date:

  1. virtually continuous data collection operations instead of starting and stopping every 10 years, with ensuing benefits for data quality including maintenance of a permanent enumeration staff and improvement through constant experience; and

  2. an increased frequency of available census data at all points throughout the decade (except for the smallest geographic units for which census data are produced, which might be updated once every 10 years).

In its Design Alternative Recommendation (DAR) #14, the Census Bureau indicates a commitment to fully investigating the feasibility of introducing a continuous measurement program in conjunction with the 2000 census (Bureau of the Census, 1993b:6):

We are fully committed to designing a program which would produce data continuously throughout the decade. Collecting data with such a program would be a fundamental departure from collecting "long form" data from a sample as an integral part of the decennial census. By definition, however, the 1995 Census Test, which is a one-time data collection activity, cannot explicitly address this option. The Census Test will enable us to develop accurate and cost-effective methods for the "year zero" portion of a continuous measurement system.

The panel is encouraged that the Census Bureau plans very serious consideration of the implementation of a continuous measurement program. We believe that this radically different approach to the way in which census data are viewed and collected is worthy of serious investigation, by the Census Bureau and



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 81
A Census that Mirrors America: Interim Report 5 CONTINUOUS MEASUREMENT Following the early work of Kish (e.g., 1981), there have been several proposals in recent years to extend the collection of census data throughout the decade, rather than restricting it to once every 10 years. Herriott et al. (1989), Horvitz (1986), and Kish (1990) have proposed various data collection schemes that involve this key concept of extending the collection of census data in a more or less continuous fashion. More recently, Alexander (1993) has put forth a relatively detailed proposal for incorporating continuous measurement as part of the 2000 census, and we discuss this proposal below. There are two essential features of the continuous measurement proposals for census data that have been made to date: virtually continuous data collection operations instead of starting and stopping every 10 years, with ensuing benefits for data quality including maintenance of a permanent enumeration staff and improvement through constant experience; and an increased frequency of available census data at all points throughout the decade (except for the smallest geographic units for which census data are produced, which might be updated once every 10 years). In its Design Alternative Recommendation (DAR) #14, the Census Bureau indicates a commitment to fully investigating the feasibility of introducing a continuous measurement program in conjunction with the 2000 census (Bureau of the Census, 1993b:6): We are fully committed to designing a program which would produce data continuously throughout the decade. Collecting data with such a program would be a fundamental departure from collecting "long form" data from a sample as an integral part of the decennial census. By definition, however, the 1995 Census Test, which is a one-time data collection activity, cannot explicitly address this option. The Census Test will enable us to develop accurate and cost-effective methods for the "year zero" portion of a continuous measurement system. The panel is encouraged that the Census Bureau plans very serious consideration of the implementation of a continuous measurement program. We believe that this radically different approach to the way in which census data are viewed and collected is worthy of serious investigation, by the Census Bureau and

OCR for page 81
A Census that Mirrors America: Interim Report others. We also understand the reasons why that option is not part of the 1995 census test. We do note, however, that in order for continuous measurement to be a realistic option for the 2000 census, a major effort will be required to evaluate continuous measurement options in parallel with work on the 1995 census test. Since continuous measurement will not be part of the 1995 census test, and because the Census Bureau is still relatively early in its development of a continuous measurement program, the panel has relatively little in the way of recommendations to include in this interim report. We do anticipate having considerable discussion of this approach in our final report. Nevertheless, in view of the interesting possibilities of this option, its radically different approach to census data collection and reporting, and the need to move ahead with planning, we do offer some comments and suggestions based on current developments for continuous measurement. Recommendation 5.1: The Census Bureau should continue to explore the feasibility of a continuous measurement component to the 2000 census. FEATURES AND GOALS OF CONTINUOUS MEASUREMENT Although a number of variants of continuous measurement have already been proposed, and no doubt many more can and will be proposed in the future, there is one fundamental dichotomy among these proposed methods: whether continuous measurement is a replacement for or a supplement to a decennial population count. Kish (1990), for example, writing for an international audience, has proposed a "rolling census" in which there would be no complete enumeration of the population decennially. The Census Bureau's Design Alternative Recommendations (DARs) are clearly based on the presumption that there will be a population enumeration in 2000 and that continuous measurement would substitute only for content not needed for purposes of reapportionment and redistricting. This presumption derives from the view that it is unconstitutional to eliminate the decennial enumeration (and, thus, that a rolling census such as that proposed by Kish could not replace the U.S. decennial census under the prevailing legal view). Since we lack the expertise to evaluate this legal view, we do not challenge the requirement that a continuous measurement program must be designed in conjunction with a decennial enumeration. A legal review prepared by the Congressional Research Service (Lee, 1993) and subsequent work commissioned by the Panel on Census Requirements (see Committee on National Statistics, 1993) also support the position that a decennial enumeration must accompany any program for continuous data collection. This conclusion does have important implications for the merits of the continuous measurement option, however. Specifically, it seems very unlikely that any continuous measurement program would be less expensive than the alternative of collecting all census data at the time of the enumeration. Rather, the most likely benefits of continuous measurement would be improved quality and timeliness for various aspects of enumeration and increased frequency of data collection. (These

OCR for page 81
A Census that Mirrors America: Interim Report potential benefits are discussed in the section below on potential benefits of continuous measurement.) Timeliness refers to the efficiency with which data are collected, processed, and made available to users; frequency refers to the periodicity of summary information from data collection activities. We find this a useful distinction to maintain in our discussion. The panel believes that the Census Bureau needs to establish a research program to determine whether the benefits of continuous measurement will outweigh its costs. Because a continuous measurement program involves many highly interrelated components, it seems necessary to begin a consideration and evaluation of continuous measurement from some realistic and concrete starting point. The panel believes that the prototype by Alexander (1993) provides a very useful starting point for evaluating continuous measurement. Alexander identifies five component features and four goals of continuous measurement. The five features are: continuous improvement frame construction, using a continuously updated master address file to build a structured list of addresses from which survey samples can be drawn; an intercensal long-form survey, based on monthly household surveys, which will produce 5-year moving averages for census tracts and 1-year moving averages for larger geographic areas; an integrated estimates program, which produces population counts at large-area and small-area levels, and works toward integrating administrative records sources into the production of these estimates; a general purpose frame, which is an enhanced continuous improvement frame created after the first decennial enumeration, for use by federal household surveys; and drastic reduction in 2000 census content (to be replaced by intercensal long-form data and estimates). The goals identified by Alexander are: to improve the quality of the enumeration of the 2000 census, by eliminating the distraction and burden of collecting content data; to produce small-area data with sampling error comparable to 1990 long-form data, but to make these estimates more frequently than once per decade; to provide a platform for integrating data from the decennial census, household surveys, demographic estimates, the TIGER system, and administrative records, thereby eliminating duplication of effort; and to take advantage of the continuous nature of the process in order to allow continuous improvements in the quality of operations, make more effective use of experienced staff, and efficiently share data collection resources across different programs.

OCR for page 81
A Census that Mirrors America: Interim Report The panel endorses the proposal that efforts be made to integrate census operations into the Census Bureau's full spectrum of household data collection activities. We also endorse the goals laid out by Alexander. We believe that the Census Bureau's next steps should be to carry out a systematic program of research to determine the extent to which these goals are achievable. Recommendation 5.2: The Census Bureau should establish a formal set of goals for a continuous measurement program. The Census Bureau should then establish a research plan to determine the extent to which those goals are achievable. Alexander's prototype calls for a sample of 250,000 housing units per month, to be contacted initially by mail. For a 50 percent subsample of the nonrespondents to the mail survey, a follow-up by telephone would be attempted. Of those in this subsample for whom no telephone contact could be established, a further 50 percent would be subsampled for a personal visit interview. This use of multiple modes of data collection is aimed at achieving the necessary level of response with efficient use of resources. Because the primary response mode in Alexander's prototype would be by mail, it would not be possible to integrate the continuous collection of census data with other household surveys conducted by the Census Bureau. Those surveys are invariably too complex to conduct by mail, while the use of mail to collect census data is undoubtedly a necessity on cost grounds. However, the continuous measurement program would result in direct benefits and cost savings for household surveys by providing a continuously updated, high-quality frame, the first feature of Alexander's prototype. Under the prototype specifications for the integrated long-form monthly household surveys, the sample sizes of 1-year and 5-year cumulations would be 3 million and 15 million housing units, respectively. By comparison, the size of the long-form sample from the 1990 census is approximately 17 million housing units. POTENTIAL BENEFITS OF CONTINUOUS MEASUREMENT The use of a continuous measurement program in conjunction with a decennial enumeration of the entire population would have several major indirect benefits for the enumeration itself. The first is that the continuous updating of a master address file would be automatically and systematically undertaken. This in turn would have benefits for two key aspects of the count—coverage and cost. An address file that is continuously updated accrues the benefits of having trained staff, one of whose primary functions is continually to seek improvements in the procedures for updating the file. Thus, not only will the address list for the enumeration have updated data, it will be an updated product with improved data management procedures. Consequently, there should be fewer missed dwellings in the mail component of the

OCR for page 81
A Census that Mirrors America: Interim Report enumeration and fewer erroneous address inclusions—which, in turn, will reduce the cost of following up mail nonresponse and decrease the level of undercoverage due to missed dwellings. The second benefit for the enumeration is the elimination of long-form data. (Also, in the decennial year, perhaps some data that in the preceding census were short-form data and were collected from every household will instead be gathered in continuing surveys, needed from only a large subsample of households.) By restricting data collection at the time of the count to just a few data items, there will be less burden on households to complete the portion of the census that is associated with the enumeration. This reduced respondent burden is likely to lead to a modest increase in mail response rates and should make it easier to collect data during nonresponse follow-up. These factors will consequently reduce costs of the enumeration (in addition to the fact that, with no long-form data, less data will need to be processed as part of the enumeration). By concentrating census operations in the decennial year on the task of enumerating the population, it may be possible to devote greater and more diverse efforts to the task of reducing undercoverage. The primary source of improved data quality resulting from a continuous measurement process would be an increased frequency of census data, especially for larger geographic areas. The Alexander (1993) prototype includes annual updates for larger geographic areas (such as individual states and metropolitan areas) and updates every 5 years for small areas, such as block groups, the units for which long-form data has been available in recent censuses. This contrasts with a once-a-decade census, for which data at all levels become increasingly out-of-date as each decade progresses. Early in the decade, planning and decisions are often made using data from the preceding census, which are as much as 13 years old. For many characteristics in society, these data are of very limited usefulness since they bear little resemblance to the current situation in many geographic areas. Although it is obvious that access to more frequent data at a variety of geographic levels is highly desirable, it will come at some cost. A critical element in determining whether a continuous measurement process is appropriate and what exact form it should take will be the assessment of the tradeoff of the benefits of increased frequency of census data over time versus the increased cost that will likely ensue from collecting data continuously. Such an assessment will need to consider how quickly the relative distributions of census long-form variables change at various geographic levels. Recommendation 5.3: The Census Bureau should undertake an extensive and systematic evaluation of the benefits from having more frequent census data available for both large and small geographic areas. One example of the use of census data where infrequency has negative effects is in the distribution of funds for education under Chapter 1 of the Elementary and

OCR for page 81
A Census that Mirrors America: Interim Report Secondary Education Act. The formula for distributing these funds requires estimates of the number of children in poverty at the school district level. There are about 15,000 public school districts in the United States, varying greatly in size of population covered, from a few hundred to several million. Clearly such data become dated very rapidly after a decennial census, resulting over time in less equitable allocation of what are substantial Chapter 1 funds. A quantitative evaluation of the effects for the Chapter 1 program of more timely medium-and small-area estimates of the numbers of children in poverty might be an important component of an evaluation of the benefits of continuous measurement. A second likely benefit from a continuous measurement program is improved quality of operations and, hence, of the data. By having a permanent professional census team, rather than making a special effort to mount the resources to conduct a census once a decade, improvements in the quality of census operations are likely to occur and to be maintained much more effectively. Put another way, a continuous process leads naturally to the important development of a permanent infrastructure incorporating a strong institutional memory of the entire census process. Such an ongoing commitment on the part of a census staff is also consistent with the recommendation in Chapter 3 that the Census Bureau maintain an ongoing presence in hard-to-enumerate communities, with a view to reducing differential undercount. A third benefit is that a continuous measurement operation would appear to provide an environment more conducive to introducing data from administrative record sources as part of census data. Alexander (1993) includes this point in his integrated estimates program. Investigation and incorporation of administrative sources of data would be an ongoing initiative. A further benefit that the panel sees is that it would be possible to incorporate periodic checks of the reliability of administrative data. One of the weaknesses of such data is that changes in administrative systems over time can lead to undetected bias being introduced into the series of estimates. By combining a continuous measurement program with a decennial enumeration, it appears likely that the accuracy of estimates could be improved by using the decennial data to augment the data from the continuous component. The relationship between the decennial short-form data and the continuous measurement data would need to be continuously reevaluated, at least implicitly through the estimation and error estimation procedures, if not explicitly as well. The gains from such an approach might make it feasible to release more timely estimates for certain areas, because smaller samples would be needed to produce reliable estimates. At least, this could be done early in a decade when the relationship between the two data sources remains strong. Finally, a continuous measurement procedure, through the continuous updating of a master address file, would provide a quality frame for the Census Bureau's household surveys program. This should result in substantial cost savings across the full range of these surveys, thus adding to the attractiveness of the option. It should also lead to improvements in the quality of data from these surveys by reducing coverage error.

OCR for page 81
A Census that Mirrors America: Interim Report TIMING AND IMPLEMENTATION Since continuous measurement is a very different way of approaching the collection of census content, the panel recognizes that introduction of such a program would likely be viewed with some skepticism and misgiving. Users of census data, and those responsible for paying for that information, will need to be convinced that continuous measurement has real benefits, even if the Census Bureau's research activities indicate strongly that this is the case. We believe that the Census Bureau is aware of this skepticism. Because of the response that would probably greet the introduction of continuous measurement, the panel believes that it would be unwise to plan its introduction right after the decennial census enumeration. At that point, the stakes would be very high, since no long-form data would have been collected as part of the enumeration under the approach discussed here. Any growing pains of the continuous measurement program would receive microscopic attention at that time, with heightened concern that the program might fail to deliver fully as advertised. This would also be a time of intense activity for census operations as the enumeration procedures are completed, and efforts to complete a successful enumeration with less undercount than in previous censuses would understandably be paramount in the attention of management at that time. Therefore, we believe that the successful introduction of a continuous measurement program in conjunction with any decennial census, whether it be for 2000 or later, must take place several years prior to a census year. At such time, data users stand to gain something they would not otherwise have—timely estimates late in the decade—and the program would have the opportunity to prove its worth by the time of the enumeration, thus substantially reducing concerns about the lack of long-form data collection in conjunction with the enumeration. Recommendation 5.4: The goals for a continuous measurement program (see Recommendation 5.2) should include phasing in the continuous measurement program during the latter half of the decade prior to the relevant census year. One particular aspect of a continuous measurement program that needs to be addressed before such a program could be implemented is that of incorporating changes in the collection instrument over time. There are two types of such changes. The first is the addition of new data items and correspondingly the deletion of those no longer deemed relevant. The second is changes in question wording or form design in order to incorporate improvements in data collection methodology. As mentioned, administrative record data might come to replace data obtained from the household sample. Changes in mode effects can also occur without any changes in the form if, for example, there is a decrease in the proportion of the population (or of particular subgroups) who return the form by mail, resulting in a greater proportion (or different portion) of the data being obtained by telephone.

OCR for page 81
A Census that Mirrors America: Interim Report Even for decennial censuses, introducing changes between one census and the next is difficult, because users expect to be able to make comparisons of data over time. This becomes a much greater issue in the case of continuous measurement, because all estimates produced consist of cumulations of at least several months of data. However, for larger geographic areas, the continuous measurement system will provide much more useful time series of annual estimates, another benefit of the more timely data provided by this approach. Periodic household survey programs must also deal with the problem of providing reliable time-series data in the face of innovations in data collection. In general, survey programs are not critically dependent on cumulations, as an integrated estimates program would be. There are procedures for handling such changes in periodic household surveys: these procedures involve using both forms of instrument for a period of time, thus allowing users to evaluate the effects of the change on the series. For continuous measurement, this approach would likely add to the cost, since such split panels generally have to have more than half of the usual sample size in each part. Also, taking account of changes in estimates resulting from a form change might be more difficult for cumulations than it is for time series. Recommendation 5.5: As part of its research into the feasibility of and methods for implementing a continuous measurement program, the Census Bureau should undertake a thorough study of the consequences of changes in the instrument over time, as well as changes in mode effects. A plan must be established for incorporating the effects of such changes into the cumulated estimates and into the time series produced by the continuous measurement program. The panel is encouraged by the approach that the Census Bureau is taking to the consideration of continuous measurement, but we wish to make clear that the panel is not yet endorsing or proposing the adoption of continuous measurement as part of the 2000 census. There is substantial research and analysis that must be carried out before it is clear whether such an approach is a good one. We do think it is a real possibility, however, and we look forward to the results of the Census Bureau's evaluation activities.