Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 259
–4– Considerations for the 2010 Census In carrying out our primary charge regarding the selection of experi- ments and evaluations for the 2010 census, the panel inevitably had to con- sider plans for the conduct of the census itself. Moreover, the conduct of every census inevitably affects the Census Bureau’s overall research program for the decennial censuses. Thus, in this chapter the panel presents three recommendations concerning some census operations with a view to their contributions to improvement to census methodology. Although we under- stand that the design of the 2010 census is relatively fixed, we hope that the material in this chapter may still be of use to the Bureau. 4–A TECHNOLOGY The Census Bureau will be using more technology in the 2010 census than in previous censuses, and this has raised some concerns that the panel would like to see addressed in the final plans for 2010. The concerns in- volve the functioning of the handheld computing devices to collect field enumeration data and the operation of the management information system for the 2010 census. By management information system is meant the vari- ous software systems that manage and monitor, somewhat interactively, the mailout-mailback process, nonresponse follow-up, field enumerator hiring and firing and compensation, questionnaire data capture, and other major census processes. We don’t know the full extent to which these systems need to interoperate, but at least some modest degree of interaction is re- quired, for example between the Master Address File (MAF)–TIGER system and the handheld devices in providing electronic maps for the handheld de- 259

OCR for page 259
260 INTERIM REPORT vices to display. The two primary concerns are whether the transmission of data using the handheld computing devices could be compromised in some manner (or could be lost unintentionally through mistakes and technologi- cal problems) and whether the needed interoperability of the components of the management information system could be hampered either by the adapt- ing of software or the acquisition of newer software releases for the various components of the system between the dress rehearsal and the 2010 census. With respect to the security of the transmissions of the handheld com- puting devices, the motivation to do harm to the census counts may be rel- atively modest given the lack of a financial incentive, and this may result in less chance for a security breach. However, this argument is not com- pelling. Furthermore, not only is there interest in reducing the opportunity for a security breach, there is also the matter of being able to assure census data users that the counts are valid. To accomplish this, the Census Bureau should carry out an independent validation and verification of the function- ing of the handheld devices. This could be accomplished in the following ways, either in the 2008 dress rehearsal or in the 2010 census: 1. Establish a dual recording stream for all data from mail-in, telephone, or handheld devices: one file to go to the contractors and one to be retained by the Census Bureau. In the event of catastrophic failure by a contractor or a serious challenge to the results, it will be important to have all the raw data in the hands of the Census Bureau. 2. It is practical to develop simple programs, written and run by Cen- sus Bureau personnel, that will search large data files for patterns of interest. In this way, unexpected or curious results can be efficiently discovered and checked, and this can contribute to the validation and verification effort. 3. Related to points (1) and (2), the Census Bureau should develop quantitative validation metrics, a priori, to check for data set self- consistency and comparison of redundant data. Other important general operational measures that we recommend for the 2010 census, either to determine whether any security breaches have oc- curred or to prove that the 2010 Census was secure (and which are probably already carried out), include: • Retention of an archive of all raw data with date and time stamps. In the event of serious software failure, it would be important to be able to “replay the census” from these raw data. • Use, by the Census Bureau and contractors, of dedicated processing systems that run no other applications and have highly secured net- work connections and secure accounts. • Use of periodic system checkpoints to monitor and analyze software systems for intrusions or unauthorized manipulations of data.

OCR for page 259
CONSIDERATIONS FOR THE 2010 CENSUS 261 • Strict control over handheld devices, including their inventory, indi- vidual device identification, and permission to operate (turn them on, turn them off, enable data transfer, disable data transfer, etc.). • Use of methods to prevent and detect bogus data streams, including data that impersonate handheld devices. With respect to concerns about configuration control of the management information system of the 2010 census, the processing history of the dress rehearsal could be retained and the software systems intended for use in 2010 could be used to “replay” the dress rehearsal soon before the 2010 census to identify any systems that fail to interoperate. That is, assuming that the management information system for the dress rehearsal functions well, saving the processing history would then provide a means for deter- mining whether modifications or updates of components of the management information system between 2008 and 2010 had raised any interoperability problems. (This is referred to as regression testing.) In addition, all informa- tion system errors encountered during the dress rehearsal should be captured in a form that allows them to be used during the software development work between the dress rehearsal and the start of the 2010 census. Recommendation 9: The Census Bureau should use dual- recording systems, quantitative validation metrics, dedicated processing systems, periodic system checkpoints, strict control over handheld devices, and related techniques to ensure and then verify the accuracy of the data collected from handheld comput- ing devices. Recommendation 10: The Census Bureau should provide for a check to ensure that the subsystems of the management infor- mation system used in 2010 have no interoperability problems. 4–B DATA RETENTION BY CENSUS CONTRACTORS Given the very successful use of contractors to carry out several decen- nial census processes in the 2000 census, it is expected that the use of con- tractors will be expanded in 2010. The component processes that will be contracted out in 2010 include (1) the decennial response integration sys- tem (DRIS), which involves systems management of the process of question- naire response and data capture; (2) the automation of field data collection (FDCA); (3) the data access and dissemination system II (DADS II); (4) the 2010 census communications campaign; and (5) the printing contract. The fact that these systems will be operated by contractors raises an additional complication. Any data collected as part of developmental or operational testing of these systems prior to their use in 2010, as well as any data col- lected in monitoring the operations of these systems while in use in 2010,

OCR for page 259
262 INTERIM REPORT may be viewed as proprietary. This would limit the Census Bureau’s ability to assess the performance of these systems in looking toward 2020. While the contractors themselves may issue their own evaluation studies, this is in- sufficient given that contractors have a bias in evaluating their own systems. We assume that contractual agreements about the sharing of such data, if they have not already been provided for, are now too late (especially for de- velopmental testing results). In that event, the Census Bureau should try to develop some informal understandings of data sharing with their contractors to address this issue. If it is not too late, such data-sharing clauses should be included in contracts. 4–C CENSUS ENUMERATION AS PART OF TELEPHONE QUESTIONNAIRE ASSISTANCE The current plans regarding the use of Telephone Questionnaire Assis- tance (TQA)1 are for it to function primarily as a means for assisting the public in making correct responses to the census form, in particular for com- plicated situations involving residence rules or responses to the race and eth- nicity questions. In addition, this is a method for people to obtain assistance in filling out the census questionnaire when English is not their primary lan- guage. On occasion, this has also been a vehicle for households to provide their responses to the census questionnaire. However, this possibility was not encouraged in 2000. For the 2010 census, we think the Census Bureau should consider mak- ing more transparent to respondents this option of collecting the informa- tion for the entire census questionnaire over the telephone once someone calls TQA. Specifically, whenever someone connects to TQA, the willingness of the operator to take the complete information, instead of just providing the specific help requested, should be made known to the caller during the initial part of the interaction. Our understanding is that this was not done in previous censuses due to the resources needed, especially the number of operators, and due to the additional procedural complications, especially of providing this opportunity for those receiving the census long form. How- ever, given that this is a short-from-only census, we think that the need to get the information as soon as possible, when possible, should outweigh other concerns about making this option more frequently used. This could be especially important if the hourly wages of field enumerators increase 1 Telephone Questionnaire Assistance was an operation used in the 2000 census in which people could call a toll-free number to get help in filling out their census questionnaire, to arrange to be sent a replacement questionnaire, to arrange to be sent a language guide, or to provide their census questionnaire information in situations in which they were not provided a census questionnaire.

OCR for page 259
CONSIDERATIONS FOR THE 2010 CENSUS 263 substantially in 2010, since collection of such information may importantly reduce the cost of the nonresponse follow-up. If this change is not implemented in 2010, the Census Bureau should collect sufficient information to carry out an evaluation after the census is completed as to the percentage of callers to TQA who ultimately sent back their census questionnaires to estimate the additional nonresponse follow- up costs due to the lack of collection of the entire census questionnaire over the telephone. Also, a possible experiment that should be considered is to sample the callers and ask those sampled if they would mind providing their information at that time by telephone to better estimate the additional resources required. Recommendation 11: The Census Bureau should strongly con- sider, for the 2010 census, explicit encouragement of the collec- tion of all data on the census questionnaire for people using Tele- phone Questionnaire Assistance. In addition, the Census Bureau should collect sufficient information to estimate the percentage of callers to Telephone Questionnaire Assistance who did not ul- timately send back their census questionnaires. This would pro- vide an estimate of the additional costs of nonresponse follow- up due to the failure to collect the entire census questionnaire for those cases. The Census Bureau should also consider carry- ing out an experiment whereby a sample of callers to Telephone Questionnaire Assistance are asked whether they would mind providing their full information to better estimate the additional resources required as a result of expanding Telephone Question- naire Assistance in this way. In conclusion, the panel is enthusiastic about the opportunity to collabo- rate with the Census Bureau on its plans for selecting and designing produc- tive experiments and evaluations in conjunction with the 2010 census and, more broadly, a more productive research program overall. The Census Bu- reau has a very proud history of innovation, including the development of punch card machines, the first nonmilitary application of computers, sur- vey sampling, hot-deck imputation, FOSDIC (Film Optical Sensing Device for Input to Computers), to name a few, and we hope to help continue this important tradition.

OCR for page 259