Cover Image

Not for Sale



View/Hide Left Panel
Click for next page ( 2


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
OPENING REMARKS, FIRST DAY DR. NORWOOD: I want to welcome all of you to this workshop on dual- systems estimation [DSE] that, as you know, is sponsored by the Committee on National Statistics Pane] to Evaluate the 2000 Census. This panel has been working for some months now attempting to learn as much as we can about this census and to gather background information about all aspects of it. We have had several meetings. We have, of course, read much of the literature. There is a lot of it, so I cannot say that we have read absolutely all of it, but we have done a pretty good job of that. We have listened to the Census Bureau describe its plans. We visited census offices during the dress rehearsal and visited several of the processing centers. We will be doing much more of that. This workshop is another of our efforts to learn as much as we can. I want to emphasize that this panel has no position on any of these issues at this time. We remain con~pletely objective not an easy task, in this, one of the most politically charged census efforts in our history. But we are not interested in the politics of this effort. I spent my whole life staying out of politics. We are here to learn about the plans for 2000, not to reconsider what was done in 1990. Our purpose is not to provide specific a(lvice to the Bureau on the issues that will be presented today, although, of course, it is possible that the discussion on some of the issues might prove useful to the Census Bureau. We are here doing the job that has been asked of us, to use our knowledge and experience to evaluate all aspects of this monumental effort of measurementoperational, preparation, collection, compilation, survey design, statistical methods so that we can produce an informed evaluation. We know that the issues to be discussed here today and tomorrow have been the subject of considerable controversy, in the Congress, the courts, and in the statistics profession. I expect that differences of opinion will surface among the speakers today. Indeed, I hope that will be the case. But let there be no mistake: this pane] has no position on dual-systems estimation or on the issue of adjust- ment itself at this time. We have deliberately invited a number of professional statisticians who we know have different views on the issues to be discussed. We are sponsoring this conference today as a part of our effort to review the statistics of adjustment in a professional, non-political environment, so that we will have a fuller understanding of the issues we will need to consider in completing our evaluation. Let me just take a moment to tell you how I plan to run this conference. I shall try my best to keep to what I think is almost an impossibly tight time schedule. Howard Hogan, from the Census Bureau, will present each set of issues. He will have a block of time in order to make those various presentations today and to- morrow. Members of the panel will, I am sure, raise questions or make comments. Of course, others of the specially invited guests will have an opportunity to do so as well. The tinting of all the breaks will be very carefully monitored. I ask that you return to your seats as quickly as possible after them. The speakers will be expected to stick to the specific issues before us. Com- ments, of course, must be relatively brief and to the point. I know many of the

OCR for page 1
2 ~ PANEL TO REVIEW THE 2000 CENSUS people here Quite well. and that mav be a little difficult But I think we will manage it. ___O_ I hope that you will forgive me when I am forced to curtail the discussion I fear that if I do not keep the time with an iron handand I have asked for a little monitor here we will just not accomplish our goal today and tomorrow. Once again, let me emphasize that we are getting at this, really, with no holds barred, and no comment, no question that is raised by members of the panel shouIct be taken as an indication of any position of the whole panel. The panel has not discussed dual-systems estimation as yet. We are here, reallyand rather eager, as a matter of fact- to engage in an intellectual discussion, not to take sides. I would like to, first, introduce the special guests. We have a lot of guests here, and I am pleased that they are all here. We have eight participants who are sitting toward the front here, who will be participants: Barbara Bailar from NORC [Na- tional Opinion Research Centeri; Stephen Fienberg from Carnegie Mellon; David Freedman, University of California at Berkeley; Bruce Spencer, from Northwest- ern; Philip Stark, University of California at Berkeley; Marty Wells from Cornell; Don Ylvisaker, University of California at Los Angeles; and Alan ZasTavsky, from Harvard. We have some other invited guests as well, whom I would like to introduce: Lynne Billard, from the University of Georgia; Hermann Habermann, from the United Nations; Charlie Tones; Ben King, who is chairman of the Committee on National Statistics Pane} on 2010; ~ Mary MuIry; John Rolph, from the University of Southern California, also chair of the Committee on National Statistics; and Kirk WoTter, from NORC. I should say that we also have Ken Prewitt, the director of the Census Bureau. We also have Bill Barron, who is the deputy director of the Census Bureau. Ken, would you like to say a few words? DR. PREVVITT: Thank you, lanes. It obviously goes without saying that the Census Bureau takes this meeting very seriously. We are much indebted to the Academy and to the panel for having organized it, for having given so many pro bono hours to struggling with this as we try to put Census 2000 in place. A word or two on Census 2000, then I will turn more specifically to the agenda today. As I have said many times to the pressCensus 2000 is on schedule and on budget, as of today. That does not mean that we will not have operational difficulties that have yet to be encountered. But right now we are in as good a shape as we could hope to be in terms of mounting it. Obviously, the census has started. Our operations in remote Alaska have been launched successfully, and not just on the PR front, but also operationally successfully. The hiring is slightly above target right now. We want a pool of 3 million. We wanted to be at 45 percent this week; we are at 48 percent. Our budget estimates look good. None of our major operations are not within the budget boundaries that we had set for ourselves. As you will hear in a monument, the A.C.E. listing is on schedule. There is no major operation right now that is not where it needs to be for this ~ The panel referenced here is the Panel on Research on Future Census Methods, which is charged to advise the Census Bureau during its early planning for the 2010 census.

OCR for page 1
PROCEEDINGS, SECOND WORKSHOP ~ 3 moment in the launching and implementation of Census 2000 MAF [Master Address File] production, and I could go on down the line. I just wanted to put that as part of the setting, because, as I will get to in a moment, obviously one of the issues for DSE has to do with operational capability, as well as some of the technical issues. Indeed, as I have said to this panel and I would like to say to this audience, if it [the census] does unfold as robustly as we hope it is on an operational front I should also say, on the promotion front it is going to be much, much bigger than any of us could have imagined. I do not mean the Census Bureau's promotional effort; I mean the country's promotional effort. I just left a meeting this morning with members of the California Assembly. The California Assembly has just voted $24.7 million to, quote/unquote, "help" us do the census. States and cities and counties all over this country are investing their own resources, their own labor effort into a successful census. Needless to say, we welcome it, but it also creates some operational complex- ities at the edges. There will be many, many, many things connected to Census 2000 that will not be under the direction or control of the Census Bureau itself. In that sense, it is a very different kind of census. I do not want to underestimate that. It is a very big deal. There will be many, many little problems that will be created because we have between 40,000 and 50,000 different groups and local governments who, for their little area or their constituency, believe that they are doing the census not just promoting it, but actually helping us do it. That raises Tots and Tots of complicated issues at the edges. I could give you endless anecdotes on that. AS a generic issue, as I say, it is to be welcomed, but it is not without its complications. I met yesterday with the GAO tU.S. General Accounting Office] and challenged them, in effect. When they actually evaluate this census, they are going to have to evaluate not just what the Census Bureau did, but what the country clip. This is a metaphor that we have used, but it is within the realm of accuracy. It is not hyperbole. It really is a census being shared with the American people, with all the implications and complications that that creates. So I just wanted to make a few comments of that sort to set the stage for today's meeting. With respect to DSE, as I read and hear the criticisms, it seems to me that they fall into three broad categories. There is, of course, the concern or criticism that the Census Bureau, in its design, is engaged in partisan politics; secondly, that the Census Bureau is perhaps not sufficiently competent to manage an operation as complex as the dual-systems estimation operation and sort of a sidebar on that one, at least as I read the literature, that even if it were, it maybe floes not have the wherewithal (because it may not be doable) to get the public to comprehend it. That is, there is a sort of public comprehension problem, quite separate from the competency of the Bureau to actually do it. Third, of course, and by far the most important issue as far as we are concerned because it is, fortunately, fact- basedare the serious (differences of opinion and viewpoint about whether DSE will actually improve census coverage and counts. Let note just talk about each of those very briefly. The first one, of course, is not of our doing "our doing" including everyone in the room that is, the issue of the

OCR for page 1
4 ~ P~ELTO~VIEWTHE2000CENSUS partisan concern. It has taken what is really a very important kind of technical, professional debate and captures! it and used it for other purposes, well outside the boundaries, if you will, of the technical and professional debate. Nevertheless, it creates part of the environment in which we are all caught up. It is not goof! for the Census Bureau, it is not good for the national statistical system, ant! I think it is also not good for the statistical community, to have had its work be pulled into, in effect, a debate that is about partisan motivation and partisan attention. I think, as everyone in this room knows, no one has put on the table any evidence that the design was selected because it had a known outcome for partisan purposes. To this date, no one has put any evidence on the table that it would be implemented with that in mind. Nevertheless, it is the shadow, in effect, that hangs a bit over Census 2000. It is not the task of this meeting. Nevertheless, I think this meeting ought to be aware of the fact that anything that we can do to make it clear that this is a debate about a technical issue, that it is fact-based, that there are reasonable differences of viewpoint about whether DSE will improve the census coverage, and that it has nothing to do with anybody's partisan affiliations or interests the more that the statistical community can make that statement, the better it is for all of us. Let me turn to the second issue, which is the issue of the operational compe- tency. I turn to this one obviously not in any defensive manner. I would hope we would be extremely frank. But I do think it is important to distinguish between those criticisms of DSE that are based upon technical judgments and those that are based upon whether the Census Bureau can do something of this complexity in a competent manner. The Census Bureau is remarkably nondefensive about conversations about its technical competence and operational competence. I have only been at the Census Bureau for a year, and I have been part of a Tot of other institutions, a Tot of good ones Stanford University, the University of Chicago, Rockefeller University, SSRC [Social Science Research Councili, and so forth. We should all be so lucky as to be associated with an institution that is as comfortable with criticism and self-criticism as is the Census Bureau. It is a remarkable cul- ture in that sense and I say this as an outsider in which the more you beat us up, the better we like it. In some respects, it means we are probably going to do it better next time around. I do not have to remind anyone in this audience (but I do have to remind from time to time our political and public commentators) that, when the country debates the undercount, after all, they are debating the report card the census gave to itself. There is no other way to talk about how well the Census Bureau clid, except to Took at the Census Bureau's own work on how well it did and its own publication of that effect, including its operational failures and flaws and so forth. So when I say that we want to talk seriously about operational competency, I do not mean that in any kind of defensive way. If that is the area that sort of organizes some of the criticisms of DSE, then I would much rather have it discussed that way, because I think it is actually better in the Tong run for DSE. If the problems are really the competency of the agency, then that is at least something we can work on.

OCR for page 1
PROCEEDINGS, SECOND WORKSHOP ~ 5 So I urge us not to be hesitant, if that is one of the lead arguments, to make that a straightforward lead argument, and we will respond to it as best we can. In that connection, I add a footnote. This is, to me, just as a reader of the liter- ature, slightly disturbing the debate about the coding error in 1990 and whether it did or clid not move a seat.2 Much of the conversation about competency uses that as one of its lead pieces of evidence. Since the issue of whether it did or did not move a seat (on which there are people in this room who have quite different positions) is itself a matter of fact, I would actually urge the statistical community to sort that one out, because it really is a matter of fact, having kind of a clean, clear position on whether it would or would not have moved a seat. I would also urge those who use the coding error as an important piece of ev- idence about competency to recognize that, of course, a coding error can occur in any operation, not just in the DSE operation, but also in the enumeration opera- tion. I do not mean to back away from it, but I at least would like, insofar as that is the lead piece of evidence on competencyand not the only one; I appreciate that. There are very, very strong friends of the Census Bureau who were very concerned about the pre-Supreme Court design, from the point of view of its operational complexity and whether we could have pulled it off within the Integrated Coverage Measurement [ICM] within the nine months. So we are very comfortable having that conversation. But I see it as a somewhat different conversation from whether DSE itself will improve coverage the way that the Census Bureau hopes that it will. On that front, there are many, many issues, and that is what will be discussed today. That is why we are so pleased by the day-and-a-half, as we turn to that. The issues of, obviously, correlation bias and heterogeneity assumptions and the fifth cell and the "wily trout" and all those kinds of issues, are extremely important, to see how wide the (lifferences of viewpoint are within the statistical community. If we can actually focus on what really are the differences, then we can have a really quite important debate. That does not mean we erase them; no one is expecting to erase them. But at least let us frame them in the most precise terms that we possibly can. Obviously, the presentation that Howard will shortly be making moves in that direction. I will just conclude with one final comment in the area of technical differ- ences, which is, as I say, obviously what the meeting is to be about. Howard will talk about this. John will talk about it a bit as well. I just want to flag it as an issue. It would be very useful to make sure we understand the difference between distri- butional accuracy and numerical accuracy. They are different things. It is very difficult to maximize both of them, as we will try to present. It is important for the audience and the panel to know that the Census Bureau believes that it should 2Based on the 1990 Post-Enumeration Survey, the Census Bureau developed an initial set of ad- justed census counts through dual-systems estimation. Subsequently after Commerce Secretary Robert Mosbacher's decision not to adjust the 1990 censusa computer coding error was discov- ered by the Census Bureau, which necessitated revision of dual-systems estimates and undercount estimates.

OCR for page 1
6 ~ PANEL TO REVIEW THE 2000 CENSUS favor numerical accuracy. That is its operating philosophy in the decennial [cen- susi. If the criticisms focus primarily on distributional accuracy or proportionate shares, but not coverage, then we actually have a difference in viewpoint that is, in part, about the philosophy of the census, and not just the technical operations or technical components of DSE. If we actually are having an argument about the philosophy of the census, that is very important to sort out and make clear that that is, at least in part, the nature of the differences. So we do hope that, before the day-and-a-half is over, we have come to some kind of understanding of why the Census Bureau believes it has to favor numerical accuracy as its core mission for the conduct of the decennial, and why that neces- sariTy precludes us from maximizing in every instance the distributional accuracy. I urge us, as we have the conversation, to make sure that we know when we are talking about a difference in the philosophy of what the decennial ought to be as against the actual technical components of DSE. Obviously, lanes and panel, if this meeting can make any kind of headway whatsoever and I am sure it will on trying to really identify cleanly what the differences are between the Bureau's recommended decennial and what some of the concerns have been, it will be extremely valuable. The more cleanly we have identified it, the more likely we are to be able to take some steps that would move us in the right direction. Thanks very much. DR. NORWOOD: Thank you. Before we go further, I would like to ask people who have not been introduced to very quickly stand up and tell us your names and whom you are associated with, if you have an association.... I guess we have the staff of the Committee on National Statistics to thank for these plush quarters. I would like to thank, especially, Joshua Dick and lamie Casey for their hard work in getting all this put together. We are a little bit behind, so I am going to take some time from John Thomp- son, who can come up and just give us a very brief statement on where the census Is. CENSUS 2000 UPDATE MR. THOMPSON: I will be very brief on where we are. I do want to talk just a few minutes about accuracy. As Ken said, the Alaska enumeration is under way. I hope you all saw this picture in some of the various pictures. You Took really good. We are well under way to printing maps. We have to print over 20 million maps. We have just about finished printing the maps for the first major operation, which is where we go out and deliver questionnaires in non-city-style areas. We have a program called "New Construction," where local governments in city-styTe address areas can review their address lists one final time before the census. We have delivered well over 90 percent of the materials to the local gov- ernn~ents. They have until, basically, Census Day, April 1, to give us any new housing units that have been built.

OCR for page 1
PROCEEDINGS, SECOND WORKSHOP As Ken said, recruiting is going very well right now, but the big challenge is yet to come. Our ciata-capture centers are all open. We have been doing what we call "operational tests" and "dry runs." We are finishing the operational tests and dry run in Phoenix, and we just started our last operational test and dry run in our [effersonville facility. Our Local Update of Census Addresses Program this is where local governments can review the address list and challenge is drawing to a close. We have completed our appeals process for the rural governments, for the most part. We have sent the materials out to the city-styTe governments, and they are in the process of starting their appeals. That should come to a close probably by the end of February. Howard will talk to you about where we are on the A.C.E. [Accuracy and Cov- erage Evaluation Programi. Let me turn very quickly to a discussion of accuracy. As Ken said, there are several ways we view accuracy at the Census Bureau. One way we call numeric, and that is making a particular area as accurate as possible, or as close to truth as possible, as opposed to another kind of accuracy that we call either distributive or distributional, which is taking a set of areas or population groups and trying to bring the relative shares of these groups as close to a true distribution of relative shares as possible. Let me first say that both of these concepts are very important in the uses of census data. I think a Tot of attention has been focused on distributive accuracy, but not very much on numeric accuracy. As Ken said, as we plan the census- and, actually, those who give us a number of recommendations on how we should plan the census we focus, by design, on numeric accuracy. That is, we design procedures that are designed specifically for unique areas or unique population groups. If these procedures are carried out to perfection, then we will, in fact, get distributive accuracy. But we do not use distributive accuracy as a criterion for our planning. If we were to do that, for example, we would not do our partnership program; we would not let the state of California, for example, spend $24 million to try to improve the count; we would not do coverage improvement programs that we thought would improve the count for one population group, because that population group is not uniformly distributed across states, which would hurt dis- tributive accuracy. So we really focus on doing the best job we can for areas where we see problems. That is also true of those who give us advice. For example, we get advice that we should apply more advertising to specific groups. We get advice that we should select particular tracts in selected cities and try to make these tracts as accurate as possible, without regard to the effect on the overall distribution of states or cities. As we plan the census, we really focus on trying to achieve numeric accuracy, because that is really what I think we can focus on and plan for and try to develop solutions for. When we evaluate the census, we evaluate the census in terms of both numeric accuracy and distributive accuracy although I have to note that most of the eval- uations to date have focused on the PES [Post-Enumeration Survey] in terms of distributive accuracy, and on numeric in terms of most of the rest of the census operations and coverage improvement programs. I think that is something we will

OCR for page 1
8 ~ PANEL TO REVIEW THE 2000 CENSUS probably correct in our evaluations of 2000. We will look at distributive accuracy for all the census programs as we analyze it. I hope this puts accuracy in a little bit of context as we go through the discus- sion of the dual-systems estimation and the assumptions that underlie it. With that, I will turn it back to Janet. DR. NORWOOD: Thank you. We are now ready to start. Howard Hogan is going to have the hard job of being up here for most of the time. Howard is going to talk to us, first, about a series of topics that involve the Accuracy and Coverage Evaluation survey. A.C.E. OVERVIEW DR. HOGAN: Thank you, and thank the panel. The only awkward thing is that I really feel I am supposed to be here talking to the panel, and I will be spending most of my day with my back to my primary audience. But the compen- sating thing is to Took out and see so many friends from over the years, people who have been involved in this issue, some of them, even longer than I have. It is an amazing group that we have assembled, both on the platform and in the audience. My time is completely flexible, because it is really the panel's time as far as I am concerned. I will talk a little bit about the model. I am going to go through the survey itself fairly briefly, talk a little bit about the changes from 1990, and then go into the assumptions of the model and how we relate that to the survey. Most of what I will be talking about is in the background materials and so I am not coins to dwell on that. I assume you guys will interact. O cat DR. NORWOOD: Let me just say that we will save the discussion part for after Howard is finished, but if the panel or any of the invited guests have particular questions or if they want to get a little bit more information for understanding purposes, please raise your hand or speak up. DR. HOGAN: The question is, do we have the transparencies in the folder? Most of them, with only a couple of exceptions, are blowups of what you have in your folder. The couple that are not I can give to lanes, and we can get copies made. I want to introduce the subject of today's meeting. Let me assume that ev- erybody here has probably seen it at least 10 times, so I will not go through the model. But I did want to put up the model explicitly, so that when we want to talk about tcell] Nat versus N:2, we will have the rows and the columns the same. This is the basic paradigm, breaking down the population of the four cells. The question, then, is how to estimate N++ ttotal population], how we can estimate this. The Census Bureau adopts the traditional Peterson or Chandrasekar and Dem- ing noodle] this is in my paper, as well as several of the other papers that says that our estimate, based on the indepenclence assumption (I will not review that here for you all), is the cross-product divided by the number of matches, the num- ber in both. The way I think about it and the way I am going to discuss it today is

OCR for page 1
PROCEEDINGS, SECOND WORKSHOP ~ 9 that our estimate of the population is really made up, in my mind, of two compo- nents: the number of people in the census, the number of people correctly enumer- ated in the censusthat is, first, the thing we have to measure and second, the proportion of all people who are in the census. So, first, we have to measure the number of people in the census. That is just not given. Then we have to measure the proportion of the true population in the census. lust so we can define some terms, there is another way we can Took at this. We can take the standard independence assumption and apply it only to the fourth cell of those missed by both the census and the survey I am completely abstracting from the question of sampling for the momentand estimate the fourth cell as those in one and not the other, in the other, not the one, divided by those in both again, the standard assumption there. One term we will probably be discussing today I thought I would at least give it a symbol we can discussis those left over. After we take the standard dual- systems assumption of the fourth cell, we have an estimate of those missed by both. Many people believe that there are other people missed by both, a residual in the fourth cell that is not picked up by the standard assumption. I am just going to define that residual as N$, So we have something to talk about when it comes up. In these kinds of terms, the true population, then, is those in both, those in the survey and not in the census, those in the census and not in the survey, plus, by definition, those not involved. This is the true decomposition of the true population. We can then decompose the final cell into those modeled by the standard Chandrasekar and Deming or Peterson assumption and this residual, those left over. It has a number of terms the unreachable, the unmeasurable, the "wily trout," the fifth cell. There is really no standard term for that. But I believe all those terms are attempts to describe what I have defined here as N*. After you take the clual-systems estimate, you can then decompose it into the four cells that it is capable of measuring, plus sort of a fifth group here, which I have kind of defined as measurement error, matching error, balancing error, missing-data error all the other components. Therefore, taking what is really in the operationalized dual-systems model, which lacks, by construction, this un- reachable population but includes the measurement error, then the bias in the dual-systems estimate is sort of the difference between the unreachable and the measurement error. That is the bias in the clual-systems estimate. The unadjusted census is made up of those captured by both and those cap- tured only by the census. But it also includes two terms that we need to take account of. It includes the traditional statistical adjustments to the census, the traditional statistical additions to the census, through our traditional imputation process. We clefine that as II, or the census imputed population. It also includes enumerations that are not "correct] in the census. I will say much more about this. So the bias in the census is, of course, the difference between the true population and the census count. That is those picked up only by the survey, the mocleled part of those missed by both, the unmodeled, the unreachable part missed by both. But that is balanced off in the unadjusted census by the traditional census imputation,

OCR for page 1
10 ~ PANEL TO VIEW THE 2000 CENS US statistical adjustment process, and any erroneous inclusions in the census. So a lot of the discussion really is whether the bias here in the DSE is smaller or lower or distributed differently than the bias in the census. So that is kind of what I see as the model. The question, then, is how to operationalize that model, how to get the num- bers to put in it so that we can actually measure it. We do that through the accuracy and coverage evaluation. This is in the handout. It is at the end of the paper by Danny ChiTclers and Debbie Fenstermaker [Childers and Fenstermaker, 2000], if you want to react along or take notes. I am just going to walk through the operations, because when we discuss how the operations match into the model, we need to know what the operations are. The first operation, which we and the panel have discussed extensively in previous meetings, is the sample selection. I will not spend much time on that, because we have discussed it. Essentially, we take a sample of block clusters. The sample we took was about 2 million "housing units in a sample of] block clusters, because it was designee] to support the original 750,000 ICM sample. That was done last summer. Then we did this in the fall. This is completed. It is not only on schedule, as Ken said, but it is actually completed. We go out and list all the housing units in those block clusters. We did this in all the housing units the full 2 million [in the sampled] block clusters. That is indeed done, and I think was done well. We then need to reduce the 2 million clusters not clusters; I mean 2 million housing units. If I have been saying 2 million clusters, I apologize clusters that include 2 million housing units, down to a number of clusters to support 300,000 housing units. We do our block-cluster reduction. That is also done. This was done this time with sort of a change from 1990 that I think was quite an improvement. When we did our second stage of sampling, this block- cluster reduction, we had the most recent census housing unit counts, and we also had the number of housing units the A.C.E. listers listed. So we actually had a fairly good measure of the size from both the census and the A.C.E. We have the number in the block as listed by the census, the number in the block as listed by the A.C.E., and we actually have the difference. When we did our subsampling, we took account of both those differences, and I think drew a fairly good sample of large and medium blocks. Small blocks are those blocks where the census said there were zero, one, or two housing units. Those who followed 1990 know that the small blocks were quite a problem. We have done a number of things to reduce the problem, reduce the sampling issues introduced by the small blocks. For example, we do have the differences the number measured by the census, measured by the A.C.E., and the difference. That helps us control the blocks. Also we did a different clustering algorithm this time. Rather than having the whole universe of about 3 million small blocks to deal with, we were able to group about two-thirds of those with larger block clusters, and so the small-block sampling algorithm has to deal with a much smaller universe. We are using a two-stage small-block sampling scheme. So I believe, in terms of simply the sampling issues, we are much better in handling the small-block clusters than we were in 1990.

OCR for page 1
PROCEEDINGS, SECOND WORKSHOP [Transparency] In an operation that is basically brand new, we now have a housing unit match- ing ant! follow-upbrand new since 1990. We did test this in our 1990 series of tests and the dress rehearsal. It is not brand new since the dress rehearsal. We did do it then, and it was fairly successful. We take, for the blocks that remain in sample, the A.C.E-listed housing units and the census-listed housing units, and we actually do a matching between them. This helps us in a number of ways. For one thing, when we do the block sample reduction, we will have the two linked. That helps us quite a bit in maintaining the overlap between the sample of people that we use to measure the proportion of people enumerated and the sample of enumerations that we use to measure the number of people counted in the census and are the proportion of the census enumerations that are correct. So we want to maintain that overlap. This matching helps us do that. It also helps us identify any listing problem in the A.C.E., whether the A.C.E. is listed in a wrong block or whatever. It gives us a link that will help us in later interviewing and later matching. So it is operationally quite an important step, although it does not really feed into some of the equations directly. One thing we do not do is, if we do find a housing unit that the census has and the A.C.E. has, we do not add that to the A.C.E. We do not violate independence there. We note that that is a leftover census unit and carry that alone separately. So that is large block subsampling. . . . The housing unit matching is starting right about now. we are starting that on tinge. It will continue, together with the matching and the follow-up and the after-follow-up housing unit matching, until around April. This is something that happens before the census and gets us our files, our A.C.E. files and our census files, all lined up before the census takes place. Based on that, if a block has more than 80 housing units, then we subsample it. Since we have already linked the census and the A.C.E. together, the subsampling is such that we are able, largely, to maintain the same housing units in both the census sample and the person sample. That brings us to sort of the heart of the A.C.E., the person interviewing. This has two components, one of which we tested in the dress rehearsal but is new sincel 990. We use the telephone to interview some of the housing units from the A.C.E. How do we do that? We have linked from the housing unit matching which units in the census correspond to which units in the A.C.E., and so we know if the census person has completed the questionnaire and mailed it back, and, under fairly restricted circumstances, that people have completed their census question- naires and mailed them back. It is a house-number/street-name kind of address, single family, very well controlled. We will then allow the A.C.E. interviewer to telephone the housing unit and conduct an interview over the phone. That can start fairly early. That can actually start in May, where we do the telephoning to the housing units to get the independent A.C.E. interview. They have the phone number, but they have none of the other census information. This telephone interviewing really gets us two things. It gets us some inter- views early, and I think that is important. But judging from dress rehearsal, it did

OCR for page 1
PROCEEDINGS, SECOND WORKSHOP ~ 1 13 rates and the last resort ratesall the missing data. When you do not have a self- responseone thing is someone filling out a mail return or an interview with a household member in nonresponse follow-upwhen you do not have at least that, that should be an indicator of some of the quality of the census. The other thing that I have not heard mentioned is what happens with the primary-selection algorithm. We have talked about the problems of having all these different modes of responding the Internet and "Be Counted" forms and everything. What happens with the primary selection algorithm? How many forms do they throw out? How many times do they combine households across forms? Nobody has said what is going to be available on that, but there should be something available from the primary selection algorithm that would give you insight into your duplication problem. As far as the A.C.E. is concerned, as I have said, I think this housing unit matching the listing that they are doing, and then matching against the census address list gives you a lot of insight into the quality of the foundation for the A.C.E., and also what has gone on with the census. What would be really nice to see is the return rate for those listed blocks that the A.C.E. listed. I know you are getting them by tracts, but if you get them for those blocks, you would have what the geocoding error was going to be and some idea about the duplicate housing units and so on. Then, if you had the return rate for those blocks- if there is a lot of duplication, you could have return rates that are higher than the number of housing units listed by both operations. I am concerned about the mover match rate. I think you ought to be very concerned about what happens there, and also the missing data rate and the non- interview rate with the A.C.E., and just generally look at the quality control of the interviewing, those results. There is supposed to be some quality control on the matching, too: how many qualify as being extreme blocks, where they look at the difference between enumeration rate and the non-match rate? How many have to be reviewed from that? I think you will have some data that will give you some insight into what is going on. The problem is going to be synthesizing all that. I wish you a Tot of luck on that. DR. NORWOOD: Thanks. Don? DR. YLVISAKER: Thank you. I will not have very much to add, I think, after what others have said. One thing is that the more data we are allowed to look at, the better off we are. I think that point has been made. In terms of doing things like Toss functions, from a statistical, scientific point of view, we can set up problems and take Toss functions and do data analyses, and so on, and were data available, we could be doing these kinds of things a lot, and maybe gaining something in total. Some specific Toss functions I think are presumptuous on our part. There are so many users and so many share problems and so many things going on that our focusing on specific things isit is a data- analytic device and much fun and maybe a Tot can be learned, but we are not solving people's problems.

OCR for page 1
114 PANEL TO REVIEW THE 2000 CENSUS The real loss functions that I am concerned about are, in fact, the Bureau's Toss functions, which are very much subjective. That is, if we do this, then what are the consequences? If we do that, then what are the consequences? The big things I see in this current census I am very much taken with the publicity business. I realize the downside of that, but I think one is building on something for the future. I am hoping that there will be a real effect to publicity, the schools program, that sort of thing. I think this is all positive. The other large item I see is the multi-race issue. It is the first time we have seen it. We are very new at this kind of business. I have concerns, which I have already expressed. Things that I have heard so far that I will endorse there is the treatment of movers. Movers across post-strata are of some concern. I am sure this point has been made. I will not add anything further to that. I think that is about all I have to say. DR. BILLARD: Most of the comments that I might have made have already been made. I will add my concerns also about the potential next time we will be more concerned about overcounts than undercounts, in connection with all of these "Be Counted" efforts, get-out-the-vote types of efforts. I guess it remains to be seen how that will work out. If we think that the interest out there is so low that the mail return rate is going to drop off anyhow, maybe there will not be a problem with these potential duplicate counts. But I think it is a little naive on my part to hope that. Coming back to what I would like to see out of the panel, for myself, the more I read and the more I listen and the more I try to learn about what is going on, the more I think I do not know anything at all. I guess what I am saying is that I get more confused. There is just more out there than I could have possibly imagined. So, in some sense, I would like to see a summarizing list, I suppose, of the different methodologies, itemizing, in a shorthand way, the strengths and weaknesses of each, to help me try to pull together all that I am trying to learn. But I would also like to see a third sort of column, so to speak. We look at what we all agree we do not know, but would like to know. We recognize that there are some things even in the methodologies we have that we really do not know how to deal with. What I would like to see is which of these we know are impossible to do, that we just cannot do. Maybe theoretically we know it is not possible. I do not I OCR for page 1
PROCEEDINGS, SECOND WORKSHOP ~ 115 DR. FIENBERG: All my points have not been covered. I did not come to the meeting with prepared remarks, and although I have a quote from Santayana, I will not read it. I also did not plan to talk about lessons from 1990, but after you were given a brief homily on some of them yesterday, I thought it appropriate to at least fill in some of the details. Then I want to give you a few comments about thinking about what 1990 really says about 2000 and A.C.E., and then give you my issues ant! concerns. Let me begin by recalling why it is we are so focused on dual-systems estima- tion. That is because of the concern of the differential undercount in the 1980s, largely for blacks and only secondarily for other groups. We measured it. It was, as Larry and others have said rightly, a distributional problem, but a different one than a Tot of people have been focused on. We actually learned not just about blacks, but the fact that you could measure that differential in a very substantial way for other minority groups. We have been talking about those evaluations from 1990 as if they were some massive block and tied all to the secretary's marching orders. I would like to remind people of P-16/Part 1.5 P-16/Part 1 was the evaluation of the evaluation post-strata. That is the report that I think people should really be paying attention to about what we learned and why it worked, or why we think it worked or why I think it worked in 1990. The author is sitting down below. P-16/Part 2 really then carried that and asked questions of numerical and dis- tributive accuracy. It provided whatever evidence we had to glean about how that was also being handled, but that was not the starting point. Everybody has talked about the errors and mode} failures. I am not here to tell you that errors do not exist or that the model is correct. The model is wrong. All moclels are wrong. So the question is, how good is it, and how can we evaluate it? Transparency One of the issues here is to point to the extent of error in the 1990 PES. I just want to give you the numbers that go behind the comment. David Freedman yesterday stressed the PES net number of 5 million and talked about 3 million errors which I will not grant, but I will take as a starting point for discussion. If you take the PES grossand I read this out of David's most recent publication, collaboratively and take the 10 million EEs and the 15 million omissions, they net out to the 5 million. Take 25 million; 3 over 25 is not a very big proportion. But I ask you this question: what if the net was zero? That is, what if the number that you got for the nation was exactly correct? Would you care? I hope the answer is yes, because if you got the number for the nation correct, but you got the group shares wrong and the geography shares wrong, that is exactly what we have been talking about. So you could take the 3 million and compare it to a base of zero, and you wouIc! still want to use adjusted counts, if other criteria showed that that was right. David also yesterday talked about the computer error. Since Ken introduced it in his opening remarks, let me tell you the facts. David talkecl yesterday about sP-16 refers to one of the "P" studies carried out by the Census Bureau to evaluate the 1990 PES.

OCR for page 1
116 PANEL TO REVIEW THE 2000 CENSUS two sets of numbers. He said if you taste the adjusted numbers as proposed in fuly of 1991 and run them, two seats change. They go from Wisconsin and PennsyI- vania to California and Arizona, respectively in fact, if you Took at the order in which they go. Then he told you that if you took the computer error correction into account and used the CAPE post-stratification, which, as John told us, was for a totally different purpose, and took those numbers in random, only one seat changed that is, the seat that went from Wisconsin to California. In fact, the seat in that allocation does not move from Pennsylvania to Arizona. The fact is that there are three sets of counts. The micIdle one is the one that David did not tell you about, which is what would happen if you fixed the computer error and ran the numbers again. The answer is, two seats change. One seat goes from Wisconsin to California, and the second seat goes from Pennsylvania to Arizona. The numbers are public. You can find them by going to this Web site. This is the only reason I really came up, to give the Web site. Those who do not recognize what the extension is about, it is this. That is a Web page for Who Countsi6 If you quickly scroll through the book reviews and a few other stray comments and get to the bottom, there is a pointer to a paper called "History, Mythmaking, and Statistics: A Short Story About the Reapportionment of Congress in the 1990 Census." On it are the three sets of figures and the results of running it in the apportionment formula. The problem with block-level adjustments we had that discussion yesterday, but it is important to know why we are doing it. We are not doing block-level adjustments because we care about block-level numbers. It is a device. Block-level adjustments are there as a device so that numbers will add up, whatever way we choose to aggregate things for other purposes. When the Academy pane] met in the ninetiesI am sorry Ben King had to disappear, because he was there at the time, we had a Tot of discussion about whether or not there was another way to do this so that we did not have to put people into blocks, because then we would not have to argue about block-level accuracy. The Census Bureau prevailed in its notion, and I lost in my notion that what we really needed was a black box that produced the right aggregate numbers that would always be consistent. So that is the past. I was actually very impressed by the materials in the book that we reviewed. There are details there I did not know, some because they were just created, but others because I did not understand them. I thought the exposition moved us forward a Tot, and that exposition is crucial. I think the current activities are working to clarify and refine the results. My best guess is that they should lead to improvements. I do not have a Tot to anti to the others' suggestions on the A.C.E. but there are Tots of lacunae. I dicT not agree with some of the choices. Marty disagreed with others. I would guess that each of us disagreed with an overlapping but different set. 6The Web page Dr. Fienberg cites is http://lib.stat.cmu.edu/~fienberg/WhoCounts.html, which deals with his co-authored book: Anderson, Margo T. and Stephen E. Fienberg il999~. Who Counts' The Politics of Census-Taking in Contemporary America. New York: Russell Sage Founda- tion.

OCR for page 1
PROCEEDINGS, SECOND WORKSHOP ~ 117 I think the outcome is far from certain. The question is, is there an alternative? And I do not see one on the table. I want to stress that. Demographic analysis is not the quick fix. Howard Hogan gave us some of the reasons. They are given in much greater length in one of the analyses we did. They are just not there to do this job that you are talking about doing, at any level, except perhaps at the national level for blacks. So here we are in 2000, and the Bureau is under way. The census is going, although not the biggest parts of the operation. I was struck as I read the mate- rials by the reference list. It was great material, but none of the things that were referenced seemed to have occurred outside the Census Bureau. I thought it appro- priate to mention that there is research that goes on outside the Census Bureau, and some of it actually has relevance, I believe. Let me take the topic of correlation bias and heterogeneity and say that no- body has solved your problem, but if you Took at the paper by Kadane, Meyer, and Tukey, in 1998 [ASA [fourna] of the American Statistical Associationi,7 or Richard Cormack's recent exchange in the Journal of Epidemiology, with Ron Regal and [Ernest] Hook,8 or [Tuba] Alho's work on heterogeneity9 or, I dare to add, my paper in [RSSA tiourna] of the Royal Statistical Society, Series A] last year, with Johnson and Tunker,~ there are clues about what you can do and what you can learn, and the direction of the biases that are associated with heterogeneity and correlation structure. That leads me to suggest that there is actually a need for a short-term research plan, at the Bureau and outside. I do not think the Bureau can do this research between now ant! when it needs to get things done. I do not think it could com- mission it all on the outside, but I think it could commission some, and it could probably rally the community in some ways that it has not, that could be sup- portive of the in-house effort. Most of the empirical stuff has to go on inside the Bureau. We cannot, outside, do that for you. But I do think that there are things that could be done that would at least inform the analyses and help out, and maybe would help out in the longer term evaluation, too. I have two major concerns about census process. One we have talked about repeatedly, ant! that is how we are going to take stock of the effects of all of these local improvement efforts. But something else that was not mentioned that I think really does need some consideration by the pane] and it hardly got mentioned- the census consists of two parts. It consists of the short form and the long form. 7Kadane, Joseph B., Michael M. Meyer, and John W. Tukey {1999~. "Yule's Association Para- dox and Stratum Heterogeneity in Capture-Recapture Studies." fournal of the Amencan Statistical Association, pp. 855-859. See Cormack, Richard M. t1999J. "Problems With Using Capture-Recapture in Epidemiol- ogy: An Example of a Measles Epidemic." Journal of Clinical Epidemiology, pp. 909-914; and Hook, Ernest B. and Ronald R. Regal ~ 1999~. "Recommendations for Presentation and Evaluation of Capture-Recapture Estimates in Epidemiology." fournal of Clinical Epidemiology, pp. 929-933. 9 See, e.g., Alho, Juha M. (1994~. "Analysis of Sample Based Capture-Recapture Experiments." journal of Official Statistics, pp. 245-256. iFienberg, Stephen E., Matthew S. Johnson, and Brian W. Junker ~ 1999~. "Classical Multilevel and Bayesian Approaches to Population Size Estimation Using Multiple Lists." fournal of the Royal Statistical Society, Series A, pp. 383-406.

OCR for page 1
118 ~ P~ELTO~VIEWTHE2000CENSUS Bob reminded us about the importance of the Tong-form data. That is where all the content is. We are supposedly doing that on a sample basis. Does anybody have an evaluation planned to see how well we fulfill the sampling plan? It sounds like a trivial issue, but it is not. Just begin to think about all those extra forms that we are adding in and all the other devices, and what it does to the one-in-six sample. I have no idea. Indeed, an evaluation plan for the Tong form seems to me Tong overdue, even if this may be the last time we see it. Which brings me to the collateral issue: do we have any other data? The an- swer is, we may actually have some data from the American Community Survey [ACS], and that may be of value for evaluation here as a different kind of touch- stone. Actually, it is interesting, because we have asked the question in reverse. Can we use the Tong form as benchmark for the ACS, which is an important pub- lic issue if you are going to stress the Academies "reference not clear] ? But we can glean information front the ACS about the census process because they are out there now gathering data, and it is going to continue through the census process. Evaluation criteria: it strikes me that we need the list. It needs to be public. It comes in two parts. There are the things you are going to do before you release the numbers on April 1, 2001, presuming that you make a decision that the quality is of sufficient magnitude to release the adjusted counts, as well as the raw counts. But then you need the other ones. I think that list has to beit is not going to be definitive, but we need it in public. I think that is what other people have been asking about. But I do want to suggest that it comes in two parts. It is not obvious which things will go in the first part, beyond the simple quality control checks. Finally, I was struck by something when Bill Eddy and I did our little piece of data analysis and our search through the P.L. 94~-171] file. Something we did not mention was that, as we did that and we looked at the numbers on the Census Bureau's Web site, there are lots of I's. In particular, the second block cluster we identified was one that had because we found it in the P.L. 94 file 1 Hispanic and 70 American Indians. In other settings, when I would say that, the people from the Census Bureau would hide uncler their chairs, because that appears to violate virtually every reporting rule that relates to confidentiality. There is, of course, the issue of what those numbers are. In fact, it is my belief that those numbers are correct only for the block total, and there has been data swapping that is, a confidentiality edit has been applied to those numbers. Many of the public uses of the products of the 2000 census will come from public census products, as opposed to confidential census products. There is a major activity under way called the American FactFinder. The question is, will you look at it and evaluate the quality and the level of error that is going to be associated with what will be released in it? It is a tough job. As Martyr said, I am glad you are doing it. DR. NORWOOD: Ken Prewitt has a few words. DR. PREVVITT: It was suggested yesterday that they would be wise, but I am no longer promising to meet that standard. Another word or two on the issue of counts and shares. As I roam the country and talk to lots of people about the census and its uses, I am actually impressed by

OCR for page 1
PROCEEDINGS, SECOND WORKSHOP 119 how many places there are in the system where the absolute count is what matters more than the share. Obviously, everything is comparative, but where there are threshold effects shot through this system Detroit was mentioned, but there are dozens and dozens of programs and activities and investments where they have to do with threshold effects, the total numbers. The Gap [retail chain] does not care how many units it puts out there, but it only wants to put them out there in communities of at least 50,000.So however many they finch that are 50,000, that is where they are going to go. Every town in the country that wants a Gap wants to be 50,000. That is a made-up example, but not widely made-up. One runs into Tots and lots of decisions. The Social Security people, of course want to know absolutely how many old people there are going to be 15 or 20 years from now. The educational planners want to know how many kids there are. Obviously, you can say that is all a share basis. You have Social Security and educational planning, and so if there are more old people and fewer young people, then you that is actually not how the system works. People are actually making all kinds of very critical public and private decisions based upon counts, not shares. One of the things that most impresses me about the public relationship to the census is the importance for many, many groups of social recognition. They actu- ally want to be counted because they want to be recognized. I spent a fascinating half-day with Arab-American leaders. They actually want to be counted, not be- cause they are in some sort of share contest except in the largest, most generic sense they simply want the rest of us to know how many of them there are here and how well they are doing. The American Indian count problem is really much, much more about social recognition than shares, I promise you. The census mea- sured about 500,000 American Indians in 1900. That number is now close to 3 million. That is not fertility. That is more and more people wanting to claim their heritage. So do not underestimate the importance of counts for a complicated dynamic called social recognition. We are the only place to provide it to groups. It is way, way beyond just whether they are going to get some share of a federal largesse. That is a reality about our society. That does not mean this is a simple process. Obviously, the census data have a deep responsibility for reapportionment and redistricting and allocation of federal funds and so forth a creep, deep responsibility. That is why it is in the Constitu- tion. We do not trivialize the importance of getting the shares as close to accurate as we can. But do not underestimate the power of counts themselves in this soci- ety. The boosterism, the Thomas Jefferson story, and so forth it is really beyond boosterism. A quick word, if I may, on the overcount issue. lust to be reassuring, we are not unmindful of the overcount complication, given the greater (legree of public investment in the census. I should say, on the Internet, we really think we have put enough firewalls in that we will not have an Internet problem. You cannot get into the Internet file unless you have a bar-coded questionnaire in front of you. You cannot fabricate a bar-coded code number that we do not already have in our files. So I do not think the Internet is going to be a source of an overcount problem.

OCR for page 1
120 ~ PANEL TO REVIEW THE 2000 CENSUS We have spent a Tot of time worrying about how to handle the "Be Counted" forms. We have restricted the amount of time they are going to be out there, and so forth and so on. That does not mean it is still not an issue, but we do have internal processes, of course, to make sure the "Be Counted" forms are not going to be counted unless we can get them back to an address that we have identified, either as a bar-coded address there already or we will go out and field-test it if there is no address, we cannot use the "Be Counted" form. So we are not unmindful of the fact that new procedures could lead to an overcount. We have spent an awful lot of time, I think, trying to protect from that. A third sort of small thing I wanted to mention and I think it feeds into this very important conversation we are having about the number of different indica- tors we are going to be looking at as we struggle with making the big decision is something we did not have a chance to talk about, but you may be interested in. We have pretty well finished our work on the non-city-styTe addresses. That is a 20-million-household address file, based on the non-city-style areas. Most of that has now gone back out to the communities and our LUCA [Local Update of Census Addresses] Program for checking. They have come back in. We have reconciled. They have appeared where they wanted to appeal. The OMB appeal process is now well under way. Indeed, it is so far uncier way that the numbers I am about to give you we are fairly confident about. It looks as if we will add about 10,000 addresses in that process. That is on a base of 20 million. That is down to the fourth decimal point. That is 1/20th of 1/lOOth. I am not promising that other operations in the census will be that accurate. Nevertheless, we take some satisfaction in the fact that what was, after all, a quite difficult field operational process, in which we now have real data insofar as the communities, the cities, the local planning groups, and so forth, are giving us their real judgment, and we have an OMB appeal process and so forth- we now are finding that, at least for that operation, the accuracy rate was remarkably high. Will the city-style addresses prove to be as accurate? We do not know. We have not gotten there yet. But at least a 20-million address file turns out to be we did a quite good job on it, operationally. Those are just odds and ends. I mention that one as a way to, if you will, lead into my only interesting point, I think, or concluding point. With respect to the D SK, obviously we have been appreciative of this session and conversation, as I knew we would be. Many things we will go back and think hard about. There has been a good conversation about the mover problem, a good conversation about the search regions, a good conversation about some of the matching issues. These things we will go back and, insofar as we can make ongoing improvements, we will certainly try to do it. is the apportionment date and April 1 is the But, as Janet said in leading off this session, finally, a decision has to be made in real time. We did not invent the fact that December 31 redistricting deadline. The reapportionment clate, December 31, was, after all, put in place in 1932, when the country was half the size and a Tot easier to count. We have macle, as a bureau, obviously, enormous productivity increases since the 1930s in order to

OCR for page 1
PROCEEDINGS, SECOND WORKSHOP ~ 121 still be within that same time frame, counting twice as many people, and so forth. But we are living within real time. That is what the law says. It is a law that obviously could be changed. You could imagine a longer time period for doing the census. Nevertheless, for now, we will have the redistricting data out by April 1, as we are obligated to do. Therefore, it seems to note- and now I will speak personally, in terms of the decision that the Census Bureau will make the director of the Census Bureau, when this decision is made, could well be someone in this room but not me. There is an upcoming election, and there is reason to believe that there would be a different Census director if there was a different administration. Indeed, there is no particular reason to believe that, even if there was a continuation of the same party in power, there would be the same Census director. I think that is too bad, because I would like whoever has to make this decision to have heard this conversation and to have benefited from this interaction with the panel. So I hope the person is in the room, if it is not me, one way or the other. DR. NORWOOD: We fully elect it to be you, Ken. DR. PREVVITT: As I Took at making the decision, it seems to me that we can use a very simple 2 x 2 table. You have a census, good or bad; you have an A.C.E., good or bad. If the current thinking of the Census Bureau is that the census is not going to be very good and it is going to have a Tot of measurement error in it, but we have an A.C.E. that is good- an easy decision we will adjust and report acljusted numbers. If you have a good census, as best we can tell from the indicators available to us response rate, level of public cooperation, address file, and so forth and a bad A.C.E., it is an easy decision. We will not adjust. That takes care of those two cells. What do you do if they are both good, based upon whatever evidence we are able to accumulate? It is actually a very difficult decision. There you are asking yourself, at the margins, is A.C.E. going to improve? It would be my own think- ing, just speaking candidly as if I happened to be the Census director, that you would really want to make sure that marginal improvement was sufficient to pay the complicated legal costs, political costs, public-relations costs, and all kinds of things. If the census is basically good, good enough for the purposes for which it is normally designedthat is, if there was not much of an undercount correc- tion/adjustment factor then my feeling is, you probably do not adjust, if they are both good. Let me say about the cell where the A.C.E. is not good, we had a dress rehearsal experience with the A.C.E., and we sweated. We deliberately sweated, because we ran into some things in our South Carolina dress rehearsal site, as most of you know, where we had to ask ourselves the question of whether the census was so bador whether the A.C.E. was bad or the PES was bac3as lay Waite put it so well, did we have a problem with the camera or a problem with the thing being photographed? We knew something was not working right. We said to ourselves and this was not an easy decision we are going to make this decision in real time. We are going to set exactly the same deadline as we wouIct have in the real census. If we cannot sort it out within that period, if we do not come to some sort

OCR for page 1
122 ~ PANEL TO REVIEW THE 2000 CENS US of reasonable agreement and conclusion internally about whether it is a camera problem or a problem with what we are filming, then we would not be able to adjust. We struggled. Howard and his team struggled for hours ant} hours and hours and hours, to meet that deadline. At the end of the day, we realized that we had taken a good picture of a very bad census that had a lot of anomalies that we had not anticipated. But we struggled with that question. The issue of whether the A.C.E. itself will be good is a decision we will struggle with when we actually have the experience. We are obviously presuming a high response rate. What if we get a low response rate on the A.C.E. ? What does that mean? So do not imagine that that is an empty cell. We hone it is an em~tv cell. but theoreticalIv it is not an empty cell. ~ ~ , , , Now you also have the problem of badlbad. You have every reason to believe that you had a bad operational census. All kinds of things go wrong. You have natural disasters. You have public relations disasters. You were not able to hire the labor pool that you needed. But you do not know if the A.C.E. is good enough to sort of correct for that. That is a very tough cell to be in. It is very hard sitting here today to know how we will invoke the criteria to struggle with a decision in that cell. We think that is a Jow-probability cell. A lot of things would have to go wrong in the next year that we think will not go wrong. Nevertheless, we have to theoretically hold it open. I would say that, under the circumstances we feel we have had, using as many data points as we can use, as many indicators, and so forth, for a census that has serious measurement problems and an A.C.E. that worked well, we will adjust. If we have a very good census and an A.C.E. that worked well, we will have a more complicated decision at the margins. If we have a good census and a bad A.C.E., we obviously will not adjust. If we have bad/bad, we will reconvene the pane] and ask for help. That is king! of how I see it. I would just conclude with one other thought. There has been a Tot of talk about the response rate. We are putting a Tot of time and effort into trying to improve that response rate. Some people in this room indeed, I will not name the person he is not confident that we are going to improve much on the response rate. He is showing such lack of confidence in my leadership on the response rate issue that he has wagered me. I would say that I am open to any wagers from anyone in the room. In fact, I made one the day before yesterday with the Government Accounting Office, which has just, as you know, written a report saying that they think that we have been too optimistic about the response rate. We set a number it was a friendly bet and then agreed that it is that number plus 10 percent going in both directions. If we get 4 percent, we will add 40 percent to the base number. If we go down 4 percentage points that is a wager I am willing to mane with anyone in the room, on exactly those terms. We set a number at 61. If it is 62, you will need that number plus 10 percent, 63, so forth and so on, and I owe you that much more going below 61. I am very confident. I just invite anyone who wants to join the wager to do so.

OCR for page 1
PROCEEDINGS, SECOND WORKSHOP ~ 123 Thank you, Janet. Thank you, panel. DR. NORWOOD: Thank you very much, Ken. I want to thank everyone who has been here and all the people who have worked so hard. I especially want to thank Howard and his staff for all the hard work and for being on the hot seat for so long. It is not over, Howard. We are still in operation. We will be after you. I want to thank Raj [Singh] for all the help he has given us in working things out. The meeting is now adjourned.