The Bureau of Justice Statistics (BJS) of the U.S. Department of Justice, Office of Justice Programs (OJP), requested that the Committee on National Statistics (in cooperation with the Committee on Law and Justice) convene this Panel to Review the Programs of the Bureau of Justice Statistics. The panel has a broad charge to:
examine the full range of programs of the Bureau of Justice Statistics (BJS) in order to assess and make recommendations for BJS’ priorities for data collection. The review will examine the ways in which BJS
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 333
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics –B– Summary of Surveying Victims: Options for Conducting the National Crime Victimization Survey This appendix reprints the executive summary of the panel’s interim report Surveying Victims: Options for Conducting the National Crime Victimization Survey (National Research Council, 2008b), which included all of that report’s findings and recommendations. The only change to the text as it appeared in the interim report is to change the numbering scheme for the findings and recommendations to include the prefix “Int-” to avoid confusion with finding and recommendation numbers in this report. The Bureau of Justice Statistics (BJS) of the U.S. Department of Justice, Office of Justice Programs (OJP), requested that the Committee on National Statistics (in cooperation with the Committee on Law and Justice) convene this Panel to Review the Programs of the Bureau of Justice Statistics. The panel has a broad charge to: examine the full range of programs of the Bureau of Justice Statistics (BJS) in order to assess and make recommendations for BJS’ priorities for data collection. The review will examine the ways in which BJS
OCR for page 334
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics statistics are used by Congress, executive agencies, the courts, state and local agencies, and researchers in order to determine the impact of BJS programs and the means to enhance that impact. The review will assess the organization of BJS and its relationships with other data gathering entities in the Department of Justice, as well as with state and local governments, to determine ways to improve the relevance, quality, and cost-effectiveness of justice statistics. The review will consider priority uses for additional funding that may be obtained through budget initiatives or reallocation of resources within the agency. A focus of the panel’s work will be to consider alternative options for conducting the National Crime Victimization Survey, which is the largest BJS program. The goal of the panel’s work will be to assist BJS to refine its priorities and goals, as embodied in its strategic plan, both in the short and longer terms. The panel’s recommendations will address ways to improve the impact and cost-effectiveness of the agency’s statistics on crime and the criminal justice system. [emphasis added] BJS specifically requested that the panel begin its work by providing guidance on options for conducting the National Crime Victimization Survey (NCVS), one of many data series sponsored by BJS and one that consumes a large share (as much as 60 percent) of the agency’s annual appropriations. This interim report responds to this request. Since the survey began full-scale data collection in the early 1970s, the NCVS has become a major social indicator for the United States. Serving as a complement to the official measure of crimes reported to the police (the Uniform Crime Reporting [UCR] program administered by the Federal Bureau of Investigation), the NCVS has been the basis for better understanding the cost and context of criminal victimization. However, and particularly over the course of the last decade, the effectiveness of the NCVS has been undermined by the demands of conducting an increasingly expensive survey in an effectively flat-line budgetary environment. In order to keep the survey going in light of tight resources, BJS has reduced the survey’s sample size over time, and other design features have been altered. When the survey began in 1972, the sample of addresses for interviewing numbered 72,000; in 2005, the NCVS was administered in about 38,600 households, yielding interviews with 67,000 people. Although this sample size still qualifies the NCVS as a large data collection program, occurrences of victimization are essentially a rare event relative to the whole population: many respondents to the survey do not have incidents to report when they are contacted by the survey. At present, the sample size is such that only a year-to-year change of 8 percent or more in the NCVS measure of violent crime can be deemed statistically to be significantly different from no change at all. In its reports on the survey, BJS has to combine multiple years of data in order to comment on change over time, which is less desirable than an annual measure of year-to-year change.
OCR for page 335
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics In approaching this work, the panel recognizes the fiscal constraints on the NCVS, but we do not intend to be either strictly limited by them or completely indifferent to them. Rather, our approach is to revisit the basic goals and objectives of the survey, to see how the current NCVS program meets those goals, and to suggest a range of alternatives and possibilities to match design features to desired sets of goals. PRESERVING THE VICTIMIZATION MEASURE There are no nationally available data on crime and victimization—collected at the incident level, with extensive detail on victims and the social context of the event—except those collected by the NCVS. It is this basic fact that is the strongest argument for the continuation and maintenance of the survey. Certainly, one option for the future of the NCVS—and the ultimate cost-reducing option—is to suspend or terminate the survey. It is an option that would have to be considered, if budget constraints require further reductions in sample size. To be clear, though, abandonment of the NCVS is not an option that we favor in any way. Annual national-level estimates from the NCVS are routinely used in conjunction with the UCR to describe the volume and nature of crime in the United States. There is great value in having two complementary but nonidentical systems—the NCVS and the UCR—addressing the same phenomenon, for the basic reason that crime and victimization are topics that are too broad to be captured neatly by one measure. The police are not a disinterested party when it comes to characterizing the crime problem, and it is unwise to have data generated by the police as a sole measure of crime nationally. The UCR tells us little about the victims of crime; although its National Incident-Based Reporting System (NIBRS) has the potential to capture some of the detail currently measured by the NCVS, NIBRS has substantial limitations and remains incapable of providing national-level estimates after 20 years of implementation. Moreover, it is clear that a substantial proportion of crime is not reported fully and completely to law enforcement authorities. Thus, there remains a vital role for a survey-based measure that sheds light on unreported crime. Recommendation Int-3.1: BJS must ensure that the nation has quality annual estimates of levels and changes in criminal victimization. The current design of the NCVS has benefited from years of experience, methodological research, and evaluation; it is a good and useful model that has been adopted by international victimization surveys as well as subnational surveys within the United States. The principal fault of the current NCVS is not a design flaw or methodological deficiency, or even that the de-
OCR for page 336
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics sign inherently costs too much to sustain, but rather—simply—that it costs more than is tenable under current budgetary priorities. In its present size and configuration, the NCVS can permit insights into the dynamics of victimization. However, in our assessment, the current NCVS falls short of the vibrant measure of annual change in crime that was envisioned at the survey’s outset. Finding Int-3.1: As currently configured and funded, the NCVS is not achieving and cannot achieve BJS’s legislatively mandated goal to “collect and analyze data that will serve as a continuous and comparable national social indication of the prevalence, incidence, rates, extent, distribution, and attributes of crime …” (42 USC § 3732(c)(3)). By several measures—comparison with the expenditures of foreign countries for similar measurement efforts or with the cost of crime in the United States—the NCVS is underfunded. Accordingly, the panel recommends that BJS be afforded the budgetary resources necessary to generate accurate measures of victimization, which are as important to understanding crime in the United States as the UCR measure of crimes reported to the police. Recommendation Int-3.2: Congress and the administration should ensure that BJS has a budget that is adequate to field a survey that satisfies the goal in Recommendation Int-3.1. OVERALL GOAL AND DESIGN CONSIDERATIONS In considering historical goal statements of the NCVS, as well as new ones, we find three basic goals to be particularly prevalent and important, in addition to the previously expressed goal of maintaining annual national-level estimates of victimization that are independent of official reports to the police: Flexibility, in terms of both content (capability to provide detail on the context and etiology of victimization and to assess emerging crime problems, such as identity theft, stalking, or violence against and involving immigrants) and analysis (providing informative metrics beyond basic crime rates); Utility for gathering information on crimes that are not well reported to police or on hard-to-measure constructs (e.g., crimes against adolescents, family violence, and rape); and Small-domain estimation, including providing information on states or localities, which we think will be crucial to maximizing the utility of the NCVS and to building and maintaining constituencies for the survey.
OCR for page 337
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics In this report, we describe various design possibilities and their implications relative to these goals; however, we do not suggest one single path as the ideal for a redesigned NCVS. In part, this is because it is difficult to justify the case that our preferred set of NCVS goals is correct to the exclusion of all others; in part, it is because of the short time frame and the sequencing of this report (since it is inherently difficult to try to consider NCVS in isolation from the balance of BJS programs). But in large part we refrain from expressing a single, unequivocal path because the potential effectiveness and cost implications of some major design choices are simply unknown at this time. We do think that it is critical to emphasize that even small changes to the design of a survey can have significant impacts on resulting estimates and the errors associated with them. Design changes made in the name of fiscal expedience, without grounding in testing and evaluation, are highly inadvisable. They risk unexplained changes in the time series and confusion among users. Recommendation Int-4.1: BJS should carefully study changes in the NCVS survey design before implementing them. One potential cost-saving design choice is to change from asking respondents to recall and describe crime incidents in a 6-month window to using a 12-month window. This would entail contacting households once a year rather than twice (and, presumably, only 3 or 4 times if one chose to keep with the current regime of keeping households in the sample for 3.5 years). This would reduce the per-unit interviewing cost and free up resources to add additional sample addresses within each single year; 12 months is also the common reference period in victimization surveys in other countries. However, it could also increase problems of recall error by making respondents search their memories over a longer period. On its conceptual strengths and its use in comparable crime surveys in other western nations, we prefer a switch to a 12-month reference period as a cost-saving mechanism over options that would simply reduce the total sample size. That said, the empirical case for implementing this change is not completely clear and warrants up-to-date research. We note that such a move requires an overlap of designs over time to safely incorporate the change to 12 months. Recommendation Int-4.2: Changing from a 6-month reference period to a 12-month reference period has the potential for improving the precision per-unit cost in the NCVS framework, but the extent of loss of measurement quality is not clear from existing research based on the post-1992-redesign NCVS instrument. BJS should sponsor additional research—involving both experimentation as well as analysis of the timing of events in extant data—to inform this trade-off.
OCR for page 338
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics It is also the case that cost savings might be achieved by refining the NCVS sample stratification schemes. The current multistage cluster design of the NCVS automatically includes households sampled from counties and other geographic regions with large population sizes, clustering the remaining geographic areas by social and demographic information to produce similar strata from which the remaining sample is drawn. The composition of the sample is relatively slow to change with each decennial census, although effort is made to include some new housing stock by sampling from housing permit data. If the NCVS continues to be conducted by the Census Bureau (see “Collecting the Data,” below), particular insight for altering the basic sample design and modifying sample strata based on an up-to-date sampling frame could come from interaction with the new American Community Survey (ACS). But, again, quantitative methodological research that could suggest exactly what benefits might or might not accrue is lacking. Recommendation Int-4.7: BJS should investigate changing the sample design to increase efficiency, thus allowing more precision for a given cost. Changes to investigate include: changing the number or nature of the first-stage sampling units; changing the stratification of the primary sampling units; changing the stratification of housing units; selecting housing units with unequal probabilities, so that probabilities are higher where victimization rates are higher; and alternative person-level sampling schemes (sampling or subsampling persons within housing units). As early as 1980, the NCVS began the use of multiple response modes. Face-to-face personal interviews after the first contact with a sample household were replaced with interviews conducted by telephone, and—after the 1992 implementation of the full NCVS redesign—some interviewing began to be done by Census Bureau computer-assisted telephone interviewing (CATI) centers using a fully automated survey instrument. The NCVS path to automation has been somewhat complicated: full conversion to nonpaper survey questionnaires was achieved only in 2006, and—as part of the most recent round of cost reductions—BJS and the Census Bureau abandoned the use of the centralized CATI centers for NCVS interviews because anticipated cost savings never occurred. However, as redesign possibilities are considered, it is important that BJS continue to seek automation possibilities and not be limited to the NCVS traditional interview formats. A particular area of focus should be self-response options, such as computer-assisted self-interviewing (effectively, turning the interviewer’s laptop around so that the respondent answers questions directly) or Internet response for interviews
OCR for page 339
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics after several visits. As with the central CATI centers, cost savings from new modes of data collection are not guaranteed, but they may put the survey in good stead for implementing new topical modules and promoting high respondent cooperation. They can also serve to reduce overall respondent burden. Recommendation Int-4.8: BJS should investigate the introduction of mixed mode data collection designs (including self-administered modes) into the NCVS. The NCVS is subject to the same pressures facing all household surveys in modern times, whether federal or private. It is increasingly difficult (and expensive) to obtain survey responses from persons or households in an age of cell phones, call waiting, and Internet chat. A significant fraction of survey costs are incurred to contact the most hard-to-find respondents. In considering design possibilities, it is important that BJS try to develop schemes that are relatively robust to declines in response rate, as such declines are virtually certain. Recommendation Int-4.9: The falling response rates of NCVS are likely to continue, with attendant increasing field costs to avoid their decline. BJS should sponsor nonresponse bias studies, following current OMB guidelines, to guide trade-off decisions among costs, response rates, and nonresponse error. BUILDING AND REINFORCING CONSTITUENCIES A continuing challenge for the NCVS is the development of constituencies with a strong interest in the data and their quality. The public is aware of the NCVS mainly due to one regular constituency—the media—and the spate of crime uptick or downtick stories that accompanies each year’s release of NCVS and UCR estimates. Likewise, findings from topical supplements (such as racial dimensions of traffic stops, measured by the Police-Public Contact Survey supplement) typically get prominent press coverage. Official statistics, like other societal infrastructures, are often highly valued but rarely passionately promoted by day-to-day users. However, the long-term viability of the survey depends crucially on building and shoring up constituencies for NCVS products and on cultivating the survey’s user base among researchers. Small-Domain Estimates The world has changed since the mid-1970s—computers are more powerful, data users are more sophisticated, and the demand for small-area geographic data is more insatiable. It is too strong to say that the NCVS can
OCR for page 340
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics remain relevant only if it provides estimates for areas or populations smaller than the nation as a whole: state and local governments, which are among the most prodigious of NCVS users, continue to find national benchmarks very valuable. However, the survey will increasingly grow out of step with potential constituencies if it cannot be used to provide estimates for smaller areas. Recommendation Int-4.5: BJS should investigate the use of modeling NCVS data to construct and disseminate subnational estimates of major crime and victimization rates. This recommendation runs counter to the principal effect of one of our predecessor National Research Council (1976) panel’s recommendations—that the separate “impact city” victimization surveys that were originally part of the National Crime Surveys suite should be terminated. However, it is very much consistent with that previous recommendation’s focus on an integrated set of estimates, including subnational geographies. These subnational estimates need not be exhaustive: expanding the sample to support estimates for the largest metropolitan statistical areas is a more sensible and cost-effective approach than a system for generating estimates for all 50 states. But they should permit insight on victimization for some smaller units than the nation as a whole. Small-domain estimates also refer to estimates by other social or demographic constructs, such as urbanicity (urban, suburban, or rural), in addition to the basic disaggregation by major race-ethnicity groups that is currently done. With particular regard to the generation of small-domain estimates, it should be noted that enhancing the NCVS to better serve constituencies is not strictly a process of addition, in terms of sample size or implementation of a full supplemental questionnaire. In some important respects, user constituencies may best be served by more creative use of the current NCVS design. In the years since National Research Council (1976) advocated eliminating the city surveys, statistical developments in small-domain estimation techniques have been considerable; hence, some small-domain estimates may be possible through modest investment by BJS in technical infrastructure for statistical modeling tasks. In addition to small-domain modeling using NCVS data, it may also be useful to explore ways to strengthen victimization surveys conducted by states and localities. Currently, BJS operates a program under which it develops victimization survey software and provides it to interested local agencies; however, those agencies must supply all the resources (funds and manpower) to conduct a survey. An approach to strengthen this program would be to make use of BJS’s organizational position within the U.S. Department of Justice. The bureau is housed in the Office of Justice Programs, the core mission of which is to provide assistance to state and local law enforcement
OCR for page 341
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics agencies; it does so through the technical research of the National Institute of Justice and the grant programs of the Bureau of Justice Assistance (BJA), among others. We suggest that OJP consider ways of dedicating funds—like BJA grants, but separate from BJS appropriations—for helping states and localities bolster their crime information infrastructures through the establishment and regular conduct of state or regional victimization surveys. Such surveys would most likely involve cooperative arrangements with research organizations or local universities and make use of the existing BJS statistical analysis center infrastructure. This approach is analogous to the Behavioral Risk Factor Surveillance System (BRFSS) of the Centers for Disease Control and Prevention, and it is similar in its partnership arrangements to the Federal-State Cooperative Program for Population Estimates (FSCPE) of the Census Bureau. Recommendation Int-4.6: BJS should develop, promote, and coordinate subnational victimization surveys through formula grants funded from state-local assistance resources. We discuss an extreme interpretation of this approach—wherein the “national” victimization survey would be effectively be the combination of the subnational surveys—in Chapter 4 [of the interim report]. However, we emphasize that we suggest that this BRFSS/FSCPE approach should be considered independent of (and as a complement to) the chosen design of the NCVS. Topic Constituencies The NCVS first added a topic supplement to the survey questionnaire in 1977, querying respondents on their perceptions of the severity of crime. Particularly since 1989, supplements have been an irregular part of the NCVS structure; the School Crime Supplement on school safety has been repeated six times and the Police-Public Contact Survey three times, with other supplements being (to date) one-time efforts. A strong program of topic supplements is an important part of the NCVS, both because of the breadth of topics that may be handled and because the ability to quickly field questions on new topics of interest is a key advantage of survey-based collection compared with official records. Recommendation Int-4.3: BJS should make supplements a regular feature of the NCVS. Procedures should be developed for soliciting ideas for supplements from outside BJS and for evaluating these supplements for inclusion in the survey. What is necessary regarding NCVS supplements is a more structured plan for their implementation, better exploration (and marketing) of sponsorship opportunities by other state and federal agencies, and greater transparency
OCR for page 342
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics in real costs of conducting a supplement. Regardless of the overall design of the NCVS, the British Crime Survey offers an attractive model: a streamlined core set of questions combined with a planned, regular slot for topical content. Recommendation Int-4.4: BJS should maintain the core set of screening questions in the NCVS but should consider streamlining the incident form (either by eliminating items or by changing their periodicity). This would reduce respondent burden and allow additional flexibility for adding items to broaden and deepen information about prevalent crimes. ATTENTION TO DATA QUALITY AND ACCESS We make a series of recommendations that are agency-level in focus, aimed at better equipping BJS to understand its own products and to interact with its users. They are presented here in initial form because they are pertinent to the NCVS. We expect to expand on them in our final report on the full suite of BJS programs and products. First, BJS currently receives periodic advice from the Committee on Law and Justice Statistics of the American Statistical Association (ASA). Although this input is certainly valuable, we think that BJS—and the NCVS in particular—would benefit from the commissioning of an ongoing scientific technical advisory board, such as is in place for other statistical agencies. This board should include subject-matter, survey methodological, and statistical expertise; spots on the board are also a vehicle for strengthening stakeholder constituencies for the NCVS. Recommendation Int-5.1: BJS should establish a scientific advisory board for the agency’s programs; a particular focus should be on maintaining and enhancing the utility of the NCVS. Several of our recommendations listed earlier identify gaps in existing research that must be filled to accurately inform trade-offs in design choices. More generally, the NCVS developmental work in the 1970s and the research conducted as part of the 1980s redesign effort are extensive, but we think that there is a paucity of recent methodological research making use of the post-1992-redesign NCVS instrument and techniques. BJS has already made some strides in fostering methodological research with its fellowship program, operated in conjunction with the ASA. We urge BJS to continue this work and to explore other creative ways to foster internal and extramural research using the NCVS and other BJS data sets, including graduate fellowships, as part of continuous efforts to assess the quality of NCVS estimates.
OCR for page 343
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics Recommendation Int-5.3: BJS should undertake research to continuously evaluate and improve the quality of NCVS estimates. Conceptually, the survey-based NCVS is ideally suited (as the official record-based UCR is not) to study the dynamics of crimes that are emotionally or psychologically sensitive, such as violence against women, violence against adolescents, and stalking or harassment. We urge BJS to develop lines of research to ensure that such crimes are accurately measured on the NCVS instrument; these might include the testing of self-response options, such as audio computer-assisted interviewing. Recommendation Int-3.3: BJS should continue to use the NCVS to assess crimes that are difficult to measure and poorly reported to police. Special studies should be conducted periodically in the context of the NCVS program to provide more accurate measurement of such events. The quality of NCVS data and its scientific rigor in measuring crime should always be the survey’s primary goal and acknowledged as its principal benefit. However, for the purpose of cultivating constituencies and users for the survey, attention to the accessibility and the ease of use of NCVS data is also vitally important. Part of this work involves reevaluation of basic products and reports from the NCVS and expansion of the range of analyses based on the data, and it involves both in-house research by BJS and effective ties with other users and researchers. Recommendation Int-5.2: BJS should perform additional and advanced analysis of NCVS data. To do so, BJS should expand its capacity in the number and training of personnel and the ability to let contracts. A necessary consequence of this recommendation is that the agency must expand its capacity, both in the number and training of personnel and the agency’s ability to let contracts for external research. Recommendation Int-5.4: BJS should continue to improve the availability of NCVS data and estimates in ways that facilitate user access. Recommendation Int-5.5: The Census Bureau and BJS should ensure that geographically identified NCVS data are available to qualified researchers through the Census Bureau’s research data centers, in a manner that ensures proper privacy protection. In the case of this last recommendation, we understand that arrangements to place detailed NCVS data at the research data centers are under development; we state it here as encouragement to finalize the work.
OCR for page 344
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics COLLECTING THE DATA It is important to note that some of the resource constraints on the NCVS are common to those on other important federal surveys, which have faced difficulties carrying out basic maintenance tasks like updating samples to reflect new census and address list information. The country needs a mechanism to alert itself to budget cuts that undermine the basic purposes of key federal statistical products. Recommendation Int-5.6: The Statistical Policy Office of the U.S. Office of Management and Budget is uniquely positioned to identify instances in which statistical agencies have been unable to perform basic sample or survey maintenance functions. For example, BJS was unable to update the NCVS household sample to reflect population and household shifts identified in the 2000 census until 2007. The Statistical Policy Office should note such breakdowns in basic survey maintenance functions in its annual report Statistical Programs of the United States Government. Any review of a major survey program—particularly one carried out with an eye toward cost reduction—must inevitably raise the question of the agent that collects the data: could survey operations be made better, faster, or cheaper by getting some other organization to carry out the survey? In this case, the U.S. Census Bureau’s involvement with the NCVS predates the formal establishment of the survey, as the Census Bureau convened planning discussions and conducted NCVS pilot work. The optimal decision on who should do the data collection for the NCVS will depend on the weight that one puts on desired objectives for the survey. For instance, an extremely strong weight on flexibility and quick response to emerging trends might argue against the Census Bureau, where implementation of a supplement can be made time-consuming through detailed cognitive testing (which ultimately improves the quality of the questions but can be slow) and passage through bureaucratic channels (e.g., clearance by the Office of Management and Budget, as required of all federal surveys). However, dominant weight on maintaining high response rates and drawing from the experience of other large, ongoing surveys would suggest that staying with the Census Bureau is the best course. Just as we do not offer a single design path for the NCVS, we do not find justification for offering a conclusion on “Census Bureau” or “not Census Bureau.” Based on the advantages and disadvantages, we suggest that “privatizing” the NCVS is not the panacea for high survey costs that some may believe it is. We have been provided no way of estimating the various costs associated with switching NCVS data collection agents; however, it is altogether appropriate to consider means of getting detailed and specific answers to these questions.
OCR for page 345
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics In the interim, we suggest that the Census Bureau would benefit both BJS and itself itself by providing greater transparency in true survey costs. Recommendation Int-5.7: Because BJS is currently receiving inadequate information about the costs of the NCVS, the Census Bureau should establish a data-based, data-driven survey cost and information system. We further suggest that BJS consider a design competition—providing some funds for bidders to specify in detail how they would conduct a victimization survey. This design competition would effectively compensate bidders for their time in developing proposal specifications, but it should be run with a statement that a formal request for proposals may result from the competition (and not that it will definitely occur). Recommendation Int-5.8: BJS should consider a survey design competition in order to get a more accurate reading of the feasibility of alternative NCVS redesigns. The design competition should be administered with the assistance of external experts, and the competition should include private organizations under contract and the Census Bureau under an interagency agreement.
OCR for page 346
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics This page intentionally left blank.