| ||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 277
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
–6–
Strategic Goals for the Bureau of Justice Statistics
6–A
PRIORITY SETTING AND CONSTRAINED RESOURCES
A POSSIBLE CRITICISM of this report—reviewing the detailed recommendations made throughout the text—is that it tends to suggest adding to the Bureau of Justice Statistics’ (BJS’s) inventory of collections rather than subtracting. The recommendations are generally geared to improvements within BJS’s various existing data collections—for instance, ensuring a high-quality independent measure of crime in the National Crime Victimization Survey (NCVS), emphasizing conceptual frameworks in BJS’s adjudication and law enforcement collections, and expanding its corrections series to study prisoner reentry into society. (There are exceptions, such as Recommendation 3.10’s suggestion to discontinue special-focus law enforcement agency surveys if they cannot be fitted into a regular, structured Law Enforcement Management and Administrative Statistics [LEMAS] supplement plan.)
The panel raised its own questions regarding this: How should these improvements be paid for? Should some existing series be cut to make ways for new ones? And, given the disproportionate share of BJS’s current resources consumed by the NCVS, should some smaller collections be stopped to free up additional resources for the NCVS or should some NCVS resources be steered to other areas?
OCR for page 278
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
The reason why we rejected explicitly suggesting that some data series be cut in order to pay for others is certainly not that the current collections are perfect. The improvements we suggest in the recommendations are testimony to that. Rather, we cannot accept the underlying premise and inherent assumption that BJS can achieve its legislated goals by cutting programs. In our assessment, we think it can be stated as a fact: BJS has been given more responsibilities than can be achieved with current resources. The resources provided to BJS to carry out its work are not commensurate with the breadth—and importance—of the responsibilities assigned to the agency by its authorizing legislation.
Because of this, the agency has for some years walked a tight line of small cuts of sample or measurement, short delays of publications, and temporary hiring freezes—each of these tolerable in itself, but cumulating over the years such that core functions have broken down. On a routine basis, decisions must be made about addressing certain responsibilities and not addressing others; trade-offs must be made in the periodicity and completeness of data collections. Maintenance and continuation of existing data collections must also be balanced with the need to comply with directives from Congress and the Department of Justice, complicating resource allocation decisions and the setting of priorities. Such decisions are hardly unique to BJS—at some point, all organizations must make such trade-offs—but BJS’s mismatch between resources and responsibility makes the decisions particularly difficult.
Thus, in setting priorities, BJS directors have perforce had a short time horizon—responding to a certain set of demands even though those decisions may have negative long-term consequences for individual data collections and the health of the agency. Certainly, in the midst of year-to-year juggling of data series in order to keep production moving, longer-term investments in research and innovation become difficult or impossible to make. The most striking example of the consequences of this extremely tough climate is the current state of the NCVS: What was once, clearly, the best victimization survey in the world is now unable to satisfy its basic function of providing annual estimates of level and change in common-law crime. This decay happened gradually as BJS administrators were attempting to respond to immediate exigencies, aggravated by an overly broad mandate. Each single cut in sample size, or other cost-cutting measure, was justifiable given then-present alternatives. Cumulatively—as demonstrated most vividly by the declared “break in series” with the 2006 NCVS data—they lead to the conclusion that “the [current] NCVS is not achieving and cannot achieve BJS’s legislatively mandated goals” (National Research Council, 2008b:Finding 3.1).
OCR for page 279
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
Statistical infrastructure (data collections and records series) shares with physical infrastructure (such as roads, bridges, sewers, and cable) the fundamental problem that it lacks glamor and can be difficult to champion as a government spending priority. Yet it is undeniably important to the public knowledge and the public good; BJS’s purview includes topics important to general welfare, spanning the measurement of interpersonal violence, the function and magnitude of law enforcement and corrections, and the operations of the judicial branch of government. The problem is not that there are parts of BJS’s legal mandate that are unimportant or unworthy, but that its current resources do not permit it to cover its mandates as effectively as possible. Given this finding, we are loath to construct a list of series to terminate or reduce out of concern that the agency fail even more visibly to fulfill the charge given to it in legislation.
Another reason for not considering a specific ranking of data collections by importance, or some metric, in order to suggest possible cuts is consistency with the approach we took in our interim report on options for the NCVS. In that report, we presented an array of possible options and described how specific design choices corresponded to particular goals for the survey. However, we made a point of “not suggest[ing] one single path as the ideal for a redesigned NCVS” (National Research Council, 2008b:4). Different people and decision makers do not necessarily put the exact same weights on the goals and objectives of a program such as the NCVS, and we did not want presume “that our preferred set of NCVS goals is correct to the exclusion of all others” (National Research Council, 2008b:4). The same logic applies to considering other collections in the BJS portfolio: deeming one collection more worthy than another involves complicated value judgments, not science. Specific constituencies for BJS data that are not represented on the panel or by groups who have spoken before the panel could make eloquent and compelling cases for their particular favored set of statistics; again, we do not wish to suggest that any weighting we could suggest is somehow paramount.
Ideally, in evaluating and weighing individual data collection programs, it would be possible to know the causal impact of particular BJS statistics—for instance, if BJS were to collect data and report estimates on X rather than Y, certain policy actions would result (or have different probabilities of resulting). Such causal analysis is very difficult and, aside from some studies on federal fund allocation based on government estimates (see, e.g., Spencer, 1980), very little work has been done to attempt even a partial understanding of the causal impact of government statistics. Although such analyses would be a worthwhile undertaking for BJS and the federal statistical system as a whole, it would involve such an extensive research agenda as to be infeasible for a small agency such as BJS. That said, there is merit in BJS devoting resources to understanding and documenting the ways in which its
OCR for page 280
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
estimates are actually used in making policy decisions at the federal, state, and local levels, and using that information to assess how well its statistical portfolio matches the needs of policy analysts.
Finding 6.1: Relatively little is understood about how BJS data are used in developing policy and could be used in improving policy development.
While hoping that tight fiscal constraints may be alleviated somewhat in coming years, we cannot assume infinite resources either. We recognize that our charge obliges us to “assist BJS to refine its priorities and goals, as embodied in its strategic plan, both in the short and longer terms.” Hence, in this chapter, we consider basic strategic goals for BJS, formalizing some notions inherent in our detailed recommendations and deriving from the broader themes we noted in Chapter 1.
6–B
CURRENT BJS STRATEGIC GOALS
To begin, it is useful to review BJS’s current plans and goals. The agency’s 2005–2008 strategic plan (Bureau of Justice Statistics, 2005a) articulates three goals for the agency:
To produce national statistics on crime and the administration of justice that facilitate measurement over time and across geographic areas;
To improve record-keeping by state and local governments and to improve the ability of states and localities to produce statistics on crime and the administration of justice;
To ensure public access to statistics and criminal justice data.
These goals mainly serve to reinforce the two parts of the agency’s formal mission statement—service as a statistical agency and support of state and local governments—with an added goal of public access and dissemination. As stated above, the first goal adds emphasis on “measurement over time,” though it is unclear whether this refers to simply continuing collections (thus building longer time series of data) or longitudinal measurement within the criminal justice system.
Although they are not written into the agency’s current strategic plan, other strategic priorities are evidenced by changes in BJS’s organizational structure (see Figure 1-3) and commentary at the panel’s public meetings. In particular, the organization was revised to create three distinct units—law enforcement, courts and adjudication, and “recidivism, reentry, and special projects” (Sedgwick, 2008:2). These changes provide resources separate from BJS’s existing correctional statistics unit to study prisoner reentry issues; they are also consistent with comments made by BJS staff in remarks to the panel about the agency’s interest in broadening its collections in the area of law enforcement.
OCR for page 281
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
Box 6-1
Summary of Recommendations and Commentary on Strategic Goal 1: A Strong Position of Independence
Move BJS Out of the Office of Justice Programs
Recommendation 5.3: BJS should be administratively moved out of the Office of Justice Programs, reporting to the attorney general or deputy attorney general.
Make the BJS Directorship a Fixed-Term Appointment
Recommendation 5.4: Congress and the administration should make the BJS director a fixed-term presidential appointee with the advice and consent of the Senate. To insulate the BJS director from political interference, the term of service should be no less than 4 years.
Strengthen BJS’s Independent Function as a Statistical Agency
Recommendation 5.2: The Department of Justice review of any BJS statistical product and related communications should not require changes to the content, the release schedule, or the mode of dissemination planned by BJS.
Recommendation 5.1: Congress and the Department of Justice should not require, and BJS should not provide, individually identified data in support of regulatory functions that compromise the independence of BJS or require BJS to violate any of the principles of a federal statistical agency.
Position BJS as a Statistical Resource to the Department of Justice
Recommendation 5.5: The BJS director needs to reach out to other agencies within DOJ, forming partnerships to propose initiatives for information collection that are relevant to policy needs.
Recommendation 5.6: The Department of Justice should build provisions for BJS collection of data and statistical information into its program initiatives aimed at crime reduction. These are not intended as program evaluation funds, but rather as funds for the basic monitoring and assessment of the phenomena targeted by the initiative.
6–C
SUGGESTED BJS STRATEGIC GOALS
In the following section, we propose four strategic goals for the Bureau of Justice Statistics. Each section includes a text box (Boxes 6-1, 6-2, 6-3, and 6-4) illustrating how our recommendations and commentary throughout this report support or relate to Goals 1–4, respectively.
6–C.1
A Strong Position of Independence
Goal 1: To establish and maintain a strong position of independence as a statistical agency; to serve as an independent and
OCR for page 282
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
Improve Ties Between BJS and the Federal Bureau of Investigation
As discussed in Section 4–C.4, a fully-functional National Incident-Based Reporting System (NIBRS) would be a valuable research tool. Accordingly, BJS should facilitate more universal implementation of NIBRS and offer technical support on issues such as small-agency bias, in order to promote better understanding of instances in which NIBRS data may best be used.
Build the National Crime Victimization Survey’s (NCVS’s) Standing as a Critical Social Indicator
Recommendation 6.1: BJS must ensure that the nation has quality annual estimates of levels and changes in criminal victimization.
Recommendation 6.2: Congress and the administration should ensure that BJS has a budget that is adequate to field a survey that satisfies the goal in Recommendation 6.1.
Recommendation 6.4: Additional resources made available for the NCVS should be used not only to increase the reliability of annual estimates but also to supplement the survey in ways that increase our understanding of criminal victimization.
Explore Means for Dedicated Funding of Victimization Studies
Recommendation 6.3: More information about the needs of victims is essential to the compensation and assistance goals of the Office of Victims of Crime. Congress should allow additional funding for the collection and improvement of victimization data to be obtained from funds obtained through the Victims of Crime Act.
Continue and Strengthen State Justice Statistics Program and Statistical Analysis Center Partnerships
Recommendation 4.1: Through its Statistical Analysis Center and State Justice Statistics programs, BJS should continue to develop its ties with the states, and more fully exploit the potential for using states as partners in data collections.
Recommendation 4.2: Developments toward longitudinal and small-area measurement systems should involve state partners who are active in data collection and knowledgeable about state justice systems.
objective source of statistical information on crime and the administration of justice.
In light of the discussion in Chapter 5 on BJS’s performance relative to the expected principles and practices of a federal statistical agency, it should not be surprising that building (and rebuilding) BJS’s position of independence ranks high in our suggested directions for the agency. In recent years, BJS has endured strong challenges to its position of independence, as documented in Section 5–A.2. Arguably, BJS’s most pressing priority in the coming years is restorative: rebuilding BJS’s position of independence as a statistical agency. We suggest an administrative shift of BJS out of the Office
OCR for page 283
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
of Justice Programs (OJP) and designation of a termed appointment for the BJS director as corrective measures. By this goal statement, we suggest that preserving a role of independence also be BJS’s first criterion for undertaking new data collections or revising existing ones. We recommend (Recommendation 5.1) that BJS should not provide individually identified data in support of regulatory functions that compromise BJS’s role; this goal statement formalizes and extends this clause to include acceptance of functions that involve collecting and analyzing data for policy-furthering, tactical, and operational purposes. Research on policy-specific evaluation issues should not be done by BJS but by other units within the Justice Department that are formally charged with this responsibility.
In contrast to the “statistical arm of the U.S. Department of Justice” verbiage that BJS has sometimes used to describe itself, it is useful to consider as a model the mission statement of the Bureau of Labor Statistics (BLS). In past years, BLS used similar “statistical arm” language. However, at least since 2001, BLS has taken the following as its mission statement (http://stats.bls.gov/bls/blsmissn.htm):1
The Bureau of Labor Statistics (BLS) is the principal fact-finding agency for the Federal Government in the broad field of labor economics and statistics. The BLS is an independent national statistical agency that collects, processes, analyzes, and disseminates essential statistical data to the American public, the U.S. Congress, other Federal agencies, State and local governments, business, and labor. The BLS also serves as a statistical resource to the Department of Labor.
BJS would be well served to adopt similar language in its self-description, emphasizing that it is first and foremost a statistical agency, and that it is a statistical resource to the Justice Department but not an “arm” to further any policy objective.
6–C.2
Building Statistical Systems and Conceptual Frameworks
Goal 2: To build, maintain, and utilize statistical systems that describe the extent and characteristics of crime in our nation and the status and response of the justice system.
More than a basic statement of topic, the language of this goal is intended to reframe BJS’s grantmaking programs such as the National Criminal History Improvement Program. As discussed in Chapter 4, a major problem with the current administration of such programs is that they are strictly a money-transfer operation with no statistical product given back to
1
The full statement includes the second paragraph: “BLS data must satisfy a number of criteria, including relevance to current social and economic issues, timeliness in reflecting today’s rapidly changing economic conditions, accuracy and consistently high statistical quality, and impartiality in both subject matter and presentation.”
OCR for page 284
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
Box 6-2
Summary of Recommendations and Commentary on Strategic Goal 2: Building Statistical Systems and Conceptual Frameworks
Articulate Interrelationships of Data Collections in Strategic Plan
Recommendation 3.7: To be useful, a BJS strategic plan must articulate a blueprint of interrelated data collection and product activities, including both current and potentially new data products. This blueprint would be used to evaluate new opportunities.
Use Criminal History Record Databases for Research Studies
Recommendation 4.3: BJS should actively utilize the NCHIP program to improve criminal history records necessary for longitudinal studies of crime.
Implement Core-and-Supplement Frameworks for the National Crime Victimization Survey (NCVS) and Law Enforcement Management and Administrative Statistics (LEMAS)
Recommendation 3.8: (Rec 4.3 from National Research Council, 2008b) BJS should make supplements a regular feature of the NCVS. Procedures should be developed for soliciting ideas for supplements from outside BJS and for evaluating these supplements for inclusion in the survey.
Recommendation 3.9: To maximize both utility and timeliness of information, the LEMAS survey should be conducted as core-supplement design in the context of a continuous data collection.
Recommendation 3.10: To improve the utility of censuses of law enforcement agencies, BJS should develop an integrated conceptual plan for their periodicity, publish a 5-year schedule of their publication, and integrate their measurement into the LEMAS as supplements.
Develop Broader Vision of Law Enforcement Than Management and Administration
As discussed in Section 3–F.2, BJS’s current data holdings in law enforcement are overly limited to administrative data (e.g., number of sworn officers or presence of certain policies or technologies). BJS should perform analyses linking these data to crime data (thus coming closer to informing assessment of the effectiveness of police policies).
Recommendation 3.11: The NCVS (and its supplements) should be more effectively used as a tool for studying law enforcement, both in terms of the types of crime that are reported (and not reported) to police and the action that results from the reporting of a crime (e.g., the Police-Public Contact Survey).
Pursue a Records-Based Law Enforcement Data Series
Recommendation 4.4: To improve the timeliness of crime statistics, BJS should explore the development of a crime reporting system based on a probability sample of police administrative records. The goals of such a system would be national representativeness, high response, high data quality, timeliness and flexibility in terms of crime classification and analysis, and national statistics for the monitoring of crime trends.
OCR for page 285
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
Explore Role of BJS’s Corrections Data Expertise Within the Federal Statistical System
As discussed in Section 5–B.11, BJS and the Census Bureau should reconcile the relative roles of BJS’s prison and jail censuses and inmate surveys and the group quarters portion of the Census Bureau’s American Community Survey to determine which series (or combination) provides the most accurate picture of the correctional population.
Stabilize and Improve Sampling in Adjudication Series
Recommendation 3.12: As court records become more accessible through computerized case management systems, BJS should implement more rigorous methods of probability sampling in its adjudication series.
Recommendation 3.13: To inform future revisions to its adjudication portfolio and to more efficiently acquire and work with court data in the future (including longitudinal analysis), BJS should develop a research program to build representative samples of courts and to assess strategies for collection of case records.
Develop Better Understanding of Prosecutor Declinations and Other Adjudication Decisions
As discussed in Section 3–F.3, prosecutors’ declinations to prosecute certain cases remain a major gap in understanding in the justice system flow model. Ideally, fuller data will arise as participation in State Court Processing Statistics–type collection becomes more common and standard; as a first check, BJS should add some basic questions on case processing to the National Prosecutors Survey, which currently has a purely administrative focus.
Outline Research Agenda for Civil Justice Collections
As discussed in Sections 2–C.2 and 2–D, civil proceedings are a major part of the overall justice system—and a major gap in BJS’s data portfolio, relying essentially on one collection. But an expanded role for BJS in covering civil justice issues raises serious definitional issues, and developing a measurement system capable of taking an accurate reading of civil disputes settled privately is a very difficult prospect. BJS should clearly articulate the strengths and weaknesses of its current work in civil justice and identify the highest priority for expanded data collection, should resources become available.
the taxpayer. Other than generating rough summary statistics of firearm-purchase background checks, BJS has not been able to utilize the criminal history record data that its grant monies help to develop at the state and local levels. This failure has occurred, despite a formal set-aside of funds in all of those grants for BJS evaluation purposes. However, as also described in Chapter 4, recent developments have finally made it possible for BJS to access the compiled history record data for research purposes, opening very exciting avenues for study.
OCR for page 286
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
To be clear, we consider methodological research to be an essential part of building and maintaining statistical systems. Although attention to innovation and methodological development is difficult in a climate of constrained resources, it is essential to the effective improvement or expansion of statistical systems. A higher priority on methodological research is not a panacea; it might not have prevented the necessity of the cost-cutting measures that produced the recent “break in series” in the NCVS but it could have provided a fuller assessment of possible risks before those changes were finalized.
By use of the term “statistical systems,” this goal statement would give lesser priority to one-time data collections that have no enduring benefit to ongoing series or planned data collections. This is not meant to stifle new one-time collections but to ensure that additions to the portfolio are done in a broader context. For example, it is not meant to say that first-time efforts like a census of law enforcement aviation units should not be undertaken, but rather that their role should be made clear (as part of the larger LEMAS survey and as a means of building and improving the frame for that broader survey) and that its utility in describing part of the justice system should be explicated.
We also consider “statistical systems” to include the partnerships between BJS and state and local authorities—partnerships that, properly, go beyond “improve[ment of] record-keeping” and dissemination of BJS products.
6–C.3
Improving Coverage of the Justice System
Goal 3: To provide comprehensive statistical coverage of all parts of the criminal justice system, including the longitudinal flow of persons and cases through the system and their return to the community.
The specific extents of what is called “the criminal justice system” can be debated. In general, the criminal justice system includes the criminal act, consequences of the act, and responses to the act by public agencies and other parties. We think the 1967 funnel model of the system remains useful and have taken it as a point of reference. Accepting this model as a premise, this goal might be restated as saying that data collections rise and fall in importance relative to their proximity to the funnel: the further a specific phenomenon is from the activities described by the funnel model, the less central it is to the mission of BJS.
By its nature, the funnel model has strong implications for setting priorities that merit further discussion. First, it suggests that BJS should focus on activities that affect the most people and affect them most extensively. By definition, this means the incidence of crime and victimization, which the funnel graphically describes as the most prevalent phenomenon in the crim-
OCR for page 287
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
Box 6-3
Summary of Recommendations and Commentary on Strategic Goal 3: Improving Coverage of the Justice System
Emphasize Longitudinal Flows and Structures
Recommendation 3.4: BJS should develop an approach to measure the experiences of individuals through the criminal justice system on a prospective, longitudinal basis, beginning as early as practicable in the process (arrest) and ending with their eventual exit (ranging from early dismissal of charge through completion of sentence).
Recommendation 3.5: BJS should develop an approach to measure the victimization experiences of individuals on a prospective, longitudinal basis, beginning from a focal victimization and following the victim forward in time measuring subsequent victimizations and possible consequences of victimization. The NCVS may be used to recruit respondents to a panel survey of crime victims.
Recommendation 3.1: BJS’s goal in providing statistics from basic administrative data on corrections should be the development of a yearly count of correctional populations capable of disaggregation and cross-tabulation by state, offense categories, and by demographic groups (age, race, gender, education).
Recommendation 3.2: BJS should produce yearly transition rates between steps in the corrections process capable of disaggregation and cross-tabulation by state, offense categories, and demographic groups.
Recommendation 3.6: BJS should develop a panel survey of people under correctional supervision to understand how individuals move between institutional and community settings, and to understand the social contexts of correctional supervision.
Facilitate Linkage in Existing Data Sets
Recommendation 3.3: BJS should explore the possibilities of increasing the utility of their correctional data collections by facilitating the linkage of records across the data series. For example, the ability to link records from the Recidivism Studies or from NCRP to the Census of Adult Correctional Facilities (CACF) would increase the ability to understand how correctional facilities contribute to recidivism.
Build Capacity for Studying New and Emerging Types of Crime
Recommendation 2.1: Consistent with its legal mandate to collect, analyze, and disseminate statistical information on all aspects of the justice system, BJS should (a) document and organize the available statistics on forms of crime not covered by the NCVS, the FBI’s UCR and NIBRS data systems, and other major data series maintained by other statistical agencies, (b) pursue research on what new statistics could be feasibly and usefully developed, and (c) propose such new data collections as the research suggests to be both feasible and useful. BJS should strive to function as a clearinghouse of justice-related statistical information, including reference to data not directly collected by BJS.
OCR for page 288
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
Build Capacity for Covering Hard-to-Reach Populations
Recommendation 5.14: BJS should study the measurement of emerging or hard-to-reach groups and should develop more appropriate approaches to sampling and measurement of these populations.
Continue Coordination With Office of Juvenile Justice and Delinquency Prevention on Juvenile Justice Data
Recommendation 2.2: In line with its original charge and to better document and understand the contribution of juveniles to street crime and violence, the victimization of youth, and the consequences for youth and society of their victimization and offending, BJS should develop juvenile victimization, crime, and justice statistical series suitable for describing the patterns of offending and victimization of youth, longitudinal progression of youth through the juvenile and criminal justice systems, and reentry into the community and criminal system. Taking on this responsibility would require additional resources.
Leverage Second Chance Act Responsibilities to Build Active Program on Recidivism and Reentry
Recommendation 3.14: BJS should mount a feasibility study of the flow of individuals between correctional supervision and community settings. Repeated interviews of samples of about-to-be-released prisoners that track their successes and failures in reintegrating with the community would enhance understanding of this critical policy issue.
inal justice system. Following this logic, and all other things being equal, describing crime, victimization, and the immediate consequences should be the principal focus of BJS. Second, however, the nature of a funnel also directs attention to the late stages of the process. As offenders make their way through the funnel, their numbers decrease but the consequences of justice system decisions become increasingly consequential in terms of impacts on lives and public budgets. At the far end of the criminal justice system, in the correctional area, the effects of criminal justice decisions are so extensive that obtaining adequate data on these populations takes on an importance well beyond their numbers. This principle for establishing the importance of any given activity need not correspond to the allocation of resources to that activity because some data can be collected more efficiently than others. Nonetheless, it should guide decisions as to where to put the intellectual energy of the agency if not in a one-to-one correspondence to its fiscal resources. A logical decision process may be to separately value activities by their proximity to the funnel, then assess their relative cost; the final portfolio is a function of both importance and cost.
OCR for page 289
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
Two of these goals may be combined and interpreted as providing an answer to one of the questions raised at the outset of this section: the primacy of the NCVS in BJS’s resource allocations. In terms of BJS’s function as an independent statistical agency (Goal 1), the NCVS can play a major role. The survey’s role as an independent counterpoint to the count of crimes officially reported to police and its capacity to assess public opinions of and interactions with justice officials make a vigorous NCVS a cornerstone of BJS’s reputation for objectivity and data quality. The NCVS’s role in measuring the input to, and largest part of, the justice system funnel of Goal 3 further argues for NCVS-intensive funding.
This interpretation is valid and we accept it. In our interim report, we considered a number of alternative structures for conducting the survey and the ways in which specific design features satisfy desired measurement goals. In doing so, one fundamental option that had to be considered “is not to conduct the survey at all”—to decide that the collection is, for whatever reasons, not worth its considerable expense. As we noted in that report (National Research Council, 2008b:79), and reaffirm here:
We reject that option. To take that option violates the legislative responsibilities of the Bureau of Justice Statistics. Furthermore, the panel thinks that BJS is the appropriate locus of responsibility for victimization measurement. As a federal statistical agency, it alone has the mandate for independent, objective, statistical measurement, with the transparency that can establish public trust in the information.
In our assessment, the NCVS is sufficiently core to BJS’s legally mandated duties and it’s basic function as a statistical agency that it is difficult to imagine an effective BJS without a strong and continuing NCVS.
For reasons we describe in more detail in our first report (National Research Council, 2008b), the costs of personal survey interviewing are increasing and response rates to surveys in general are declining (even though some federal surveys such as the NCVS continue to maintain very high response rates). The NCVS requires large sample sizes in order to derive information on relatively low-incidence phenomena, and so these broader forces on survey research render impossible the truly ideal situation: survey data on victimization with the same quality as the existing NCVS but gathered at a small fraction of the cost. Congress and the administration need to be aware that a high-quality victimization survey is an investment and that accuracy comes with a price.
Accordingly, we reiterate two recommendations from our interim report (National Research Council, 2008b:Recs. 3.1 and 3.2)—one on the overriding goal of the NCVS to serve as a continuous, quality indicator of criminal victimization in the United States and another on the need to maintain the NCVS with proper resources.
OCR for page 290
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
Recommendation 6.1: BJS must ensure that the nation has quality annual estimates of levels and changes in criminal victimization.
Recommendation 6.2: Congress and the administration should ensure that BJS has a budget that is adequate to field a survey that satisfies the goal in Recommendation 6.1.
We see the development of a core-supplement structure for the NCVS as one means of remedying the chronic funding difficulties of the NCVS. By seeking commitments from federal agency partners for regular and recurring topic supplements to the NCVS, with some portion of the payment for supplements also contributing to the core costs of the survey, BJS has opportunities to more aggressively position NCVS as a flexible and effective data collection vehicle. Another partial solution to the chronic funding difficulties of the NCVS is to consider alternative funding streams, one possibility of which is the Crime Victims Fund created by the Victims of Crime Act of 1984 (42 USC § 10601). The fund consists of monies paid as various federal fines, penalty assessments, and the proceeds from federal asset forfeitures (as well as gifts from private entities), and grants for victim compensation and support services are made by OJP’s Office of Victims of Crime from the fund. In 1999, the law enabling the fund was changed to authorize, in particular, U.S. Attorneys Offices and the Federal Bureau of Investigation (FBI) to “such sums as may be necessary” in order “to improve services for the benefit of crime victims in the Federal criminal justice system, and for a Victim Notification System.” Although creating similar access for BJS would require language in a legislative reauthorization, it stands to reason that some provision for a modest—but, importantly, stable and recurring—allocation from the Crime Victim Fund to defray NCVS expenses would be consonant with the intent of the fund.
Recommendation 6.3: More information about the needs of victims is essential to the compensation and assistance goals of the Office of Victims of Crime. Congress should allow additional funding for the collection and improvement of victimization data to be obtained from funds obtained through the Victims of Crime Act.
However, we also reiterate—as we described in Section 3–F—that more should also be expected from that investment in terms of timeliness of analysis, content of topic supplements, and methodological improvement, and we offer a follow-up recommendation:
Recommendation 6.4: Additional resources made available for the NCVS should be used not only to increase the reliability of annual estimates but also to supplement the survey in ways that increase our understanding of criminal victimization.
OCR for page 291
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
Although the NCVS may, fairly, enjoy some primacy in resource allocations, the other parts of the justice system funnel are also important. Accordingly, BJS should also give priority to filling in the funnel, that is, ensuring that data exist on every important decision made throughout the process. Some decisions—notably the declination decisions made by prosecutors and the revocation decisions made by parole and probation officers—are not well described by BJS data systems at this time. Increasing the coverage of decision points in the system may not happen overnight and the quality of these data could be designed to improve over time, but BJS should have a plan and should gather the information necessary for planning data collections or other means of describing all important decisions in the criminal justice system. The panel has judged that the most cost-efficient method to study the key transition points in the funnel requires new longitudinal measurements.
This simple statement of priorities also relegates to a position of lesser importance matters that are not criminal in their nature. We reiterate the point made in Section 2–D that civil justice issues are very important, representing a major share of the judicial proceedings in the nation’s courts. The definitional problems inherent in scoping out major new data collection in civil justice, as well as the problems of accurately measuring private, out-of-court settlements, are such that expanded data collection in the civil justice area is only practicable with substantial resources and commitment from Congress and the Justice Department. More work in civil justice is certainly worth doing if such resources become available, but a strict interpretation of priorities makes it a lower priority for the agency in the immediate future.
6–C.4
Facilitating Access and Improving Dissemination and Outreach
Goal 4: To ensure access to statistics and data on crime and justice by the American public, the U.S. Congress, the U.S. Department of Justice and other executive agencies, and state and local government agencies.
This goal is a direct carryover from BJS’s self-identified strategic goals, and we recognize BJS for including the notion of outreach in its original listing; we have adapted language from BLS’s mission statement to develop our wording.
BJS has made laudable strides in promoting access to its data and reports. Its website provides ready and relatively easy access to an extensive backfile of BJS reports, summary tabulations, and data collection instruments; so, too, does the OJP–sponsored National Criminal Justice Reference Service which also includes content from the National Institute of Justice and other agencies. The summary tabulations included with report releases are accessible to more casual data users while high-end users are well served by BJS’s
OCR for page 292
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
Box 6-4
Summary of Recommendations and Commentary on Strategic Goal 4: Facilitating Access and Improving Dissemination and Outreach
Create and Utilize a Professional Advisory Committee
Recommendation 5.8: BJS should establish an Advisory Group under the Federal Advisory Committee Act to provide guidance to BJS on the addition of new data collection efforts and the modification of current ones in light of needs identified by the group. Membership in the group should include, at a minimum, leaders and practitioners from each of the major subject matters covered by BJS data, as well as those with statistical and other types of academic expertise in these subject matters. The members of the group should be selected by the BJS director and the group should provide the director with at least two reports each year that contain its recommendations.
Institute Ad Hoc User and Stakeholder Workshops
Recommendation 5.7: To effectively get input on contemporaneous topics of interest, BJS should regularly convene ad hoc stakeholder workshops to suggest areas of immediate data needs.
Continue to Develop Mechanisms for More Accurate Data Compilation
Recommendation 5.10: To improve the utility and accuracy of the National Corrections Reporting Program (NCRP), BJS should work with correctional agencies to develop their own internal records to promote consistent data collections and expand coverage beyond the 41 states covered in the most recent NCRP.
Nurture an Active, Continuous Research Program
Recommendation 5.13: BJS should carefully study changes in the NCVS survey design before implementing them.
Improve Technical Skill Mix of BJS Staff
Recommendation 5.15: BJS must improve the technical skills of its staff, including mathematical statisticians, computer scientists, survey methodologists, and criminologists.
Build Ties With Congress
Recommendation 5.9: DOJ should take steps to ensure that congressional staff are aware of BJS data that could be used in developing legislation; DOJ and BJS should learn from congressional staff how their data are needed to inform/support legislation so that they can improve the utility of their current data and so that they can develop new data sets that could enhance policy development.
Consider Means to Improve Timeliness of BJS Estimates
Recommendation 5.11: BJS should evaluate each of its data programs to inquire whether more timely estimates might be obtained by (a) making discrete data collections into more continuous operations and (b) issuing preliminary estimates, to be followed by final estimates.
OCR for page 293
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
Improve Equitable Release of BJS Data for Analytical Purposes
As discussed in Section 5–B.4, and in keeping with the expected practice of wide dissemination of data, BJS should continue to work to ensure that all of the public (including external researchers) should have access to data releases at the same time, in formats that are conducive to use and analysis.
Enhance BJS Web Presence
Recommendation 5.12: BJS should articulate why some data collections are housed on external websites and describe the process by which links to external websites are allowed. BJS should articulate and justify the use of its insignia on external websites.
microdata holdings at the National Archive of Criminal Justice Data. It has also, in recent years, taken steps to add to the feedback and expert knowledge contribution that it already obtains through its state Statistical Analysis Center networks; its sponsorship of a data user conference in February 2008 was an instructive exercise.
Going forward, the challenge is for BJS to increase its public profile and its relevance in policy debates while maintaining the high quality standards it has set for itself. In the panel’s assessment, the creation of a formal advisory committee would benefit BJS as a sounding board for specific data collection revisions, a reviewer of new research priorities, and a source of intelligence on new and emerging topics in criminal justice. A formal advisory committee would usefully complement feedback from ad hoc user and stakeholder workshops on specific data needs. From the technical standpoint, an important means for BJS to increase its public relevance is to find ways to address long-standing concerns about the agency’s data products: though they are generally held in high regard, they are frequently seen as lacking timeliness and, particularly for the NCVS, subnational geographic detail. In addition to work on statistical modeling to improve the temporal and spatial resolution of estimates (a search in which BJS could partner with federal statistical agencies, most of which face similar pressures from its user constituencies), BJS should also consider a release schedule involving preliminary estimates subject to final review and possible revision.
6–D
SUMMARY
Congress has given the Bureau of Justice Statistics a comprehensive mandate to collect data and disseminate policy-relevant statistical information on a broad array of domains. However, BJS does not have the human and fiscal resources to fulfill this mandate. Recent history shows evidence of in-
OCR for page 294
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
terference from within the Department of Justice in its attempts to report objectively the information it does collect.
As a principal federal statistical agency BJS must become an independent, objective collector and reporter to the nation about the justice system. To achieve that independence, in the judgment of the panel, BJS should be organizationally separated from units of the Justice Department that administer programs; its director should be a presidential, fixed-term appointment; it should not participate in regulatory activities; its products must not be censored; it should renew the NCVS; it should enrich its ties with state justice systems to the benefit of national statistical information.
BJS should shape its statistical series with the entire justice system in mind, from the moment of a criminal act through all stages of the administrative consequences. This means a new blueprint for its products, exploiting new developments in computerized record systems, more timely supplements to existing surveys on new developments, guiding conceptual and statistical principles for its adjudication series, and revived attention to court statistics. The panel urges BJS to develop more longitudinal measurement tools, following the course of a criminal event and the actors involved in the event over time. Only with such longitudinal observation can the country learn about the outcome of various policy interventions. This longitudinal measurement will probably involve the mix of administrative data and self-report data from sample surveys, and should apply to all stages of the justice system. Such a plan will probably involve active partnerships with the FBI, the Office of Juvenile Justice and Delinquency Prevention, and the courts. It will clearly entail studies of the movement in and out of incarceration and in and out of the community. The panel believes that such a longitudinal vision requires new funding models to achieve success.
One action step to achieve BJS’s appropriate stature is to expand BJS’s role as a clearinghouse for all justice-related statistical information. To do this, BJS must develop mechanisms to be informed about the key information needs of the country. BJS needs a permanent technical advisory committee, with rotating membership of key stakeholders and statistical experts; it needs ongoing user workshops; it needs human resources to continuously examine new ways to utilize records and innovative measurement designs for its statistical series. The agency needs to add more highly skilled statistical staff so that it can design its own statistical programs instead of relying on other agencies and contractors. In short, BJS should be able to provide the technical leadership for the country’s justice statistics.
As it is currently equipped, BJS is not fulfilling its legal mandate. This is due neither to the current staff of BJS, who have proven capable and resilient over the 30 years since the agency’s creation, nor to its existing set of data collections. Nor is it due to the legal mandate itself: the tasks and expectations placed on BJS are appropriately broad and sprawling, reflecting
OCR for page 295
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
the complexity of crime and the importance of information on the administration of justice to the public good. Rather, the problem is that BJS lacks the position of independence and culture of innovation that are—with commensurate resources—necessary for the agency to more fully meet its legally defined expectations. BJS has accomplished a great deal in 30 years, but much work remains to be done to ensure the quality, credibility, and relevance of statistics on justice in the United States—to achieve the BJS the country deserves.
OCR for page 296
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics
This page intentionally left blank.
Items in cart [1]
