National Academies Press: OpenBook

Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics (2009)

Chapter: 2 Measurement in the Justice System

« Previous: 1 ntroduction
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 47
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 48
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 49
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 50
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 51
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 52
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 53
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 54
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 55
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 56
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 57
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 58
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 59
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 60
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 61
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 62
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 63
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 64
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 65
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 66
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 67
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 68
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 69
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 70
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 71
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 72
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 73
Suggested Citation:"2 Measurement in the Justice System." National Research Council. 2009. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics. Washington, DC: The National Academies Press. doi: 10.17226/12671.
×
Page 74

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

–2– Measurement in the Justice System I RONICALLY, IT IS ONE OF THE SHORTER ENTRIES in the statutory du- ties of the Bureau of Justice Statistics (BJS) that assigns the agency its most gargantuan task: Point 4 in Box 1-2 obliges BJS to collect and analyze data “concerning the operations of the criminal justice system” at all levels of government. The complication, of course, is that the “criminal justice system” is not a simple construct. Defining all the parts of a “criminal justice system” is difficult, much less organizing coherent techniques to measure aspects of said system, and different conceptual approaches lead to starkly different vantages of the sys- tem. An organizational theory perspective might focus on the interactions of government bodies within the system and on measures of throughput and ef- fectiveness (e.g., police success rates in clearing cases and apprehending sus- pects and caseload processing rates by the courts); a person-level approach from the victim’s perspectives might emphasize the availability and effec- tiveness of victim support and compensation programs whereas an offender- based approach would give higher prominence to physical and social condi- tions in correctional facilities as well as parole and prison reentry programs. Measurement in the justice system is also complicated by the highly var- ied units of measurement that obtain throughout the process. The most basic of violent interpersonal crimes involves a triad of units—the victim, the of- fender, and the incident—each of which evinces a distinct geography, history, and set of circumstances and contexts, and the study of each of which may lead to different conclusions. However, crime has many types; “victims” and “offenders” need not be individual humans (they may, for instance, be businesses or corporations) and “incidents” need not be one-time acts of 47

48 JUSTICE STATISTICS violence. Police deal with “suspects” who may or may not be the actual per- petrators of crimes; courts deal with “cases” or “defendants,” each of which involves one or more specific “charges,” and the connection between these labels and the individual “prisoners” who serve correctional sentences may be lost when authority for them transfers from the courts to the corrections system. An immediate consequence of this unit-of-measurement problem, which we discuss in this chapter and elsewhere in the report, is that measur- ing the flow of individuals through the justice system is extremely compli- cated. As a general concept, though, it is important to bear in mind that an approach to the “justice system” that focuses on tracking the experience of individual persons as they move through the system will present a different picture of justice processing than inferences drawn from one that focuses on the progression of “cases” through the set of discrete operations that either move them forward toward resolution or divert them out of the system. There is no single way, and certainly no uniquely correct way, to concep- tualize the criminal justice system. Hence, BJS’s task is to straddle a wide range of perspectives in selecting and defining its measurement programs, and to do so while ensuring collection at all (and widely varying, in them- selves) levels of government. In this chapter, we discuss the general challenge of measurement in the justice system. We begin by describing the basic model that we use as an orienting framework—the “funnel” model first developed by the President’s Commission on Law Enforcement and Administration of Justice (1967) (Sec- tion 2–A)—and discuss how BJS’s data collections roughly conform to that model (Section 2–B). Analysis of the funnel suggests some major gaps in U.S. justice statistics—and BJS’s statistical coverage in particular—which we de- scribe in Section 2–C; these include difficulty in addressing new and emerg- ing types of crime and contextual factors that apply to a wide range of crim- inal activities. 2–A THE “FUNNEL” MODEL OF FLOWS IN THE CRIMINAL JUSTICE SYSTEM Many date the emergence of criminal justice research on the public scene with the release of the 1967 report of the President’s Commission on Law Enforcement and Administration of Justice, The Challenge of Crime in a Free Society. The report and its many companion volumes summarized what was known about criminal justice and called in virtually every section for more information about how the system operated. The first chapter of the main report introduced as its organizing metaphor a flowchart presenting “a simple yet comprehensive view of the movement of cases through the criminal justice system” (President’s Com-

MEASUREMENT IN THE JUSTICE SYSTEM 49 mission on Law Enforcement and Administration of Justice, 1967:8–9). The chart was meant to emphasize that “a study of the system must begin by ex- amining it as a whole,” that “the criminal process . . . is not a hodgepodge of random actions” but rather “a continuum—an orderly progression of events” (President’s Commission on Law Enforcement and Administration of Justice, 1967:7). BJS continues to publish the chart—essentially identi- cal in content to the 1967 version—because the model remains popular and useful for studying flows in the criminal justice system; a current version of BJS’s publication of the chart is shown in Figure 2-1. Moving from “crime” through the many paths by which accused persons might eventually progress “out of system,” this chart illustrated some of the report’s main points. One was that the major institutions that make up the criminal justice system—the police, the courts, and corrections—are interde- pendent (President’s Commission on Law Enforcement and Administration of Justice, 1967:7) What each one does and how it does it has a direct effect on the work of the others. The courts must deal, and can only deal, with those whom the police arrest; the business of corrections is with those delivered to it by the courts. How successfully corrections reforms convicts determines whether they will once again become police business and influences the sentences the judges pass; police activities are subject to court scrutiny and are often determined by court decisions. And so reforming or re- organizing any part or procedure of the system changes other parts or procedures. Another was the complexity of the system, which the report contrasted to “the popular, or even the lawbook, theory of everyday criminal process” (President’s Commission on Law Enforcement and Administration of Jus- tice, 1967:7). This complexity was illustrated by the many forks in the road along which individuals might travel, some of which led to a quick exit whereas others promised to involve them for years to come. Finally, the report expressed concern about the fairness as well as the effectiveness of the system that was on view. It noted that, “throughout the system the im- portance of individual judgment and discretion, as distinguished from stated rules and procedures, has increased.” It concluded that “a consideration of the changes needed to make it more effective and fair must focus on the extent to which invisible, administrative procedures depart from visible, traditional ones” (President’s Commission on Law Enforcement and Admin- istration of Justice, 1967:10). Practically, the chart also served as a starting point for the commission to discuss what was not known then—and, in some cases, what is still not known—from data on the justice system. As the chart’s legend notes, “the differing weights of line [in the chart] indicate the relative volume of cases disposed of at various points in the system, but this is only suggestive since

50 Figure 2-1 Sequence of events in the criminal justice system SOURCES: Adapted from President’s Commission on Law Enforcement and Administration of Justice (1967:8–9) in 1997; posted at http://www.ojp. usdoj.gov/bjs/flowchart.htm. JUSTICE STATISTICS

MEASUREMENT IN THE JUSTICE SYSTEM 51 no nationwide data of this sort exists” (President’s Commission on Law En- forcement and Administration of Justice, 1967:8). Our review of BJS’s statistical programs generally follows the flow of this historic chart, for many of them are also organized around the institutions and organizations that make up the justice system. The review also reflects many of the commission’s original concerns. Like the commission’s report, we emphasize the importance of understanding the interface between the parts of the system as well as the decisions that structure the flow of individ- uals within each of its components and their eventual exit from the system. Reflecting continued concern about the fairness issues raised by the com- mission, we examine the utility of the data for assessing the distribution of the outcomes of these decisions, decisions that constitute “justice” for those who are subject to them. 2–B BJS DATA COLLECTIONS IN THE CONTEXT OF THE CRIMINAL JUSTICE SYSTEM For the panel’s benefit, BJS staff provided a listing of its varied data col- lections, indicating their approximate coverage of various steps relative to a stylized version of the crime sequence model (Figure 2-1) that more graph- ically resembles a “funnel.” The resulting diagram is shown in Figure 2-2. The diagram shows individual BJS data series and so does not explicitly men- tion BJS’s grantmaking functions, which would show up more directly in a listing of BJS programs. For instance, “Firearm Inquiry Statistics” are a basic summary of use of the National Instant Criminal Background Check System, which BJS does not directly administer but which it provides grants to local agencies to improve. The “ranges” of coverage indicated on the diagram are approximate in nature. This particular schematic underplays the potential range of the Na- tional Crime Victimization Survey (NCVS) and its supplements. Because its primary focus is on victimization incidents, it makes most sense to place the NCVS at the leftmost point on the diagram, corresponding to the occur- rence of crime. However, the NCVS includes victimization incidents that are not reported to police and hence do not start the criminal process, as suggested by the diagram; part of the NCVS’s unique value is its ability to provide information on the characteristics of these incidents that “leak” out of the system in the earliest stages. Moreover, the NCVS is flexible enough to provide information on incidents of interpersonal violence that may not formally be “crime,” and it can also speak to individuals’ experiences with other, later parts of the system, such as contacts with police that do not result in arrest or experiences with court proceedings. Likewise, the coverage range may overstate the scope of the Justice Sys-

52 Figure 2-2 Mapping of Bureau of Justice Statistics data series to sequence of events in the criminal justice system SOURCE: Adapted from chart provided by the Bureau of Justice Statistics to the panel for its first meeting, February 2, 2007. JUSTICE STATISTICS

MEASUREMENT IN THE JUSTICE SYSTEM 53 tem Expenditure and Employment Extracts, which appear to be the widest- ranging and, seemingly, most comprehensive of the data series. This measure of local governments’ reported spending on justice-related activities dates back to a special study in the late 1960s in which the Census Bureau com- piled spending totals on police, corrections, and judicial processing for some of the large jurisdictions in their annual finance and employment surveys. Based on this initial work, a supplemental mail survey of local governments specifically on justice expenditures was first developed and fielded by the Census Bureau in 1969 and repeated until 1979. Though sample size was boosted in 1979 (with the passage of the Justice System Improvement Act that created BJS in its present form), it was canceled 1 year later for bud- getary purposes. An expenditure survey has subsequently been conducted on a sporadic basis (albeit not since 1997). For regular collection of expenditure data, BJS has since adopted the original strategy for the collection: The an- nual Justice System Expenditure and Employment Extracts are derived from data collected by the Census Bureau’s Governments Division.1 The extracts are comprehensive relative to the system because they attempt to capture expenditures for the system’s major components; however, they suffer from being less substantive (comporting with the categories in the Census Bureau’s finance surveys) and less geographically detailed than might be possible with a survey-based collection (U.S. House of Representatives, Committee on the Judiciary, 1991:160–161). Tabulations from the Justice System Expenditure and Employment Extracts are currently released in electronic-only format as a set of spreadsheet files (e.g., Bureau of Justice Statistics, 2007a); the most recent report summarized fiscal year 2003 data and was released in 2006 (Hughes, 2006). 2–C GAPS IN THE COVERAGE OF CRIME AND JUSTICE STATISTICS Study and comparison of the “funnel” model of flows through the crim- inal justice system (Figure 2-1) and the illustration of BJS data series that correspond to those flows (Figure 2-2) suggest a few fundamental issues in BJS’s statistical coverage. One such issue is that the stylized BJS mapping (Figure 2-2) emphasizes a forward flow through the system from stage to stage. However, in doing so, it understates a key element that the fuller conceptual model of Figure 2-1 depicts: There are significant “leaks” in the funnel structure. These leaks, or diversions from the forward flow of the process, include matters that are of 1 Specifically, the data are derived from the Census Bureau’s Annual Government Finance Survey and Annual Survey of Public Employment. The Governments Division is part of the Census Bureau’s economic directorate, and the Census Bureau’s government programs were recently reviewed in National Research Council (2007).

54 JUSTICE STATISTICS keen interest for understanding the system as a whole: for instance, acts of violence that never enter the system because they are not reported to police, dismissals of charges in the early stages of prosecution, alternative court resolutions that prevent trials from following the usual channel, and plea agreements that circumvent parts of the process. These leaks are inherently difficult to measure because, by their nature, they may not show up in the extant data resources at any particular stage in the process. Another issue with the forward-flow sieve model is that it tends to cast the late stages of the process (e.g., corrections) as terminal or ending states. Accordingly, the model understates issues of reentry into the community from corrections, as well as recidivism. Figure 2-1 generally shows the last stage in the process as an individual being “out of system”; it does not explicitly show the trajectory that would place individuals back in custody (e.g., a “feedback loop” all the way to the start of the process and the possibility of either committing a new offense or being victimized again). Any search for gaps in BJS’s program coverage—types of crime that are not described well or at all in current collections or additional frontiers in the understanding of crime and justice on which statistical information would be valuable—will inevitably generate a considerable list. The topic area of crime and justice is sufficiently broad that is relatively easy to rattle off long lists of important and interesting topics that a better funded BJS could take on; the perennial problem is reconciling that “ideal” list of desired knowledge with realistic resource assignments. Singling out “gaps”—major or minor— in BJS’s coverage is not meant as a criticism of BJS in any way, but rather a reflection of practical realities and a suggestion of possibilities. Indeed, BJS itself goes through this exercise, recognizing gaps in its coverage. In a 2007 meeting with the executive committee of the Justice Research and Statistics Association, BJS staff articulated a “top 20” list of data needs and information gaps that it had developed as part of the process of preparing its budget requests (Sedgwick and Ramker, 2007:3): 1. Statistics on the extent and usage of private security services; 2. Expanded analysis of elderly victims of crime, including data on the prosecution of elder abuse and mistreatment; 3. Data on crimes of human trafficking; 4. Development of a statistical system to study outcomes of ex-offender employment programs, and prisoner reentry in general; 5. Standardization of the Record of Arrest and Prosecution (“rap sheet”) criminal histories maintained by law enforcement and accelerated adoption of new standards for rap sheets; 6. Information on the characteristics of “frequent fliers” who are incar- cerated in local jails on a repeated basis;

MEASUREMENT IN THE JUSTICE SYSTEM 55 7. Development of an infrastructure to study recidivism using criminal history record databases; 8. National data on the use of lethal force by police; 9. Data on the nature and extent of citizen complaints concerning behav- ior of local law enforcement; 10. Information on the sentencing of felony identity theft offenders; 11. Inventory of law enforcement “cold case” forensic units; 12. Systematic collection of data on security threats and conditions in state and local courthouses; 13. Fuller collection of contextual information on weapon usage, gang in- volvement, and drug involvement in incidents included in the National Incident-Based Reporting System (NIBRS); 14. Data on predation and exploitation of children via the Internet; 15. Data on juveniles processed as felony defendants in (adult) criminal courts; 16. Creation of an establishment survey of U.S. businesses on computer security; 17. Expanding participation by law enforcement agencies in common information-sharing databases; 18. Fuller data on law enforcement use of excessive (but nonlethal) force; 19. Fuller data on electronic crime and identify theft; and 20. Articulation of particular challenges to law enforcement (including staffing and resources) in a post-9/11, heightened security environ- ment. In the balance of this section, we discuss four major “gaps” in BJS’s data collection portfolio—things that are conspicuously absent in Figure 2-2. Two of these gaps are related because, as noted earlier, the “funnel” model of the justice system is perhaps too easily equated with the processing of violent crime; other classes of crime, such as white-collar offenses, and civil judicial proceedings are not well captured by current systems. A third gap is that contextual factors associated with crime are inherently difficult to describe— and even characterize consistently—at all steps in the criminal justice system; we describe the relation of drugs and crime as an example. A fourth major gap that is plainly apparent from a comparison of Figure 2-2 with the general framework of Figure 2-1 is the processing of juvenile offenders and victims. In discussing this topic, we also consider the relationship between BJS and one of its sister data-gathering units in the Department of Justice, the Office of Juvenile Justice and Delinquency Prevention (OJJDP). Our assessment of

56 JUSTICE STATISTICS what should be done concerning each of these gaps is similar, and so we close the section and chapter with a common discussion. We defer our commentary on a fifth—and possibly most severe—“gap” or flaw in BJS’s existing coverage until the next chapter; this gap is the concept that the justice system is more than the sum of its parts, and that longitudinal flows throughout the system as a whole are not well measured at present. However, this topic is best considered after we have discussed more of the content of BJS’s portfolio in Chapter 3, and so we defer the discussion of longitudinal structures until Section 3–F.1. 2–C.1 White-Collar Crime The degree to which the measurement of “crime” is primarily focused on violent or street crime and major property crimes such as theft and arson is due, at least in part, to the long-standing definition of “Type I crimes” in the Federal Bureau of Investigation’s (FBI’s) Uniform Crime Reporting (UCR) program (which we describe in more detail in Section 4–C.1). The major definitions and conceptions of crime from the UCR were carried over to BJS’s NCVS, and other programs. Yet both the UCR and the NCVS— the nation’s two principal indicators of crime—have been critiqued for be- ing slow to catch up to new crime types. A report of the FBI’s Criminal Justice Information Services Division (Barnett, 2000:2) conceded that “it is well documented that the major limitation of the [UCR] Summary Reporting System is its failure to keep up with the changing face of crime and crimi- nal activity.” Likewise, our panel’s interim report discussed the challenges involved in achieving the NCVS’s full flexibility in studying new types of victimization (National Research Council, 2008b:Sec. 3–C.1). A focus on certain forms of violent and property crime does not account for all important types of crime, or crime types that emerge with the in- troduction and maturation of new technologies. A particular gap is in the measurement of many forms of what could loosely be labeled as white-collar crime. The term “white-collar crime” has been in currency since sociologist and criminologist Edwin Sutherland introduced it in 1939. Some interpret the term narrowly (e.g., Sutherland’s early focus on crimes committed by a person of high responsibility in the pursuit of his or her occupation) whereas others interpret it more broadly. For instance, the Federal Bureau of Inves- tigation (1989:3) has defined white-collar crime as: those illegal acts which are characterized by deceit, concealment, or violation of trust and which are not dependent upon the application or threat of physical force or violence. Individuals and organizations commit these acts to obtain money, property, or services; to avoid the payment or loss of money or services; or to secure personal or business advantage.

MEASUREMENT IN THE JUSTICE SYSTEM 57 We use the term more broadly to refer to crimes such as corporate fraud, health care fraud, financial institution fraud, money laundering, government fraud, consumer fraud, public corruption, and Internet crimes. Recent survey evidence suggests that the public views white-collar crime at least as seriously as traditional forms of crime, and offenses committed by organizations or by higher-status persons as more serious than those committed by individuals or lower-status persons. With sponsorship from the Bureau of Justice Assistance, the National White Collar Crime Center (NW3C) conducted a “National Public Survey on White Collar Crime” in 2005. Based on the survey, the NW3C concluded that the general public views white-collar crime as seriously as traditional crime types. In addition, the survey also yielded the finding that (Kane and Wall, 2006:3): Crimes involving physical harm are seen as significantly more serious than those crimes that incur a monetary loss only; organizational of- fenses are viewed more harshly than those committed by individual offenders; and crimes committed by high-status offenders (those in a position of trust) are seen as more severe than those crimes committed by [non-high-status] persons. There is also evidence that white-collar crime is a concern to law enforce- ment. Amidst its recent major change in strategic priorities to the deterrence of terrorism, the Federal Bureau of Investigation (2004a) prominently in- cluded a white-collar crime plank in its strategic goals for 2004–2009: IIH.1 Reduce levels of corporate fraud by targeting those groups or in- dividuals engaged in major corporate fraud schemes that signifi- cantly impact the investing public and financial markets. IIH.2 Reduce the incidence of large scale health care frauds, involving both government-sponsored and private insurer programs. IIH.3 Reduce fraud perpetrated by criminal enterprises targeting finan- cial institutions. IIH.4 Disrupt and dismantle the most significant money laundering in- stitutions and facilities. IIH.5 Reduce the impact of telemarketing, insurance, and investment fraud on businesses and individuals, particularly schemes origi- nating from outside the United States. IIH.6 Address those investigative matters which represent the most significant economic losses within federally-funded procure- ment, contract, and entitlement programs, environmental crimes, bankruptcy fraud, and anti-trust offenses. In its strategic plan, the FBI further argues that “the ability of the U.S. Government and industry to function effectively [is] threatened by complex frauds” and that continuance of its “successful efforts in the white collar crime arena” is important to “ensure the integrity of government expendi- tures of taxpayer funds [and] protect individuals and businesses from catas- trophic economic loss.”

58 JUSTICE STATISTICS However, the FBI strategic plan leaves it unclear how the FBI and the Justice Department plan to assess its success toward achieving these goals. Indeed, it is unclear how such an assessment plan could be developed be- cause of the lack of data. Responding to a questionnaire for a United Na- tions intergovernmental expert group on fraud, the U.S. Department of Jus- tice, Criminal Division, Fraud Section (2006:6–7) had to acknowledge that “there is no single government agency or private-sector entity that compiles statistical data on the principal types of fraud that occur within or affect the United States.” The Fraud Section followed that disclosure by outlin- ing a lengthy list of what it described as “some of the more frequently re- ported types of fraud” committed in the United States—including advance- fee fraud,2 telemarketing fraud, and identity document (including passport and visa) fraud—though it could provide no quantitative evidence on the actual frequency of these crimes. When BJS was founded in 1979, white-collar crime was originally in- tended to be a part of the agency’s portfolio; introductory text declared that the purpose of the law creating the agency was to “provide for and encourage the collection and analysis of statistical information concerning crime (including white-collar crime and public corruption)” (93 Stat. 1176). However, this parenthetical was stricken from law in 1984 (98 Stat. 2079). Though not part of the agency’s formal charge, BJS is not completely silent on issues of white-collar crime. As we discuss in Section 3–A, BJS’s supple- ment to the NCVS on identity theft has developed into a regular feature of the survey; the NCVS cybercrime supplement also touched on experiences with Internet fraud. However, the information that is available on this class of crime is not well organized or displayed on the BJS website. There is no front-page link to white-collar crime as a topic (the BJS home page is dis- played in Figure 5-5). Using its “BJS only” search engine box, a query for white-collar crime provides only a few links to old reports available only in paper form and stray links to the Survey on Prosecutors in State Courts and the Compendium of Federal Justice Statistics. Broadening the search to in- clude “OJP and NCJRS”—BJS’s parent Office of Justice Programs (OJP) and the OJP–sponsored National Criminal Justice Reference Service (NCJRS), respectively, produces some more useful links. Chief among these is a des- ignated “topic page” for white-collar crime on NCJRS, but the presentation on that page (http://www.ncjrs.gov/App/Topics/Topic.aspx?topicid=73, ac- cessed 11/6/2008) lacks hard information, with a link to BJS’s report on the NCVS identity theft data (Baum, 2007) being the only one that suggests ob- vious empirical data. Other links on the page are to National Institute of 2 “Advance-fee fraud [can] encompass any type of fraud scheme in which victims are in- duced to pay money to criminals for nonexistent “taxes,” “fees,” or “customs duties” before the criminals are expected to provide whatever goods or services (e.g., offshore tax shelters) they have promised to victims.

MEASUREMENT IN THE JUSTICE SYSTEM 59 Justice–sponsored research, including the connections between white-collar crime and terrorism and a legal analysis of the global scope of intellectual property law. The NCJRS page does provide a link to the nonprofit, private-sector NW3C, which is supported by funds from BJS’s sister organization in OJP , the Bureau of Justice Assistance. The NW3C specializes in white-collar crime law enforcement training and houses an academic research group; in a joint venture with the FBI, the NW3C operates the Internet Crime Com- plaint Center. Though the NW3C has fielded surveys on public perceptions of white-collar crime, it is not clear whether any kind of formal arrangement exists (or has been broached) between BJS and NW3C, such as BJS maintains with the National Center for State Courts and the Urban Institute’s Federal Justice Statistics Resource Center. In fairness to BJS, there are at least three major reasons that partially explain the dearth of information on white-collar crime: • The definitional problems involved in even conceptualizing a data col- lection program in white-collar crime are immense, and are more than a small agency with limited resources—already grappling with massive and ill-defined data collection areas such as adjudication—can reason- ably take on. As noted above, white-collar crime can be defined both narrowly and expansively, and selecting a set of activities to study is an initial and formidable hurdle. • An even more significant conceptual hurdle is that opening the door to studies of white-collar crime would involve a major shift in focus and style for BJS: it would require a fuller examination of the con- cept of businesses and corporations as actors in the justice system, as the victims or perpetrators of crime. BJS’s forays in this area, using businesses as the unit of analysis, have been relatively rare. The Na- tional Crime Survey program (now the NCVS) originally included a commercial victimization component, but that part of the survey was short-lived. BJS’s work with the Federal Justice Statistics Resource Center uses “defendant-cases” as the unit of analysis as it tracks flows through the federal court processing system, and this does permit the notion of a business as a defendant. And, most recently, BJS has fielded a survey of businesses on their experiences with cybercrime, on a pi- lot basis in 2001 (Rantala, 2004) and as a full-scale survey of about 8,000 establishments in 2005 (Rantala, 2008). But—those exceptions aside—BJS has tended to use individual persons or agencies (e.g., cor- rections departments or law enforcement agencies) as the unit of anal- ysis. • By its nature, fuller collection of information on white-collar crime would necessitate the collection of financial and monetary data, in

60 JUSTICE STATISTICS terms both of estimated losses and of award amounts. Accurate mea- surement (and disclosure) of financial data is a long-standing challenge for statistical agencies. That said, we think it fair to say that BJS’s data collections (and the information available on its website) have historically been dominated by violent crime and “street crime.” This produces—or at least contributes to— a partial and perhaps misleading image of crime and criminals in America. Knowledge about the nature of corporate and white-collar crime—its levels and trends, its handling in the court system, and its impact on criminal justice system operations—would be as useful and informative as data on “street crime.” If BJS is to be positioned as the primary data-gathering agency for crime and criminal justice in the United States, it would be most sensible for BJS to take the lead to organize the available statistics on white-collar crime and other new forms of crime, as we discuss in Section 2–D. 2–C.2 Civil Justice A second major gap in BJS’s overall portfolio is similar to white-collar crime in that it arises from the principal focus on violent or street crime, and on crime against the person. The construct described by both Fig- ures 2-1 and 2-2 is commonly referred to as the “justice system” or the “criminal justice system,” and the degree to which this nomenclature is used interchangeably severely underplays the scope of civil justice matters. Civil proceedings account for a major portion of the activity of the nation’s court systems—ranging from prosecution of nonviolent crimes to property dis- putes to divorce and custody arrangements—yet are covered by only one basic data collection in BJS’s portfolio. On one level, the question of whether BJS belongs in the business of studying civil justice, as opposed to criminal justice, is clearly answered: collection and analysis of information on “civil disputes” and “civil justice” are explicitly mentioned in points 3, 6, and 7 of BJS’s list of legal duties (Box 1-2). On the other hand, BJS is also bound by a clause in OJP’s enabling legislation that exemplifies the tension between “civil” and “criminal” justice (42 USC § 3789n): Authority of any entity established under this chapter shall extend to civil justice matters only to the extent that such civil justice matters bear directly and substantially upon criminal justice matters or are inextrica- bly intertwined with criminal justice matters. The Civil Justice Survey of State Courts, administered every 4–5 years since 1992, has most recently been conducted by the National Center for State Courts (NCSC) on BJS’s behalf. The sample is meant to be representa- tive of large counties but is constructed in a somewhat unusual manner; in 2001, the survey represented a sample of civil bench and jury trials in “46

MEASUREMENT IN THE JUSTICE SYSTEM 61 jurisdictions chosen to represent the 75 most populous counties,” where 75 was chosen “based on cost and practicality” (Bureau of Justice Statistics, 2004b:1). Generally, these jurisdictions are counties, chosen from the 75 using strata defined by their level of civil dispositions recorded in 1990 and their population (Bureau of Justice Statistics, 2004b:3); however, only part of the caseload in Los Angeles County (the central district of the Los Angeles County Superior Court) is included in the sample, whereas some of the “ju- dicial districts” sampled in New England may span multiple counties. The number of counties actually included in the sample has been stable over the years, at 46, 45, and 46 in 1992, 1996, and 2001, respectively. For sampled jurisdictions, the NCSC reviewed tort, contract, and real property rights cases disposed of by trial during calendar year 2001; either NCSC or court staff coded details from cases onto a standardized form. For general civil cases, NCSC coded as many cases as possible but used sampling “based on ‘take rates’ generated by WESTAT” in jurisdictions where the number of such general trials became unworkable. However, “every medical malpractice or product liability case was included to over sample these case types” (Bureau of Justice Statistics, 2004b:3). Ultimately, the 2001 survey included “data on 6,215 civil jury trial cases, 1,958 civil bench cases, and 138 other civil trial cases that met the study criteria” (Bureau of Justice Statistics, 2004b:4). It is very much to BJS’s credit that, by sponsoring the Civil Justice Survey of State Courts, it takes a first step toward measuring and monitoring a ma- jor part of the nation’s judicial workload. BJS staff also deserve credit for, arguably, making the civil justice survey one of the agency’s most scrutinized and reported data series; BJS has published numerous analyses characteriz- ing verdicts from the data series (Cohen, 2004, 2005a), the amount of trial awards (Cohen, 2005b), and the nature of appeals (Cohen, 2006). BJS has also worked with state partners to collect and analyze related data on spe- cific case types, distinct from the civil justice survey; for instance, Cohen and Hughes (2007) analyze data from closed medical malpractice insurance claims in seven states. However, BJS’s current approach to measurement of civil justice suffers from the key limitation that it is based on only cases that end in a bench or jury trial and not other civil dispositions including settlements, summary judgments, and default judgments. As Cohen and Smith (2004:80) summa- rize, disposition through trial—whether a jury trial or a bench trial presided over only by a judge—“represent[s] the pinnacle event in the civil justice pro- cess,” yet “both jury and bench trials are relatively rare.” Civil lawsuits result in trial only “when at least one party refuses to settle” pretrial, out of court. These settlements constitute “the vast majority” of activity in civil proceed- ings; by their nature, they are also generally private and hence difficult to measure accurately or systematically. To be sure, the fact that case-based

62 JUSTICE STATISTICS measurement systems miss the major activity of settlement does not mean that they lack value: documentation of trial outcomes—and the “perceived trial outcomes and potential award amounts” that are derived from analysis of completed cases—influences the propensity of parties to pursue or settle civil litigation. Negotiated pretrial, out-of-court, settlements are part of a broader class of alternative dispute resolution (ADR) techniques. In adjudication gener- ally, but particularly in civil matters, the nature and extent to which ADR techniques are applied are important but, to date, largely unanswered ques- tions. ADRs include referral, by mutual consent of the parties, to an in- dependent facilitator, mediator, or arbitrator (the difference between the specific techniques being the formality of the proceedings—an arbitrator’s decision is imposed on the parties while a mediator facilitates agreement on a resolution but does not directly impose rulings). As a measure of the grow- ing acceptance of ADR techniques, several state court systems maintain spe- cific offices to monitor and suggest ADR avenues; as of 2004, 15 state court administrative offices completely fund and staff ADR offices and another 16 states partially fund such offices (Rottman and Strickland, 2006:Table 21). However, aside from general perceptions that ADR techniques are being increasingly used to divert cases from entering the court system, hard sta- tistical information on ADRs is scant, including the types of cases to which they are applied, the specific techniques used, the nature of the agreements, and the number (and success or failure of) appeals of ADR settlements to the courts. In an Internet “frequently asked questions” page, the NCSC notes that “[ADR] programs and rules vary widely from state to state, and even from court to court within a single state. As a result, national data are nearly impossible to find” (http://www.ncsconline.org/WC/CourTopics/ FAQs.asp?topic=ADRMed). The Federal Mediation and Conciliation Ser- vice, an independent federal government agency, provides summary statisti- cal information in its annual reports; however, the agency’s primary focus is on resolving disputes between labor unions and management, which is but one part of the larger phenomenon. It should be noted that BJS has produced reports on one particular subset of civil justice cases, making use of data other than the Civil Justice Survey of State Courts. Using the Civil Master File of the Administrative Office of the U.S. Courts—which in turn combines case filing data from U.S. federal district courts—BJS has regularly generated tabulations of civil rights com- plaints, including disposition (jury trial, bench trial, or directed verdict) and award amount. Kyckelhahn and Cohen (2008) describe trends in civil rights filings between 1990 and 2006.

MEASUREMENT IN THE JUSTICE SYSTEM 63 2–C.3 Juvenile Justice System Arguably, the most striking difference between the justice system “fun- nel” model (Figure 2-1) and BJS’s statistical coverage of that model (Fig- ure 2-2) is that the former contains a parallel track that has developed over the past several decades. The separate stream of processing of juveniles shown in Figure 2-1, including the operation of separate courts and cor- rectional facilities for juveniles, is essentially absent from the BJS coverage funnel.3 It is true that the two tracks—and the experiences of juveniles and adults in crime and justice—are not as entirely separate as the stylized fun- nel model suggests. That is, some juveniles are tried as adults in criminal proceedings, some are confined in adult correctional facilities or transition into the adult justice system, and—of course—both juveniles and adults may commit or be victims of violence. Hence, it is possible for juveniles to show up in BJS’s data collections that are principally intended to cover the adult population; for instance, DeFrances and Strom (1997) summarize informa- tion from the National Survey of Prosecutors on juveniles processed in state courts, Strom (2000) studies characteristics of inmates of state prisons under age 18, and Strom et al. (1998) and Rainville and Smith (2003) describe re- sults from the State Court Processing Statistics project on juveniles charged with felonies in adult courts. Further, in at least one instance, BJS was explic- itly mandated by Congress to conduct a study within juvenile correctional facilities: the data collections requested by the Prison Rape Elimination Act of 2003 (see Section 5–A.1). BJS has also studied decade-long trends (1993– 2003) in juvenile victimization and offending, utlizing both NCVS and UCR (Supplementary Homicide Reports) data (Baum, 2005). However, with those exceptions—and despite BJS’s legal list of du- ties (Box 1-2) being replete with references to data gathering on juvenile delinquency—it is one of BJS’s sister agencies in OJP that has assumed prin- cipal responsibility for collection and organization of information on juve- nile justice. Under its establishing legislation, OJJDP has the authority to (42 USC § 5661(b)(2)): undertake statistical work in juvenile justice matters, for the purpose of providing for the collection, analysis, and dissemination of statisti- cal data and information relating to juvenile delinquency and serious crimes committed by juveniles, to the juvenile justice system, to juvenile violence, and to other purposes consistent with [OJJDP’s charter]. A separate provision of law obligates the directors of both BJS and the Na- tional Institute of Justice to “work closely with the Administrator of the 3 In this section, we make a simplification and follow a common practice by defining a juvenile as someone under age 18; note, however, that 18 is not the legal age of maturity for various purposes (including treatment in the adult justice system rather than the juvenile justice system) in all states.

64 JUSTICE STATISTICS [OJJDP] in developing and implementing programs in the juvenile justice and delinquency field” (42 USC § 3789i) In comparing OJJDP’s statistical collections on juveniles with BJS’s series for adults, it is important to keep in mind that the two agencies have quite different missions. BJS’s duties under the law (Box 1-2) are primarily ori- ented toward data collection, with some provisions for assistance to state and local authorities. By comparison, the stated purposes in OJJDP’s enabling legislation (42 USC § 5602) are exclusively concerned with programmatic support for state and local assistance: The purposes of this section of [this section of law] are— (1) to support State and local programs that prevent juvenile involve- ment in delinquent behavior; (2) to assist State and local governments in promoting public safety by encouraging accountability for acts of juvenile delinquency; and (3) to assist State and local governments in addressing juvenile crime through the provision of technical assistance, research, training, evaluation, and the dissemination of information on effective pro- grams for combating juvenile delinquency. The agency’s formal mission statement is likewise geared toward influencing policy at the state and local levels: OJJDP “provides national leadership, coordination, and resources to prevent and respond to juvenile delinquency and victimization” and “supports states and communities in their efforts to develop and implement effective and coordinated prevention and interven- tion programs” (http://ojjdp.ncjrs.org/about/missionstatement.html). That said, in carrying out its mission, OJJDP develops and sponsors a series of data collection programs—in some cases developing operations parallel to BJS work for adult facilities and processes, and in others mak- ing use of BJS data. Its efforts in data gathering and coordination are routinely made available in an online “Statistical Briefing Book” (http: //ojjdp.ncjrs.gov/ojstatbb/index.html) that is maintained by the Pittsburgh- based National Center for Juvenile Justice (NCJJ) through a grant from OJJDP OJJDP also regularly produces a “National Report” in its Juvenile . Offenders and Victims series (Snyder and Sickmund, 2006). Regarding original data, OJJDP sponsors the collection of primary data in three areas: (1) juvenile facilities and juveniles in residential placement; (2) juvenile court statistics; and, to a more limited extent (3) juvenile vic- timization and offending. On the first of these, corrections and residential placement, OJJDP has sponsored a person-based Census of Juveniles in Res- idential Placement (CJRP) in odd-numbered years and a facility-based Juve- nile Residential Facility Census (JRFC) in even-numbered years since the late 1990s.4 In their structure, OJJDP’s collections mirror the strategy used by 4 Typically, the reference date for the CJRP is in October of an odd-numbered year; “how-

MEASUREMENT IN THE JUSTICE SYSTEM 65 BJS in its series on adult prisons and jails, alternating between inmate-based and facility-based data; they also bear similarities because OJJDP employs the Census Bureau as its data collection agent as is true of the adult series. However, the OJJDP person-based series differ from their adult counterparts in that they rely entirely on facility records and reporting rather than direct personal interviewing. Collectively, the CJRP and JRFC provide informa- tion on facilities (type, physical layout, counts, use of locked doors/gates) and individual juveniles held in residential facilities (demographic character- istics, placement authority, most serious offense, adjudication status, security status, etc.). Because they use juvenile-specific facilities as their frame, the OJJDP collections do not capture those juveniles held in adult prisons or jails; mental health and drug treatment facilities are also not a part of the CJRP or JRFC frames, and so juveniles housed there are not determined either. Although the OJJDP Statistical Briefing Book provides a tool for gen- erating national and state summaries from the CJRP 5 more detailed data , from the CJRP are available to researchers only on a case-by-case basis. Through NCJJ, OJJDP makes available juvenile court statistics with in- formation on the activities of juvenile courts in the United States and the cases disposed of in these courts. Similar to the FBI’s UCR program, data in this archive seem to be based upon voluntary submissions by juvenile courts. Thus, the number of participating courts varies each year; in addition, some courts provide detailed information for each case whereas others provide only aggregate counts. These data are used to provide national portraits of juvenile offenders and juvenile court activity, including information on court caseloads, variation in delinquency cases by demographic characteris- tics of youth involved, detention, disposition of cases, and the flow of cases through the juvenile justice system. Status offense cases are also considered, but to a lesser degree, and juveniles waived to adult courts are not followed beyond the waiver decision. Finally, with regard to juvenile victimization and offending, OJJDP does not routinely sponsor supplements to BJS’s NCVS. However, it sponsored a module on crime, delinquency, and arrest for inclusion in the National ever, a set of unforseen circumstances prevented the 2005 mailout from taking place in Octo- ber of that year.” The mailout was pushed to February 2006, hence an apparent deviation in OJJDP’s annual collection strategy (http://www.ojjdp.ncjrs.gov/ojstatbb/cjrp/asp/methods.asp). The CJRP and JRFC continue and extend an earlier data collection fielded by the Justice De- partment since the early 1970s: the Census of Public and Private Juvenile Detention, Correc- tional, and Shelter Facilities, better known as the Children in Custody census. The innovation in creating the two newer, separate series was the addition of individual-level characteristics in the CJRP rather than summary-level counts. 5 As part of its strategy to protect confidentiality and prevent individual juveniles from being identified from such tabulations, “OJJDP has adopted a policy that requires all published table cells be rounded to the nearest multiple of three” (http://www.ojjdp.ncjrs.gov/ojstatbb/cjrp/asp/ methods.asp).

66 JUSTICE STATISTICS Longitudinal Survey of Youth 1997 (NLSY97), conducted by the Bureau of Labor Statistics.6 The NLSY97 is based on a national sample of youth who were 12–16 years old as of December 31, 1996; they are interviewed annu- ally, with the tenth round in 2006 reaching 7,559 of the original respon- dents (http://www.nlsinfo.org/nlsy97/nlsdocs/nlsy97/97sample/introsample. html). Through the OJJDP module, and related questions on general risk behaviors among youth, the NLSY97 includes questions on substance use, delinquency and deviance (e.g., status offenses, gang membership, arrests, property offenses, carrying guns), as well as incidents of criminal victimiza- tion. In addition to its own sponsored data sets, OJJDP’s Statistical Briefing Book compiles information from a wide variety of agencies and organiza- tions to permit assessments of juvenile victimization, offending, and law en- forcement experiences. Specifically, the briefing book includes results from, among others: • The NCVS; • The FBI’s UCR and, for participating jurisdictions, the National Incident-Based Reporting System; • The Department of Health and Human Services’ National Child Abuse and Neglect Data System Child File (NCANDS); • The National Institute of Drug Abuse’s Monitoring the Future Survey; and • The Centers for Disease Control and Prevention’s National Youth Risk Behavior Survey (NYRBS). These other data sources provide more richness in detail for particular inci- dent types than the NCVS can provide alone. For example, the NCANDS allows assessments of victim, caretaker, and perpetrator characteristics, and responses to abused or neglected children, and so is useful for studying spe- cific kinds of victimization of youth. The NYRBS is a school-based sample (9th to 12th graders) that allows for additional information on victimization and participation in additional types of crimes and errant behaviors such as fights, suicide attempts, and alcohol and drug use. Considering both OJJDP and BJS’s portfolios, there remain important gaps in the coverage of youths’ activities. A major one is that most of the available data collections—particularly the NCVS and the high school–based NYRBS—provide reasonable coverage of adolescents but not the complete juvenile population. Through its focus on caretakers and perpetrators, the NCANDS is unusual in its potential coverage of all ages. Further, whatever the age range covered by the data series, the data are typically aggregated 6 See http://www.nlsinfo.org/nlsy97/nlsdocs/nlsy97/97sample/introsample.html on the credit for OJJDP’s sponsorship of the questions.

MEASUREMENT IN THE JUSTICE SYSTEM 67 and reported for broad age categories (e.g., under age 12, 12–15, over 15, or simply under 18). Thus, it is not always possible to assess activities and situations for young people across the youthful life course. Importantly, too, some youthful populations are simply not considered. For example, OJJDP reports do not typically consider youth in adult courts or custodial facilities, beyond documenting who is waived. BJS gives attention to this group, but these data are not linked with the records from juvenile courts and facilities. Consequently, meaningful assessments of which types of youth are handled in the juvenile justice versus the criminal justice system are not readily available. Nor is there much information on how youth fare who transition out of the juvenile court to the adult courts. Ideally, record and data-reporting systems regarding juveniles under age 18 and adults over 18 would not be quite so hard and fast because the division at 18 does not represent hard and fast differences in maturation of young people. 2–C.4 Drugs and Crime Amidst growing expenditures in the federal government’s “war on drugs” policies, reauthorization legislation enacted in 1988 added three spe- cific clauses on data collection on drugs and crime to BJS’s statutory man- dates, as shown in items 16–18 in Box 1-2. The new clauses referred to the “drug crisis” and the “overall national anti-drug strategy,” and tasked BJS with establishing a “a national clearinghouse for the gathering of data generated by Federal, State, and local criminal justice agencies on their drug enforcement activities,” particularly data that could be used to demonstrate the efficacy of programs and intervention efforts. By this language, Congress and the administration—not for the first time, and not for the last—assigned BJS a task that was difficult under the best of circumstances and virtually impossible given tight fiscal resources. Contextual factors associated with crime pose particular difficulties for measurement. They can be difficult to define in general and can be particu- larly difficult to define consistently across multiple data sources or through- out the different steps in justice system processing. In addition, different data collection types—for instance, personal survey interviewing versus cod- ing from written police reports—may be especially strong at capturing some factors but weak at others. The kinds of contextual factors we refer to here include what Cook (1991) has described as “the technology of personal violence”—the use of weaponry, particularly firearms, in crime—which can affect the probability of success of the crime, the consequences to the victim, the responses of law enforcement, and the implications for punishment in the court system. The geography of crime—more than just latitude and lon- gitude, including social and physical conditions and community resources in an area—is another crucial and challenging contextual factor that has

68 JUSTICE STATISTICS grown into a particularly vibrant area of criminological research in recent decades. In this section, we focus on another important crosscutting contex- tual factor—the interplay between drugs and crime—as an example princi- pally because of the explicit references in BJS’s enabling legislation. Drug crimes—offenses involving the possession and sale of controlled substances—are a major area of criminal activity that resists an easy fit with the conceptual model of the crime funnel. Drug offenses account for about 30 percent of admissions to state prisons, but arrests for drug offenses tend not to follow the initial sequence of events in the funnel where a crime (or victimization) leads to a report or complaint to police and, in turn, to an arrest. Rather, the pattern of drug-specific arrests is more closely linked to targeted enforcement efforts which vary greatly across jurisdictions. In addition, within the court processing steps of the funnel, drug cases resist simple categorization because laws regulating drugs vary across states and within states, and change over time. Thus possession of a small quantity of drugs, for example, may be only a minor violation in some jurisdictions, but a felony in others. Over and above the difficulties involved in mapping drugs and drug crime to the funnel framework, drug-related activities are generally difficult to measure. The sensitive nature of inquiries on drug use make it a behavior that can be challenging to measure accurately through self-report techniques such as survey interviewing. Those data resources that do cover aspects of drug use and drug-related criminal and violent behavior include both probability surveys and records- based series, and each has unique strengths and weaknesses. A partial list of these resources—past and present, conveying the range of data collections that have touched on drug-related issues—includes the following: • The National Survey on Drug Use and Health (NSDUH; formerly the National Household Survey on Drug Abuse) is sponsored by the Sub- stance Abuse and Mental Health Services Administration (SAMHSA). It has been in operation since 1971 and has utilized RTI International, Inc., as its data collection agent since 1988; it is perhaps the main source of statistical information on the use of illegal drugs by the U.S. population. The Monitoring the Future Survey has been conducted by the Institute for Social Research at the University of Michigan since 1975, asking secondary school students about drug use as well as other risk behaviors. Both of these surveys share a public health orientation, asking about drug use and dependency, and offer relatively little infor- mation to distinguish criminal drug activities. • The Arrestee Drug Abuse Monitoring (ADAM) program (formerly the Drug Use Forecasting [DUF] program) was a data collection spon- sored by the National Institute of Justice that produced data between 1987 and 2003. In select, participating sites, new arrestees were in-

MEASUREMENT IN THE JUSTICE SYSTEM 69 terviewed within 48 hours and asked a battery of questions on ar- rest history, drug use patterns, drug acquisition, and prior participa- tion in treatment programs. Survey results were combined with—and could be compared against—the results of urinalysis to detect the pres- ence of 10 drugs (but focusing in particular on cocaine, marijuana, methamphetamine, opiates, and phencyclidine [PCP]). A wide-ranging redesign in 1999 gave ADAM a sounder basis in probability sampling, started a “calendaring” routine in the questionnaire to cue arrestees to document drug use patterns over longer period of times, and posi- tioned ADAM to have relatively easy “crosswalk” connections for link- age to other data collections such as NSDUH (National Institute of Jus- tice, 2003:4, 13). DUF and ADAM data were subject to criticism over their representativeness and because they could function only as an indirect indicator of drug market activity (Caulkins, 2000:397–398); ultimately, the major flaw of the program is that it became unsustain- able in light of constrained budgetary resources. Data collection was suspended in 2004 and has not since been reactivated. • Like the ADAM program, the Drug Abuse Warning Network (DAWN) avoids the use of self-reporting in measuring drug use. Instead, this SAMHSA-funded surveillance system collects data from hospital emer- gency departments (on drug-related visits) and medical examiner of- fices (on drug-related deaths). DAWN suffered from well-documented problems and key limitations (e.g., Caulkins et al., 1995), prompting a major redesign effort between 1997 and 2003 (Substance Abuse and Mental Health Services Administration, 2002). As of November 2008, links to publications based on “New DAWN” data on the program’s website (http://dawninfo.samhsa.gov), from 2006 onward, all include a “caution” note that “SAMHSA is currently reviewing the estimates in this report and expects to publish revised estimates at a future date,” suggesting potential instability in the redesigned program’s estimation routines. • The Drug Enforcement Administration’s (DEA) System to Retrieve In- formation from Drug Evidence (STRIDE) compiles information from drug sales to undercover federal agents. By their nature, STRIDE data are uniquely positioned to provide information on the prices paid for drugs in those undercover transactions and the quality (purity) of the purchased drugs. However, the reliability of these data for economic and policy analyses—for instance, how closely the prices paid in the transactions logged by undercover federal agents track with prices in the broader illegal drug market, and the degree to which they represent federal (DEA) interdiction priorities rather than local-level activities— was ruled to be inadequate by the National Research Council (2001a).

70 JUSTICE STATISTICS That previous National Research Council (2001a) panel reviewed the extant data sources on drugs, drug markets, and the connection between drugs and crimes, and provides fuller descriptions of these and other data programs. In policy analysis, public conversation, and research, measures of drug crime tend to be proxied by measures of enforcement, though the empiri- cal relationship between drug crime and drug arrests is poorly understood. Drug arrest rates grew substantially through the 1980s, for example, while self-reported drug use among high school seniors was falling. Still, patterns such as these have multiple interpretations. Enforcement efforts may have been moving in the opposite direction of trends in use. Drug enforcement may have been reducing drug use. Increased enforcement may also have reduced survey respondents’ propensity to report drug use. The summary of the state of quantitative knowledge of drugs and crime by Caulkins (2000:394, 395) remains apt: In the drug policy arena we have an abundance of numbers, but the glass of insight is at best half full. . . . We know quite a bit about drug offenders within the criminal justice system but much less about their activities on the street. We know quite a bit about how many drug users there are but little about why there are so many. In contrast, we understand why people sell drugs but know little about how many upper level dealers there are, let alone how they operate. . . . More generally, existing data systems are reasonably adequate for describing patterns and trends but generally are incapable of explaining them, in part because opportunistic instead of random samples and the absence of control groups makes it difficult to tease out causal relationships. Though BJS refers to some of the data sources listed above on its web- site, it generally attempts no analytical work based on those data. “Drugs and crime” is a top-level link on the BJS home page (http://www.ojp.usdoj. gov/bjs/drugs.htm), the principal link on which is to the electronic BJS pub- lication Drugs & Crime Facts (Dorsey et al., 2004); that report does briefly attempt to pull together a series of findings about drugs from a number of BJS and non-BJS data sets. A number of BJS analyses have also summarized findings of drug-related questions in BJS’s standard data series; for instance, Wilson (2000) summarizes both the Annual Survey of Jails’ information on drug services provided by facilities and the Survey of Inmates in Local Jails’ queries on arrests for drug offenses and prior drug use (these two collections are summarized in Section 3–B.2). Still, in a section of BJS’s strategic plan that briefly itemizes data collections under major section headings, the sec- tion on drug crime statistics notes only that “many ongoing BJS statistical series collect and analyze data on drugs and crime” (Bureau of Justice Statis- tics, 2005a:Fig. 1); in the absence of a coherent overview of what is and is not known from existing data, the interactions of drugs and crime must be considered a gap in the coverage of BJS’s overall portfolio.

MEASUREMENT IN THE JUSTICE SYSTEM 71 2–D ASSESSMENT: FILLING THE GAPS In the panel’s assessment, the four topics we have described in Sec- tion 2–C constitute clear gaps in BJS’s statistical coverage of the events and interactions inherent in the criminal justice funnel model of Figure 2-1: Finding 2.1: The data on crime currently collected by BJS are primarily focused on street crime. This focus on certain forms of violent and property crime does not account for important or emerging types of crime—notably, many forms of white-collar crime such as corporate fraud, health care fraud, financial in- stitution fraud, money laundering, government fraud, consumer fraud, public corruption, and Internet crimes. The broad area of civil justice proceedings—distinct from criminal justice—is rep- resented by one principal data series in BJS’s extensive portfolio, and is limited by its construction to cover only completed court cases (and not out-of-court settlements). BJS’s slate of cross- sectional series also does not readily provide for comprehensive analyses of contextual factors such as drugs and their impact on crime and violence. Finding 2.2: Responsibility within the U.S. Department of Jus- tice for coordinating and organizing data collections on juveniles is generally assumed by the Office of Juvenile Justice and Delin- quency Prevention (OJJDP), instead of BJS. Though BJS’s series do cover some segments of the juvenile population (e.g., juve- niles housed in adult correctional facilities), the results of BJS and OJJDP studies are not well integrated. Within both BJS’s and OJJDP’s statistical coverage, there remain substantial gaps in data for juvenile offenders and victims with respect to their processing through the justice system “funnel.” Clear though these gaps are, it is equally clear that considerable care and caution are in order when suggesting what to do about them. The four topics we have profiled share the basic quality that they are massive and complex, and that crafting full and effective data collection strategies for them would require major innovations in BJS’s current concepts and protocols. • A full focus on white-collar crime would require BJS to shift from its historical norm of using either people or a relatively limited number of establishments (e.g., correctional agencies) as their unit of analysis and grapple with the unique problems of businesses as a unit of study. BJS coverage of white-collar crime would ultimately benefit the large number of agencies that are already involved to varying degrees in the monitoring of such crimes, but would also require extensive coordi-

72 JUSTICE STATISTICS nation with those agencies; this would likely involve the need for an ongoing interagency advisory or coordinating board.7 • Civil trials are a large part of the overall justice system, posing harms to both persons and businesses, and so fuller knowledge of civil jus- tice would be highly beneficial. However, civil justice is also replete with serious definitional issues (hard-to-define concepts, exacerbated by extensive state-by-state variation in legal standards); even defining the range of possible ADRs and determining their applicability in vari- ous states is difficult, much less generating reliable counts of their use. Further, attempts to make the measurement problem more tractable by focusing on filed cases that enter the system miss the large fraction of potential “cases” that are resolved in private. As with white-collar crime, effective expansion of civil justice data collection would neces- sitate involvement and coordination with a number of other actors, including the Administrative Office of the U.S. Courts, the individual state court systems and NCSC, and the American Bar Foundation. • It is complicated, or impossible, to reach the youngest end of the juve- nile population through traditional self-interviewing methods, because of their age and cognitive development. The NCVS limits itself to re- spondents ages 12 and older and other existing data series similarly focus on adolescents and teens; although it is possible that age levels might feasibly be pushed lower (e.g., to age 10 or 11), we know of no evidence of the possible effect of such a switch on the accuracy of self- reports. Collection and release of data on juveniles are also subject to a wide array of legal and ethical restrictions, and studies of juvenile jus- tice raise special sensitivities and heighten the involvement of intensely interested interest groups. Measuring the entry of juveniles into super- vision or residential placement is somewhat complicated because re- ferrals come from a variety of sources other than the police, including family members or guardians and state child welfare bureaus. OJJDP’s person-level measure of juveniles in correctional facilities relies on in- direct responses, through reference to facility records and contact with administrators. A fuller assessment of the quality and coverage of data that may be available in school or juvenile facility records would have to accompany expanded data collections on the juvenile population. To be clear, these conceptual and operational complexities are only one part of the difficulty in suggesting that new data collections be developed to 7 These agencies include, at a minimum, the Federal Trade Commission, the Federal De- posit Insurance Corporation, the Federal Reserve Board, the Office of the Comptroller of Cur- rency, the Office of Thrift Supervision, the National Credit Union Administration, the Mortgage Bankers Association, the U.S. Department of the Treasury, the National Center for Missing and Exploited Children, the FBI, the U.S. Department of Justice Criminal Division and Tax Divi- sion, the U.S. Postal Inspection Service, and the Securities and Exchange Commission.

MEASUREMENT IN THE JUSTICE SYSTEM 73 fill these gaps. The other—and more acute—difficulty is that none of these gaps can be filled without extensive planning, new financial resources, and additional personnel. We think that it is clear that these areas of study are things with which the principal statistical agency of the U.S. Department of Justice should be concerned, but it is equally clear that it is unreasonable to expect that major progress could be made on any of these gaps within what is effectively BJS’s current flat-funding situation. Addressing any of these gaps would require commitment and a sense of high priority from the Justice Department, the administration, and Congress. Given these complexities, our basic suggestion on how to proceed bor- rows from two sources: the “clearinghouse” role defined by law for BJS on data on drugs and crime (point 17, Box 1-2) and OJJDP’s detailed online Statistical Briefing Book. In any of these gap areas, the necessary first step is a structured accounting of what is and is not known from existing data resources, both internal and external to BJS. BJS’s Drugs & Crime Facts is a first step toward such an accounting, as is the compilation of data from a wider variety of sources in the online Sourcebook of Criminal Justice Statis- tics. Still, these gap areas would benefit from a more analytical approach and more complete exploration of existing data sets (and their limitations); BJS’s website (and reports) should more completely catalog external data sources and research, particularly for subject areas where BJS’s own collec- tions are limited. Through such a mapping of problem areas—and the more refined list of specific information needs that the mapping would suggest— BJS would have a more useful template for soliciting input on new data collections, should commitment of resources be secured. Recommendation 2.1: Consistent with its legal mandate to col- lect, analyze, and disseminate statistical information on all as- pects of the justice system, BJS should (a) document and orga- nize the available statistics on forms of crime not covered by the NCVS, the FBI’s UCR and NIBRS data systems, and other ma- jor data series maintained by other statistical agencies, (b) pur- sue research on what new statistics could be feasibly and usefully developed, and (c) propose such new data collections as the re- search suggests to be both feasible and useful. BJS should strive to function as a clearinghouse of justice-related statistical infor- mation, including reference to data not directly collected by BJS. Given our panel’s charge to consider BJS’s relationship to other data- gathering entities within the Department of Justice, we think that the gap of coverage of the juvenile population warrants its own specific recommen- dation. Having concluded that the funnel model of the justice system (Fig- ure 2-1) is a useful and sound one, and given the numerous references to juvenile delinquency in BJS’s legal mandate, we think it odd that BJS’s ced-

74 JUSTICE STATISTICS ing of a complete branch of the funnel to another Justice Department entity is as complete as it is. In short, the measurement of juvenile justice is some- thing in which the principal statistical agency of the U.S. Department of Justice should be fully engaged: Recommendation 2.2: In line with its original charge and to better document and understand the contribution of juveniles to street crime and violence, the victimization of youth, and the consequences for youth and society of their victimization and offending, BJS should develop juvenile victimization, crime, and justice statistical series suitable for describing the patterns of offending and victimization of youth, longitudinal progression of youth through the juvenile and criminal justice systems, and reentry into the community and criminal system. Taking on this responsibility would require additional resources. We hasten to add, however, that this recommendation should not be con- strued as saying that BJS should necessarily usurp (or “reclaim”) data collec- tion functions from OJJDP Like BJS, OJJDP has invested considerable time . and effort in developing its relationships with its data collection providers, and upending those relationships should not be taken lightly. What we do envision through this recommendation is BJS-OJJDP collaboration on re- search on the full juvenile population, including, at a minimum, fuller study of juveniles processed by adult courts and correctional facilities.

Next: 3 Overview of Bureau of Justice Statistics Data Series »
Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics Get This Book
×
Buy Paperback | $80.00 Buy Ebook | $64.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Bureau of Justice Statistics (BJS) of the U.S. Department of Justice is one of the smallest of the U.S. principal statistical agencies but shoulders one of the most expansive and detailed legal mandates among those agencies. Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics examines the full range of BJS programs and suggests priorities for data collection.

BJS's data collection portfolio is a solid body of work, well justified by public information needs or legal requirements and a commendable effort to meet its broad mandate given less-than-commensurate fiscal resources. The book identifies some major gaps in the substantive coverage of BJS data, but notes that filling those gaps would require increased and sustained support in terms of staff and fiscal resources.

In suggesting strategic goals for BJS, the book argues that the bureau's foremost goal should be to establish and maintain a strong position of independence. To avoid structural or political interference in BJS work, the report suggests changing the administrative placement of BJS within the Justice Department and making the BJS directorship a fixed-term appointment.

In its thirtieth year, BJS can look back on a solid body of accomplishment; this book suggests further directions for improvement to give the nation the justice statistics--and the BJS--that it deserves.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!