–3–
Current Demands and Constraints on the National Crime Victimization Survey

THIS CHAPTER BUILDS ON THE DISCUSSION of historical goals in Chapter 2 by examining some contemporary issues and challenges facing the measurement of victimization, in particular the demands and constraints placed on the National Crime Victimization Survey (NCVS). We begin in Section 3–A by discussing survey nonresponse, an emerging challenge facing modern surveys of all types, including federal surveys like the NCVS. Section 3–B discusses basic challenges of self response in measuring victimization, including discussion of crimes that are not well measured in police reports and that are inherently hard to measure: the capability of the NCVS to provide information on these is at once a great strength of the survey and a major, ongoing technical challenge. Section 3–C expands the discussion of analytical flexibility from the previous chapter to include issues of flexibility in measuring new types of victimization as well as changes in basic methodology, including subnational estimation, to meet user needs. We then turn to a basic underlying question—What is the value of measuring victimization?—and consider how the cost of the NCVS compares with various benchmarks in Section 3–E. Section 3–F turns to basic issues related to the coexistence of two related measures of crime in the NCVS and the Uniform Crime Reports (UCR): the general correspondence of the two series over time and resulting questions about the need for two independent



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 41
–3– Current Demands and Constraints on the National Crime Victimization Survey T of historical goals in HIS CHAPTER BUILDS ON THE DISCUSSION Chapter 2 by examining some contemporary issues and challenges facing the measurement of victimization, in particular the demands and constraints placed on the National Crime Victimization Survey (NCVS). We begin in Section 3–A by discussing survey nonresponse, an emerging challenge facing modern surveys of all types, including federal surveys like the NCVS. Section 3–B discusses basic challenges of self response in measur- ing victimization, including discussion of crimes that are not well measured in police reports and that are inherently hard to measure: the capability of the NCVS to provide information on these is at once a great strength of the survey and a major, ongoing technical challenge. Section 3–C expands the discussion of analytical flexibility from the previous chapter to include issues of flexibility in measuring new types of victimization as well as changes in basic methodology, including subnational estimation, to meet user needs. We then turn to a basic underlying question—What is the value of mea- suring victimization?—and consider how the cost of the NCVS compares with various benchmarks in Section 3–E. Section 3–F turns to basic issues related to the coexistence of two related measures of crime in the NCVS and the Uniform Crime Reports (UCR): the general correspondence of the two series over time and resulting questions about the need for two independent 41

OCR for page 41
42 SURVEYING VICTIMS measures. We conclude in Section 3–G, considering both the historical goals of the NCVS (Chapter 2) and the challenges described in this chapter to assess the basic utility of the NCVS. 3–A CHALLENGES TO SURVEYS OF THE AMERICAN PUBLIC 3–A.1 The Decline in Response Rates With a response rate of 91 percent among eligible households (84 percent of eligible persons) as of 2005, the NCVS enjoys response and participation rates that are highly desirable relative to other victimization and social sur- veys. However, the NCVS response rates have declined over the past decade; in 1996, the NCVS household- and person-level response rates were 93 and 91 percent, respectively (Bureau of Justice Statistics, 2006a). Figure 3-1 il- lustrates the recent growth in the noninterview rate in the NCVS and one component of that rate in particular: refusals by anyone in the contacted household to participate in the survey. The figure presents these noninter- view and refusal rates for both initial contacts (interview 1, conducted by personal visit) and for all data collection in the year (including telephone and personal interviews for contacts 2–7 with sample addresses); the initial and aggregate rates generally track each other closely. The decline in response rates is a situation faced by almost all house- hold surveys in the United States (Groves et al., 2002). For instance, the General Social Survey, a cross-sectional household survey conducted by the National Opinion Research Center at the University of Chicago, has expe- rienced a declining response rate in recent years, from response rates in the high 70s (percentages) and a peak of 82 percent in 1993 to 70–71 percent in 2000–2006.1 Remedies to address declines in response rate continue to be developed. The Substance Abuse and Mental Health Services Adminis- tration, contracting with the Research Triangle Institute, implemented a $30 incentive in 2002 in order to induce respondents to return questionnaires for the National Survey of Drug Use and Health due to declining response rates. The highly detailed National Health and Nutrition Examination Sur- vey, conducted by Westat, has experienced a similar reduction in response. There is little evidence that the loss of response rate over time is primar- ily a function of what organization conducts the survey: many of the federal surveys collected by the U.S. Census Bureau (including the NCVS) for var- ious sponsors have shown declines as well. Atrostic et al. (2001) describe measures of nonresponse for six federal surveys (including the NCVS) be- tween 1990 and 1999, documenting consistent declines in response; Bates (2006) updates the series through 2005. An example cited in those works 1 See http://www.norc.org/Projects+2/GSS+Facts.htm [8/20/07].

OCR for page 41
CURRENT DEMANDS AND CONSTRAINTS ON THE NCVS 43 Figure 3-1 Noninterview and refusal rates, National Crime Victimization Survey, 1992–2005 NOTE: Refusal rate is defined as the number of eligible interviewing units not interviewed because occupants refused to participate divided by the total number of eligible interviewing units. Refusals are a component in the noninterview rate, which also includes interviews not completed due to other reasons (e.g., language difficulty or no one at home). Noninterviews are termed “Type A” results. Rates are based on unweighted data. SOURCES: Data from Bates (2006); definitions from Atrostic et al. (2001). is the Consumer Expenditure Diary (CED) survey, data from which are an input used to derive the consumer price index. Between 1991 and 2003, the initial nonresponse rate (failure to respond to the first interview, which, like the NCVS, must be done by personal visit) steadily increased from about 15 percent to about 30 percent; in 2005, the CED had an overall nonresponse rate of 31.1 percent (Bureau of Labor Statistics, 2007:87). A notable exception to the pattern of declining response rates in fed- eral surveys is the American Community Survey (ACS), the replacement for the traditional decennial census long-form sample that asked census respon- dents for additional social and demographic information. However, the ACS also holds a distinct advantage over other Census Bureau surveys because— inheriting from the decennial census—responses to the ACS are required by law (and respondents are so advised). A test conducted by fielding the ACS with wording on the mailing materials suggesting that response is volun- tary (e.g., “Your Response Is Important to Your Community”) rather than mandatory (e.g., “Your Response Is Required by Law”) demonstrated a rad- ically reduced mail response rate: an overall drop of 20.7 percentage points (Griffin et al., 2003, 2004).

OCR for page 41
44 SURVEYING VICTIMS The fact that declines in response rates are not isolated to private surveys rather than federal surveys (or vice versa) suggests that large-scale changes in the relationship between survey data collectors (generally) and the U.S. public have occurred in recent years. While there is no convincing empirical evidence to test alternative theories of the causes of the decline, the most popular hypotheses include: • the lack of trust in the institutions requesting the survey participation; • confusion in potential respondents’ minds between marketing ap- proaches and survey participation requests; • loss in discretionary time at home due to increased commute time to work and other out-of-home activities; • increase in the sheer volume of survey participation requests, making participation in any particular survey less novel; and • the increased investment in electronic and other devices to prevent strangers from contacting the public. There are two principal forms of nonresponse, each of which appears to have separate causes. The first is unit nonresponse: for a household survey like the NCVS, this is nonresponse that arises because the household at a particular address could not be contacted or declined to participate at all. The inability to contact U.S. households is driven both by apparent increases in the out-of-home activities of the public and by changes in how the public views approaches from strangers. There are now more walled subdivisions, locked multiunit structures, and intercom systems that permit residents to control the access of strangers to their housing units. For telephone con- tact, answering machines and “caller ID” features permit residents to limit telephone contact to those persons they wish to talk to. Hence, populations that invest in these housing features and appliances are disproportionately not contacted. These tend to be urban dwellers, younger, more transient persons, and those who live alone. This broader, structural form of non- response is inherent to all surveys. As we discuss in Section 5–B, it is an open question whether the administration of the NCVS by the U.S. Census Bureau is a net positive or negative (or neither) in affecting unit response. The second type of nonresponse is within-unit nonresponse: given that contact is successfully made at an address, do all the survey-eligible persons at that address cooperate and answer the survey questions? There is evidence that persons who are interested inherently in the announced topic of the sur- vey tend to respond (Groves et al., 2004). There is also evidence that women cooperate more prevalently than men (DeMaio, 1980); that urban dwellers cooperate less frequently than those in rural areas (Groves and Couper, 1998); and that those who live alone and middle-aged persons are less coop-

OCR for page 41
CURRENT DEMANDS AND CONSTRAINTS ON THE NCVS 45 erative. To the extent that person nonresponse relies on interest in the sur- vey’s topic area, the purview of the NCVS presents complications on both ends of a continuum. For people who have been victimized—particularly highly sensitive crimes like sexual assault—interviewers may face a difficult task in building rapport so that respondents are willing to talk about their experiences. Likewise, interviewers have to be trained to handle the oppo- site situation: people who have not experienced recent victimization and hence attempt to bow out of the survey because they think it irrelevant. Survey design features may have some role in affecting response rate, or at least in curbing the loss of response rate. There is evidence that longitudinal surveys of persons—in which multiple contacts are made with the same households and people, forging longer term “relationships” be- tween interviewers and subjects—have experienced lower declines in par- ticipation. The NCVS is a longitudinal survey of addresses, not persons, and thus may be affected by turnover of individual persons or families at sampled addresses. However, nonmovers—people who remain at the same address over time—can experience up to seven NCVS requests and thus—conceptually—the NCVS response rate should enjoy some benefit from those repeated contact efforts. As Lepkowski and Couper (2002) note, however, the propensity to respond in later waves of a longitudinal survey is dependent on the enjoyment of the prior wave. If NCVS respondents in one wave find the survey less than pleasant, there may be lower propensity to respond in the next wave. Telephone surveys appear to suffer more dramatic nonresponse rate in- creases than face-to-face surveys. This finding primarily comes from evi- dence from random-digit-dialed surveys (Curtin et al., 2000). The NCVS does use the telephone for waves 2–7 of interviewing, but, given this is com- mon to other longitudinal surveys, it is unlikely that the use of the telephone in NCVS interviewing is, in itself, a principal cause of lower response rates. It is important to note that nonresponse rates are only proxy indicators of one aspect of the quality of NCVS estimates. The key issue is whether the propensity to be successfully measured among NCVS sample members is correlated with the likelihood of victimization. Tests conducted along- side the British Crime Survey (BCS) and the Scottish Crime Survey (SCS) provide useful evidence along these lines. Lynn (1997) describes a BCS ex- periment that urged nonrespondents to provide some limited information; people who said that they did not want to be interviewed were pressed to give very short answers about the extent of recent victimizations against them. These capsule assessments were found to be consistent with victim- ization estimates among people who completed the survey. Similar find- ings were registered in an SCS test documented by Hope (2005); that test also compared responses gained by face-to-face interviewing compared with telephone response, since a change to telephone collection was being con-

OCR for page 41
46 SURVEYING VICTIMS sidered for the 2004 administration of the SCS. In the test, face-to-face and telephone interviews were conducted in parallel; the face-to-face interviews recorded a 67 percent response rate compared with 49 percent by telephone, with the difference attributed to refusals to be interviewed. When victim- ization estimates from the two modes of administration were compared, the telephone rates were found to be consistently higher, raising the possibil- ity that the telephone administration had the effect of oversampling persons with incidents to report. A follow-up test recontacted some respondents from the first survey and compared victimization estimates for the group of people who responded on the first contact with those who had their refusals “converted” to responses in the second pass. Victimization rates were found to be lower for the “converted” group than the initial respondents, corrob- orating the hypothesis that refusals are more likely to include nonvictims (people with no incidents to report) rather than victims. 3–A.2 The Rise in Survey Costs From the fiscal and operational standpoint, the major consequence of in- creased nonresponse is increased survey costs. These are incurred when the data collection effort seeks to maximize response rates, devoting field re- sources to repeated attempts to contact households for interviews. Cost in- flation would be more modest if only one call were made to each household. In the NCVS and other surveys seeking high-quality estimates, repeated call- backs and efforts at persuasion are introduced on sample cases that have not yet been interviewed. Repeated calls in face-to-face surveys require the in- terviewer to drive to the sample unit and attempt contact. If no one in the household is at home, another call—often on another trip—is required. If a contact is achieved but the householder is reluctant to participate at the time, another call is made. What results from such a recruitment protocol is that noninterviews require more effort than interviews; the cost of a failure is larger than the cost of a successful interview. As the difficulty of making contact and gaining cooperation increases over time, the costs of the total effort increase if response rates are to be maintained. In short, attempting to achieve high response rates in a survey of a pop- ulation presenting growing difficulty in making contact and gaining cooper- ation will lead to cost inflation. 3–A.3 The Linkage Between Response Rates and Nonresponse Error It is traditional to attempt to maximize response rates in an effort to reduce nonreponse error. This flows from a simple deterministic view of nonresponse error in a sample mean (like the number of victimizations re- ported divided by the number of persons) as a function of nonresponse rates and the difference between respondent and nonrespondent means. Increas-

OCR for page 41
CURRENT DEMANDS AND CONSTRAINTS ON THE NCVS 47 ingly, empirical studies have shown that a stochastic view of nonresponse is more appropriate, viewing each decision to be a respondent as subject to uncertainty. In this view, high correlation between the likelihood of partici- pating and the survey measures produces nonresponse bias in such descrip- tive statistics. Which NCVS estimates might illustrate such links to response propensities is at this point an unknown question. Some NCVS estimates might be biased from the nonreponse and others might not. New studies are appropriate to gauge what value BJS should place on high NCVS response rates, given both the current design and for future alternative designs. Given the ubiquity of the nonresponse problem across federal surveys, recent U.S. Office of Management and Budget (2006b) guidelines call for analyses of nonresponse bias when either unit or item nonresponse hits cer- tain levels. The NCVS unit response rate is such that this threshold has not been crossed; still, we know of no effort by BJS or the Census Bureau to mount a full nonresponse bias study for the NCVS. 3–B CHALLENGES OF SELF-RESPONSE IN MEASURING VICTIMIZATION 3–B.1 Cognitive Challenges: Telescoping and Forgetting As noted above, the NCVS emerged from the National Crime Survey only after several years of conceptual development and methodological re- search. The research was path-breaking in that it helped launch what is sometimes called the cognitive aspects of survey measurement (CASM) movement (aided by a Committee on National Statistics workshop, National Research Council, 1984). Much of the labor of that redesign effort (Bider- man et al., 1986) targeted improved reporting among respondents to the NCS. Importing key notions from cognitive processing models, it was noted that autobiographical reports were fraught with weaknesses. Memories were viewed as being formed at an “encoding” step, in which sense-based obser- vations were retained, often in a manner that was heavily dependent on the situation during the experience of the events. Not all encoded memories were easily retrieved upon a desire to do so. The studies found that “forget- ting” events that did occur was a challenge to the survey. Consistent with long-standing results from cognitive psychology it was found that events that did not induce emotional reactions (“nonsalient”), those that happened frequently, and those that occurred far back in time tended to be underre- ported. Thus, “forgetting” was a problem for the NCS. Research on context-dependent recall suggested that individual words and mentions of types of related events were effective “cues” to memory recall. Much of the research on the screener questions, therefore, was at- tempting to improve the rate of reporting of incidents as a way to attack

OCR for page 41
48 SURVEYING VICTIMS the problem of “forgetting.” The “short cue” version of the instrument that resulted from the research attempted to provide a rich set of cues for each victimization type. In this regard, a marriage between the incident report and the screening questions was key. The screener questions were designed to maximize recall, even at the risk of overreporting incidents through du- plicate reports about the same event or misdating of an event that occurred outside the reference period. The role of the incident reports was to dupli- cate those reports. The finding that forgetting was a function of the salience of the event to the person and the length of time since the event implied that smaller vic- timizations occurring further back in time were most fraught with reporting errors. The length of the reference period (the time from the start of the eligible time period for events to be in scope to the end of the period) and the length of the recall period (the time between the start of the in-scope period and the day of the interview) were issues that could affect the qual- ity of reports. Longer periods yielded poorer reports (Miller and Groves, 1985; Czaja et al., 1994), generally a mix of forgetting and misdating events. The redesign recommended a 6-month reference period, a recommendation based on the findings of increased measurement error due to forgetting and telescoping in 12-month reference periods. There was another antidote to misdating or telescoping errors, which was already in place in the NCS—the use of a bounding interview. A bound- ing interview in the context of the NCS was the first wave interview with each respondent, in which events in the 6-month reference period before the interview were reported. No data from the bounding interview were used in estimation (another recommendation stemming from findings of forward telescoping errors). Instead, the events reported in the bounding interview were made known to the second wave interviewer to verify that a incident reported in that interview was not a duplicate of a report in the first, bound- ing interview. This was thought to reduce forward telescoping errors in the NCS estimates. Some research in the redesign focused on whether the data from the bounding interview might be integrated through statistical models into the estimates, but that never led to such a recommendation. The panel notes that the design features of the reference period, the cuing mechanisms of the screener questions, the nature of the incident reports, and the use of the bounding interview technique are mutually connected. It is difficult to evaluate one of these features without simultaneously considering the others.

OCR for page 41
CURRENT DEMANDS AND CONSTRAINTS ON THE NCVS 49 3–B.2 Measuring Hard-to-Measure Crimes A key conceptual strength of the NCVS is its ability to elicit informa- tion about victimization incidents that are not reported to police. This is particularly the case for such personally sensitive crimes as rape and domes- tic violence, as well as simple assault and other incidents that are acts of violence but that victims may decline to report (or judge that they are not sufficiently severe to report) to authorities. The 1992 redesign concentrated on improving the screening questionnaire to more effectively and accurately probe respondents to recall and report such incidents, doing so by increas- ing the density of cues and using multiple frames of reference. Both the pre- and postredesign questionnaires emphasized the need to broach questions in language that is accessible and understandable (and not steeped in legal jargon) in order to boost cooperation and accurate recall. However, improvements to the screening and cuing procedures leave open the question of whether reporting of hard-to-measure crimes is full and complete and whether other approaches may be preferable. Concep- tually, the measurement of sensitive crimes through personal interviewing is a sounder approach than reliance on police reports, but it is important to consider that some crimes that are not reported to police may not be re- ported to interviewers, either. From the technical perspective, some hard-to- measure crimes—notably domestic violence—present continuing measure- ment challenges due to their high frequency; determining an accurate count is a formidable difficulty, and detailed information on specific incidents even more so. In this section, we briefly discuss the challenges in getting accurate survey reports in two areas: measurement of rape and domestic violence and description of repeated (series) victimizations. Rape, Domestic Violence, and Simple Assault Several researchers have reported that the NCVS yielded lower estimates of the incidence of rape and domestic violence than other surveys. For exam- ple, before the 1992 switch to a redesigned instrument, the National Crime Survey produced estimates of domestic violence that were an order of mag- nitude smaller than those produced by other surveys.2 Similar results obtain 2 Bachman and Taylor (1994) compared the then-available estimates of family violence against women from the NCVS to results from the National Family Violence Survey (NFVS). The cross-sectional NFVS was conducted twice, in 1975 and 1985, by the Family Research Laboratory of the University of New Hampshire and reached a sample of 2,143 and 6,002 households in the two administrations, respectively. The survey suggested that around 160 per 1,000 married couples experienced at least one “violent incident” in 1975 and 1985. By com- parison, the NCVS—which did not ask specifically about violence by family members before the redesign—yields an estimate of the annual rate of famly violence against women of just 3.2 per 1,000. However, the two survey estimates are not directly comparable because they frame

OCR for page 41
50 SURVEYING VICTIMS Table 3-1 Rape and Assault Rates, National Crime Victimization Survey and National Violence Against Women Survey, 1995 NVAWS NCVS Adjusted NCVS Rape 8.7 1.9 2.6 Intimate Partner Assault 44.2 6.6 26.7 Assault 58.9 25.8 80.4 NOTES: Rates per 1,000 population. “Adjusted NCVS” are estimates calculated by including the reported count of incidents in series victimizations in the estimates. SOURCE: Rand and Rennison (2005). for rape: the pre-1992 NCS instrument did not define rape for the respon- dent and did not directly ask respondents whether they had been victims of attempted or completed rape (Bachman and Taylor, 1994:506). The re- designed NCVS asked respondents more directly about family violence and rape. The new instrument asks specifically about violence or threats per- petrated by “a relative or family member.” The redesigned instrument also asks more directly about “unwanted sexual activity,” including from those who are well known to respondents. A range of probes also distinguishes between verbal and other threats, attempts, and completed rapes. Post-redesign research comparing results from the new NCVS instrument with previous versions has largely focused on broader crime categories (Kin- dermann et al., 1997) or differences by analytic groups (Cantor and Lynch, 2005) and not on specific crimes like rape or domestic violence. Still, sev- eral studies compared NCVS rates with those generated by other surveys on these categories. Rand and Rennison (2005) contrasted rape and as- sault rates in the 1995 NCVS with those from the National Violence Against Women Survey (NVAWS), a telephone survey of U.S. adults. They obtained the estimates for annual incidence shown in the NCVS and NVAWS columns of Table 3-1. Rand and Rennison (2005) suggest several explanations for the discrep- ancy between the data sources: the NVAWS may elicit more victimizations by asking about rapes and assaults more explicitly: the NVAWS may be more vulnerable to telescoping, in which incidents outside the reference period are included; or the two data sources may diverge because of their measure- ment of recurring victimization. As a one-time, single-interview survey, the NVAWS had no capacity for bounding responses, “suggesting that [NVAWS] incidents differently: the NFVS instrument asked specifically about “conflict” among family members rather than more detailed probes about violent incidents.

OCR for page 41
CURRENT DEMANDS AND CONSTRAINTS ON THE NCVS 51 estimates are likely inflated to some unknown degree” (Rand and Rennison, 2005:274). (Response rates in the NVAWS are also much lower than the NCVS, although it is difficult to know what biases might result.) The NCVS records as a single series victimization a group of six or more victimizations that were similar in nature but difficult for the respondent to recall indi- vidually. Rand and Rennison (2005:275) estimate that series victimizations account for about 10 percent of violent incidents against women. BJS publi- cations exclude series victimizations from annual estimates. After adjusting for age and crime types and counting the number of incidents among series victimizations, Rand and Rennison (2005) obtained the estimated rates of annual incidence reported in the “Adjusted NCVS” column of Table 3-1. Despite the adjustment for series victimization, rates of rape and intimate partner assault are lower in the NCVS than in the NVAWS. However, Rand and Rennison (2005:279) found that the difference is statistically significant only in the case of intimate partner assault. The discrepancy between the data sources is largest for intimate partner violence, suggesting that at least part of the divergence may be due to the classification of intimate partners rather than the measurement of victimization. Research on the NCVS redesign also suggests that measurement of sim- ple assaults (without a weapon resulting in minor injury) also depends closely on the survey instrument. With a broader screening interview that cued re- spondents to consider events they might not define as crimes, the redesigned survey recorded roughly twice the number of simple assaults than the old NCVS (Lynch, 2002). While it is difficult to gauge whether there is still underreporting of less serious personal crime in the NCVS, research on the redesign underlines the sensitivity of estimates to the survey instrument. The possibility that such crime types as sexual victimization and domes- tic violence may still be underreported in the standard personal interview context—despite improvements in cuing and screening—highlights the im- portance of researching means for incorporating self-response options in the NCVS. These include such approaches as web administration and turning the computer laptop around for parts of an interview so that respondents read and answer some questions without interaction with the interviewer. We discuss these further in Chapter 4. Repeated Victimizations Repeated victimizations may be underestimated in the NCVS because of the way in which series victimizations are handled. As described in Sec- tion C–3.d, NCVS interviewers collect specific information (using an In- cident Report form) for each victimization incident reported by a respon- dent except in instances when six or more very similar incidents occurred within the 6-month reference period. In those cases, a single incident form

OCR for page 41
70 SURVEYING VICTIMS commonly described as “statutory performance indicators” or SPIs.6 The SPI data are collected and audited annually by the Audit Commission and are also published on government websites. BCS data are formally required for several of these indicators (Home Office, 2007): for example, in the set of SPIs defined for 2006–2007, “the percentage of people who think their local police do a good job” (SPI 2a), “perceptions of anti-social behaviour” (SPI 10b), and the violent crime rate (SPI 5b). Meeting these statutory guidelines requires that the BCS be regularly funded and capable of providing estimates at the local government level. 3–F ISSUES RELATED TO THE COEXISTENCE OF THE NCVS AND THE UCR For more than three decades, the nation has had two national indicators of crime: the Uniform Crime Reporting program and the National Crime Victimization Survey. As described in Chapter 2, the two programs over- lap in the crimes they cover (and both are used to generate national-level estimates of violent crime) but also differ in some important definitional ways. Despite the definitional differences between the two measures and their complementary nature, a fundamental question still arises in public discussions of crime statistics: Is it necessary to have two data systems for the purpose of estimating and evaluating trends in crime? One part of that broader underlying question concerns the trends shown by the two series and the degree to which they agree or converge over time: In other words, do police records generally reflect victimization trends, and vice versa? A second part of the bigger question is more philosophical, con- cerning the necessity of two series: Does there remain the need for a second indicator completely independent of the official police reports? 3–F.1 Do Police Record Reports Reflect Victimization Trends? The question about the concurrence of the NCVS and UCR trends is made more salient by the fact that there appears to have been a convergence in recent years of UCR- and NCVS-based national estimates of serious vi- olent crime (i.e., rape, robbery, and aggravated assault). In other words, estimates from the NCVS of the number of crimes victims say they have re- ported to the police and the number that are recorded in the UCR program have grown closer in recent years (see Figure 3-2). On its face, the evidence of the most recent years of the series might suggest a redundancy—that na- tional crime trends may be adequately described by UCR and that the NCVS role as a crime trend monitor may have diminished. 6 Additional information on best value performance indicators is available at http://www. bfpi.gov.uk/pages/faq.asp.

OCR for page 41
CURRENT DEMANDS AND CONSTRAINTS ON THE NCVS 71 The correspondence or divergence of the UCR and the NCVS takes on extra importance when there appears to be a shift in long-term crime trends. Between 1992 and 2004 the United States experienced a long period of declining crime, by both measures. More recently there have been signs of an upturn in crime in a number of cities, and the national crime rate as measured by the UCR has increased in two consecutive years. It is precisely at turning points such as the current one that rich and timely measures of a variety of aspects of crime are required, to better understand the trajectory that the nation seems to be taking. Although the UCR and NCVS estimates of the number of serious violent crimes reported to the police have generally become more similar over the past decade, this convergence in levels is not yet fully understood, nor is it clear whether the similarities in estimates will continue in the future (see Lynch and Addington, 2007). This is because the convergence in the esti- mates does not necessarily reflect a reduction in error by one or both series. Rather, the two series can produce different estimates and varying long- and short-term trends because they measure different aspects of the crime prob- lem using dissimilar procedures. Furthermore, even if the convergence does reflect some reduction in error associated with one or both series, there is little evidence to suggest that this pattern will remain constant in the future. As evident in Figure 3-2, annual estimates of the total number of serious violent crimes derived from NCVS data have often been higher than the annual counts in the UCR. There are several reasons why this may occur. Most importantly, the NCVS data include crimes that are not reported to the police. Approximately 49 percent of violent victimizations and 36 percent of property victimizations are reported to the police (Hart and Rennison, 2003). In addition, NCVS counts may be higher if police departments do not record all of the incidents that come to their attention or do not forward the reports to the national UCR program. For some types of crimes in the NCVS and the UCR, it is possible to reconcile apparent discrepancies in annual estimates by adjusting the NCVS counts to include only those incidents said to have been reported to the police. When such adjustments are made, levels and trends in burglary, rob- bery, and motor vehicle theft appear generally similar in the NCVS and UCR. However, UCR and NCVS levels and trends in serious violent crime, such as aggravated assault and rape, exhibit many discrepancies after these kinds of adjustments are made. These differences in both levels and trends in ag- gravated assault and rape may result from changes concerning the public’s willingness to report crime to the police, changes in the way police depart- ments record crime, or some other factor. It is clear that the differences in the methodologies of the UCR and NCVS programs must be considered when assessing both levels and trends of crime in the nation. However, the fact that the extent of agreement in current levels of crimes depends on the

OCR for page 41
72 SURVEYING VICTIMS Figure 3-2 National Crime Victimization Survey and Uniform Crime Reports estimates of serious violent crimes, 1973–2005 NOTES: Serious violent crimes include rape, robbery, aggravated assault, and homicide; homicide estimates from the UCR are added to the NCVS series. “NCVS Actual” includes crimes not reported to the police as well as those that are (“NCVS Reported”). NCVS estimates before 1993 are based on data year; for 1993 and later years collection year is used (see Table C-2). SOURCE: National Crime Victimization Survey, Bureau of Justice Statistics and Uniform Crime Reports, Federal Bureau of Investigation. Data from http://www.ojp.gov/bjs/glance/tables/4meastab.htm [11/1/07]. nature of the offense makes it difficult to claim that UCR data alone could be a sufficient indicator for estimating current levels of “crime.” If the goal is to assess long-term trends in crime, then a high correla- tion between UCR and NCVS trends would suggest that either data series would serve as a reasonable proxy for some analytical purposes. McDowall and Loftin (2007) assessed the correlations between UCR and NCVS na- tional trends for index crimes for the period 1973–2003. Using a correla- tion standard of 0.80 or higher to indicate sufficient agreement in trends, they found that only two crimes came close to or exceeded this standard: robbery (r = 0.76) and burglary (r = 0.93). The next highest correlation was found for motor vehicle theft (r = 0.67). However, the remaining crime types exhibited much lower or even negative correlations. For larceny theft, the correlation was weak (r = 0.20), and for rape and assault, the correla- tions were negative (r = −0.16 and r = −0.21, respectively) (McDowall and Loftin, 2007:101). Like current level estimates, the trend correlations varied according to crime type. Analysts studying robbery and burglary can expect

OCR for page 41
CURRENT DEMANDS AND CONSTRAINTS ON THE NCVS 73 generally similar results using either UCR or NCVS trend data for this time period. For other types of crime, however, this will not be the case. One of the main hypotheses about why assault trends might differ in the UCR and NCVS series is heightened police productivity resulting in a growth of police estimates of assaults (O’Brien, 1996). Rosenfeld (2007) hypothesized that if the divergence in the two series was the result of changes in the way police were handling less serious assaults, then one should expect to see similar trends in the UCR and NCVS gun assault rates, but divergent trends in the nongun assault rates because the perceived seriousness of gun assaults and the ways in which such crimes are handled by the police are much less susceptible to change over time. In addition, if the hypothesis is correct, the ratio of nongun to gun assaults should have increased more in the UCR than in the NCVS. Using UCR and NCVS aggravated assault data for the period 1980–2001, Rosenfeld found that the correlation between the UCR and NCVS estimates of gun assaults was 0.74, while the correlation for nongun assaults was 0.16 (not significantly different from 0). In addition, the ratio of nongun to gun assault rates in the UCR grew, while the same ratio using the NCVS data did not. Thus, the two data series provided similar information about trends in gun-related aggravated assault, but they differed in their patterns for aggravated assaults without guns. The form of the UCR and NCVS nongun assault trends also suggested that changes in police recording and categorization of such incidents stabilized during the 1990s as the two series began to exhibit more similar trends. In their comprehensive examination of UCR and NCVS trends, Mc- Dowall and Loftin (2007) found that the two series for each of the index crimes began tracking each other more closely in the 1990s. McDowall and Loftin argue that this would suggest a structural break in the UCR data series, which might indicate that the estimates from the two series will continue to follow each other more closely in the future. The reason that modifications in the UCR are thought to be responsible for the increased correspondence is that there have been changes in the recordkeeping systems of police de- partments, while the NCVS methodology remained relatively more stable over the same period (McDowall and Loftin, 2007:111). The recordkeep- ing capacities of police departments improved as a result of technological innovations, as well as increased numbers of personnel involved in this task. However, the authors note that the agreement in the series is fairly recent and based on a limited number of data points: thus it is premature to con- clude that it will continue in the future (McDowall and Loftin, 2007:114). Whether it is reasonable for state and local governments to believe that their local police data accurately capture trends in crime is more contentious. When state and local governments are interested in assessing trends in crime in their own areas, they typically must rely solely on police data because vic- timization survey data are rarely available for small geographical areas. The

OCR for page 41
74 SURVEYING VICTIMS collection of reliable crime survey data is costly, and most state and local governments have not had the resources to conduct their own victimization surveys, especially on an annual basis. As a result, many wonder whether conclusions about the recent convergence in national police and victim sur- vey data apply to their local areas. The limited amount of research that has addressed the comparability of UCR and NCVS trends in local areas has used data from special tabulations of NCVS data. One such special subset allows researchers to produce vic- timization estimates for the 40 largest metropolitan core-county areas in the country (Bureau of Justice Statistics, 2007b). NCVS estimates can be gen- erated from this newly available file for comparisons to UCR data for those same places and years (Lauritsen and Schaum, 2005). Existing research us- ing these data has found that the correlations in the trends for robbery and aggravated assault vary considerably across metropolitan areas, although much less so for robbery than for aggravated assault (Lauritsen, 2006a). For robbery, the average UCR-NCVS trend correlation across the 40 largest metropolitan areas for 1979–2004 is 0.59 (range = 0.02 to 0.95), while the average correlation for aggravated assault is 0.16 (range = −0.75 to 0.76). In addition, the trend correlations tend to be higher in the more populated metropolitan areas, which may reflect earlier adoption of crime records man- agement technology by the larger police departments. Lauritsen concluded that there is a good deal of variation at the level of the metropolitan statis- tical area in the correlations between the two sets of trends that is masked at the national level, and, as a result, it would be unwise for local areas to assume that their local UCR data provide good indicators of nonlethal vi- olence trends. In addition, Wiersema (1999) developed an area-identified NCVS data set from public-use data files, coded with geographic identifiers down to the census tract level, that was briefly available through Census Bureau research data centers. These area-identified data have been used to support some subnational analyses; see, e.g., Baumer (2002); Baumer et al. (2003); Lauritsen (2001). However, the data have been taken out of circu- lation; we discuss this further in Section 5–A. In sum, although much less is known about how UCR data trends com- pare with NCVS trends for state and local areas, it appears that at the na- tional level, index crime trends have become more similar in recent years. However, just as structural breaks in UCR data collection appear to be re- sponsible for the increasingly similar trends in the 1990s, changes may occur in the future. These alterations can result from a variety of factors, ranging from resource shortages in police departments to local political pressures regarding crime rates. Changes in the NCVS estimates can occur as well, as a result of declines in participation rates, sample coverage problems, lim- ited resources, or other factors. Without both sources of crime information,

OCR for page 41
CURRENT DEMANDS AND CONSTRAINTS ON THE NCVS 75 it extremely difficult to fully understand the meaning of future changes in crime rates from either data series. McDowall and Loftin (2007) and others (e.g., Lynch and Addington, 2007) argue that too much emphasis should not be placed on the general issue of convergence in national crime trends. Rather, the complementari- ties of the two data systems should be emphasized and the selection of one series over another should depend on the research question at hand. For some types of analyses, researchers can use both data sources to assess and understand the strengths and limitations of findings. However, the NCVS is currently the only available source of national data for describing and understanding trends in certain types of crime. These crime types include those that are defined according to specific conditions of the incident (such as intimate partner violence); victim characteristics (such as violence against women or crimes against the elderly); and crimes that are believed to be severely underrepresented in police data for assorted reasons (such as hate crimes, sexual violence, and identity theft). 3–F.2 Independence of the NCVS and the UCR The origin of the NCVS as an independent estimate compared with the UCR was a development from the social and political climate of the 1960s. Cantor and Lynch (2000:97) note that “the confluence of several forces”— including a general mistrust of institutions—“made the 1960s an auspicious time for the development of victim surveys.” Specifically, “reforms of sev- eral of the Nation’s metropolitan police departments were accompanied by exposés of the previous practice of killing crime on the books”—that is, suppressing levels of reported crime. Against this backdrop, “victim surveys brought the ‘patina of science’ ” and an air of accuracy and impartiality to crime statistics; “there was greater trust that the resulting [NCVS] crime es- timates were not purposely manipulated” because “the Census Bureau and survey research agencies were not interested parties with respect to the crime problem” (Cantor and Lynch, 2000:88–89). Beyond the question of whether the UCR and the NCVS respond to the same underlying phenomena, the question can be raised about whether the need for a independent, national-level, and victimization-based measure of the traditional index crimes persists. If it were concluded that there was no need for such an independent national-level measure, then a different class of NCVS design options becomes feasible, if not preferable: for in- stance, crime-type coverage between the NCVS and the UCR would be re- allocated so that the UCR becomes the sole source of national indicators of some crimes, while the NCVS is focused more on hard-to-measure or newly emerging crime types. We think that the NCVS has strong policy relevance as a national-level

OCR for page 41
76 SURVEYING VICTIMS measure of crime independent of the UCR and should continue to function as such, for several key reasons. These include: • The NCVS as an objective measure: Given that police are not an unin- terested party in crime rates, there is always an inherent possibility of minimizing or reducing reported crime. Put more colloquially, having the police as the reporter of crime suggests the possibility of “cooking the books,” lowering reported counts or possibly declining to report altogether. The UCR can do some imputation (and does), but an inde- pendent measure as a counter to this possibility still has merit. (Note, though, that this argument is weakened in the absence of local-area NCVS-based estimates, which would be the best check on individual department reporting.) • Voluntary UCR reporting leads to coverage gaps: The UCR program relies on the voluntary cooperation of more than 17,000 seprate law enforcement agencies. Complete nonresponse to the UCR program, for individual years or for long stretches of time, occurs and is some- times pervasive for some states and large localities (see, e.g., discus- sions of UCR coverage in Maltz, 1999, 2007). Again, imputation helps to bridge gaps in national-level UCR estimates, but for representative- ness in coverage, the conceptual advantage still goes to a nationally representative sample like that employed in the NCVS. • Independent measure as “calibration” device: It is useful to have two related-but-not-identical measures in simultaneous operation simply because they may not always agree. The United States has two inde- pendent measures of jobs and employment (in the Current Employ- ment Statistics and the Current Population Survey); it has multiple measures of health insurance prevalence and of disability. The dif- ferences among the indicators enhance general understanding of the dynamics of the phenomena under study. Divergent or discrepant find- ings resulting from the two series may signal some structural problem with either of the individual measures and draw attention to poten- tial problems in methodology. An original implicit notion in the cre- ation of the NCVS—grounded in distrust of police reporting—may have been the use of the NCVS as a check on the UCR; however, it is equally valid to say that the two series can serve as an operational and conceptual check on each other. At the same time, in speaking of “calibration,” it is important to bear in mind that one data source is not always unequivocally right and the other wrong; both the NCVS and UCR are subject to measurement flaws. UCR measurement can suffer from lack of reporting by law enforcement agencies (discussed below) and underreporting due to victims’ hesitance to come for-

OCR for page 41
CURRENT DEMANDS AND CONSTRAINTS ON THE NCVS 77 ward to the police. But, likewise, crimes may not be reported to interview- ers either, as discussed in Section 3–B.2; other examples of crimes missed in NCVS exist, including the finding by Cook (1985) that National Crime Survey estimates captured only about one-third of gun assaults resulting in gunshot injury that were apparent in emergency room data. Such underes- timates might arise from disproportionate gunshot prevalence among those not part of the household population or not listed as household members (and thus not sampled), those who were never contacted or who refused to participate, as well as those respondents who failed to report incidents to the NCVS interviewer. Having an independent measure is important as long as there remains reason to believe that not all crime is reported to police and that not all crimes known to the police are completely tallied in the UCR. That said, the utility of an UCR-independent measure of crime should not prevent consid- eration of design options that reduce lockstep similarity between the UCR and the NCVS (e.g., measuring exactly the same set of “index crimes” except for homicide). For several years, the National Incident-Based Reporting System (NIBRS) has been developed as a next-generation version (and replacement for) the UCR. The presence of a strong and complete NIBRS program might further blur the line between the UCR and the NCVS as separate indicators of crime. However, NIBRS development has been slow, and its coverage (i.e., cooper- ation by agencies in providing more detailed incident reporting) is still quite small. As of September 2007—about 15 years after development of initial NIBRS protocols—only about 26 percent of law enforcement agencies that contribute data to UCR were submitting NIBRS-compliant information; see Section D–2 for additional detail. 3–G ASSESSMENT As is true of many multipurpose social indicators, the basic utility of the NCVS to the American public is difficult to characterize in tangible terms. Because it does not currently provide estimates at small areas of geography, its role in allocating federal or state funds for criminal justice improvements is limited, and it does not readily lend itself to focusing specific police in- terventions in specific neighborhoods. However, through its focus on pro- viding detail on all types of crime and violence—reported to the police or not—and its rigorous design based on a representative sample with uniform national coverage, the NCVS has undeniable importance as a critical statis- tical indicator. For an informed assessment of the state of public welfare, federal statistical agencies like BJS have a core mandate “to be a credible source of relevant, accurate, and timely statistics” (National Research Coun-

OCR for page 41
78 SURVEYING VICTIMS cil, 2004:3). The NCVS provides information on the extent, consequences, and causes of violent behavior that are not available at the same level of comprehensiveness and quality from any other source. Accordingly, direct reports from BJS on NCVS trends are frequently sought for information and for assessment of new policy, and NCVS data play an important role in na- tional appraisals of child welfare (Federal Interagency Forum on Child and Family Statistics, 2007) and public health (U.S. Department of Health and Human Services, 2000). In our assessment, the need for a victimization-based measure of crime— another indicator, separate from the official police reports of the UCR—is as significant today as it was when the NCVS was first conceptualized. This is the case not out of any inherent distrust of official reports to police or demonstrated inaccuracy therein, but rather for the reason suggested most concisely by the President’s Commission on Law Enforcement and Admin- istration of Justice (1967:18): “No one way of describing crime describes it well enough.” The importance of the crime problem in the United States de- mands ongoing monitoring from multiple perspectives. In this monitoring, the NCVS is a vital complement to the police reports of the UCR, provid- ing valuable information on the context and causes of victimization in ways that summary counts can never do by themselves (and in which even a fully implemented NIBRS would still be lacking). However, in its size and available resources, the current NCVS is not capable of matching the original vision of the survey. As the costs of col- lecting information from the U.S. public have risen, the NCVS budget has not kept pace. Budget reductions have led to cutbacks in NCVS activities, most often through cuts in the total sample size. As it is currently config- ured, the NCVS does not meet the goal of being able to accurately measure year-to-year change in crime trends. That is, the standard errors of change estimates are too large to detect changes of importance to the country; BJS has had to use averages from 2-year groups of data in order to make state- ments about change (see Catalano, 2006), even though inferences from these rolling averages are not as intuitive to users and members of the public as direct estimates of change.7 To state this as a finding: Finding 3.1: As currently configured and funded, the NCVS is not achieving and cannot achieve BJS’s legislatively mandated goal to “collect and analyze data that will serve as a continuous and comparable national social indication of the prevalence, in- 7 For additional information on the use and interpretation of rolling-average estimates from federal survey data, see National Research Council (2007); the American Community Survey will use 3-year and 5-year averages in order to produce estimates for small areas and populations.

OCR for page 41
CURRENT DEMANDS AND CONSTRAINTS ON THE NCVS 79 cidence, rates, extent, distribution, and attributes of crime . . .” (42 U.S.C. 3732(c)(3)). Clearly, given the panel’s charge to consider options for the conduct of the NCVS, one possibility is not to conduct the NCVS at all; we reject that option. To take that option violates the legislative responsibilities of the Bureau of Justice Statistics. Furthermore, the panel thinks that BJS is the appropriate locus of responsibility for victimization measurement. As a fed- eral statistical agency, it alone has the mandate for independent, objective, statistical measurement, with the transparency that can establish public trust in the information (see National Research Council, 2004). Thus, although there should be no need for the panel to do so, we feel obliged to state a recommendation that is already explicit in the mission of BJS: Recommendation 3.1: BJS must ensure that the nation has qual- ity annual estimates of levels and changes in criminal victim- ization. However, the natural corollary is that the resources necessary to ade- quately achieve this mission must be forthcoming: Recommendation 3.2: Congress and the administration should ensure that BJS has a budget that is adequate to field a survey that satisfies the goal in Recommendation 3.1. NCVS’ unique substantive niche is providing information on crimes that are particularly likely to go unreported to the police, for whatever reason— whether fear or stigma, individual distrust of authority, or the perception that a violent act or threat is not significant enough a “crime” to report. Recommendation 3.3: BJS should continue to use the NCVS to assess crimes that are difficult to measure and poorly reported to police. Special studies should be conducted periodically in the context of the NCVS program to provide more accurate mea- surement of such events.

OCR for page 41