National Academies Press: OpenBook

Firearms and Violence: A Critical Review (2005)

Chapter: 5 The Use of Firearms to Defend Against Criminals

« Previous: 4 Interventions Aimed at Illegal Firearm Acquisition
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

5
The Use of Firearms to Defend Against Criminals

While a large body of research has considered the effects of firearms on injury, crime, and suicide, far less attention has been devoted to understanding their defensive and deterrent effects. Firearms are used to defend against criminals. For example, the presence of a gun may frighten a criminal away, thereby reducing the likelihood of loss of property, injury, or death.

In this chapter, we consider what is known about the extent and nature of defensive gun use (DGU). Over the past decade, researchers have attempted to measure the prevalence of defensive gun use in the population. This measurement problem has proved to be quite complex, with some estimates suggesting just over 100,000 defensive gun uses per year and others suggesting 2.5 million or more defensive gun uses per year.

A primary cause of this uncertainty is the disagreement over the definition of defensive gun use—in particular, whether it should be defined as a response to victimization or as a means to prevent victimization from occurring in the first place. There is also uncertainty regarding the accuracy of survey responses to sensitive questions and the related problems of how to effectively measure defensive gun use, the types of questions that should be asked, and the methods of data collection. These disagreements over definition and measurement have resulted in prevalence rates that differ by a factor of 22 or more. While even the smallest of the estimates indicates that there are hundreds of defensive uses every day, there is much contention over the magnitude and the details.

Since answers to this debate precede any serious investigation into other related questions, we focus our attention on summarizing and evalu-

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

ating the DGU estimates from the various gun use surveys. We find that fundamental problems in defining what is meant by defensive gun use may be a primary impediment to accurate measurement. Finally, after reviewing the literature that attempts to count the annual number of defensive gun uses in the United States, we then consider the small set of studies that evaluate the effectiveness of firearms for defense.

COUNTING DEFENSIVE GUN USES

How many times each year do civilians use firearms defensively? The answers provided to this seemingly simple question have been confusing. Consider the findings from two of the most widely cited studies in the field: McDowall et al. (1998), using the data from 1992 and 1994 waves of the National Crime Victimization Survey (NCVS), found roughly 116,000 defensive gun uses per year, and Kleck and Gertz (1995), using data from the 1993 National Self-Defense Survey (NSDS), found around 2.5 million defensive gun uses each year.

Many other surveys provide information on the prevalence of defensive gun use. Using the original National Crime Survey, McDowall and Wiersema (1994) estimate 64,615 annual incidents from 1987 to 1990. At least 19 other surveys have resulted in estimated numbers of defensive gun uses that are similar (i.e., statistically indistinguishable) to the results founds by Kleck and Gertz. No other surveys have found numbers consistent with the NCVS (other gun use surveys are reviewed in Kleck and Gertz, 1995, and Kleck, 2001a).

To characterize the wide gap in the estimated prevalence rate, it is sufficient to consider the estimates derived from the NSDS and recent waves of the NCVS. These two estimates differ by a factor of nearly 22. While strikingly large, the difference in the estimated prevalence rate should, in fact, come as no surprise. As revealed in Table 5-1, the two surveys are markedly different, covering different populations, interviewing respondents by different methods, using different recall periods, and asking different questions.

The NCVS is an ongoing annual survey conducted by the federal government (i.e., the Census Bureau on behalf of the Department of Justice) that relies on a complex rotating panel design to survey a representative sample of nearly 100,000 noninstitutionalized adults (age 12 and over), from 50,000 households. To elicit defensive gun use incidents, the survey first assesses whether the respondent has been the victim of particular classes of crime—rape, assault, burglary, personal and household larceny, or car theft—during the past six months, and then asks several follow-up questions about self-defense. In particular, victims are asked:

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

TABLE 5-1 Comparing Sampling Design of the NCVS and NSDS

 

National Crime Victimization Survey

National Self-Defense Survey

Coverage

  • Noninstitutionalized U.S. population, age 12 and over, each year since 1973

  • Defensive gun use questions to victims (self-reported)

  • U.S population, age 18 and over, with phones, 1993

  • DGU questions to all respondents

Sample design

  • Rotating panel design

  • Stratified, multistage cluster sample of housing units

  • Telephone and personal contacts

  • One-shot cross-section

  • Stratified by region (South and West oversampled)

  • Random digit dialing

Sample size

Approximately 50,000 households and 100,000 individuals

4,997 individuals

Response rate

Approximately 95% of eligible housing units

61% of eligible numbers answered by human beings

Sponsorship

U.S. Census Bureau for U.S. Bureau of Justice Statistics

Research Network

Estimated defensive gun use

116,398 annual incidents using 1993-1994 data from redesigned survey

2,549,862 annual incidents

SOURCE: McDowall et al. (2000: Table 1). Used with kind permission of Springer Science and Business Media.

Was there anything you did or tried to do about the incident while it was going on?

Did you do anything (else) with the idea of protecting yourself or your property while the incident was going on?

Responses to these follow-up probes are coded into a number of categories, including whether the respondent attacked or threatened the offender with a gun.

The NSDS was a one-shot cross-sectional phone survey conducted by a private polling firm, Research Network, of a representative sample of nearly 5,000 adults (age 18 and over). The survey, which focused on firearms use, first assessed whether the respondent used a gun defensively during the past five years, and then asked details about the incident. In particular, respondents were first asked:

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

Within the past five years, have you yourself or another member of your household used a handgun, even if it was not fired, for self-protection or for the protection of property at home, work, or elsewhere? Please do not include military service, police work, or work as a security guard.

If the answer was yes, they were then asked:

Did this incident [any of these incidents] happen in the past 12 months?

The discrepancies in the prevalence estimates of defensive gun use can and should be better understood. Remarkably little scientific research has been conducted to evaluate the validity of DGU estimates, yet the possible explanations are relatively easy to categorize and study. The two surveys are either (1) measuring something different or (2) affected by response problems in different ways, or (3) both. Statistical variability, usually reflected by the standard error or confidence interval of the parameter, also plays some role but cannot explain these order of magnitude differences.

Coverage

Perhaps the most obvious explanation for the wide variation in the range of DGU estimates is that the surveys measure different variables. In the NSDS, for example, all respondents are asked the gun use questions. In contrast, the NCVS inquires only about use among persons who claim to be victims of rape, assault, burglary, personal and household larceny, and car theft. The NCVS excludes preemptive uses of firearms, uses that occur in crimes not screened for in the survey (e.g., commercial robbery, trespassing, and arson), and uses for crimes not revealed by respondents.1

McDowall et al. (2000) found some evidence that these differences in coverage play an important role. In an experimental survey that overrepresents firearms owners, 3,006 respondents were asked both sets of questions about defensive gun use, with random variation in which questions came first in the interview. By holding the survey sampling procedures constant (e.g., consistent confidentiality concerns and recall periods), the authors focus on the effects of questionnaire content. Overall, in this experiment, the NCVS survey items yielded three times fewer reports of defensive gun use than questionnaires that ask all respondents about defensive uses.

The McDowall et al. (2000) crossover experiment is informative and is exactly the type of methodological research that will begin to explain the sharp divergence in gun use estimates and how best to measure defensive gun use. There remains, however, much work to be done. The sample used

1  

It is well known, for example, that incidents of rape and domestic violence are substantially underreported in the NCVS (National Research Council, 2003).

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

in this survey is not representative, and the methods shed light on only one of the many competing hypotheses. Furthermore, this limited evidence is difficult to interpret. Even with a consistent sampling design, inaccurate reporting may still play an important role. For example, estimates from an NCVS type of question would be biased if victims were reluctant to report unsuccessful defensive gun use. Likewise, the estimates found using the NSDS-type survey would be biased if respondents report defensive gun uses based on mistaken perceptions of harmless encounters.

Even if we accept the notion of fully accurate reporting, or at least consistent inaccuracies across the surveys, details on the cause of these differences are especially important. If these discrepancies result because of incomplete reporting of victimization among the classes considered (e.g., rape and domestic violence) in the NCVS, then one must address the measurement error questions again. Certainly, we are interested in the behavior of all victims, not just those who self-report. If instead, the differences occur because the NSDS-type question includes preemptive uses, then the relevant debate might focus on which variable is of interest.

In any case, much of the confusion surrounding the debate seems to center on what is meant by defensive gun use. Self-defense is an ambiguous term that involves both objective components about ownership and use and subjective features about intent (National Research Council, 1993).2 Whether one is a defender (of oneself or others) or a perpetrator, for example, may depend on perspective. Some reports of defensive gun use may involve illegal carrying and possession (Kleck and Gertz, 1995; Kleck, 2001b), and some uses against supposed criminals may legally amount to aggravated assault (Duncan, 2000a, 2000b; McDowall et al., 2000; Hemenway et al., 2000; Hemenway and Azrael, 2000). Likewise, protecting oneself against possible or perceived harm may be different from protecting oneself while being victimized.

Given this ambiguity, perhaps one of the more important and difficult problems is to develop a common language for understanding defensive and offensive gun use. Uniform concepts and a common language will serve to facilitate future survey work, guide scholarly discussions, and enhance understanding of the complex ways in which firearms are related to crime, violence and injury. More generally, a commonly understood language can also influence the development of firearms policy and violence policy more generally.

2  

This lack of a clear definition may also contribute to inaccurate response. If scholars who think about these issues have yet to come up with a clear definition for the behavior of interest, it may be unreasonable to rely on the accuracy of respondents whom, in some cases, may not understand or interpret the question as intended.

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

FIGURE 5-1 Stages and outcome of potential criminal encounters.

SOURCE: Adapted from Kleck (1997: Figure 7.1).

Although defining and measuring different types of gun use (both offensive and defensive) is not a simple matter, a typology similar to the one developed by Kleck may be a useful starting point (1997: Figure 7.1). Figure 5-1 provides a rough summary of the development of a violent or criminal encounter. Firearms and other weapons may be involved at different points in the development of a crime, from threats to realized crimes and injury. At each stage of a potentially threatening encounter, one may be

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

interested in learning about the basic circumstances, about firearms use and other actions, about the intent of the respondent, and about outcomes. The relatively subjective nature of threats, which may or may not develop into criminal events, may justify placing these uses in a separate category (Kleck, 2001b:236). More generally, it would seem useful to distinguish between the more objective and subjective features of firearms use. Eliciting and interpreting relatively objective questions about whether and how one uses a gun may be relatively simple and lead to consensus on these basic matters. Eliciting and interpreting relatively subjective questions on intent may be much more complex and less amenable to consensus conclusions.3

Ultimately, researchers may conclude that it is impossible to effectively measure many aspects of defensive gun use. As noted above, counting crimes averted before the threat stage, and measuring deterrence more generally, may be impossible. Successful deterrence, after all, may yield no overt event to count. Imagine, for example, measuring defensive gun use for a person who routinely carries a handgun in a visible holster. How many times has this person “used a handgun, even it was not fired, for self-protection?” (i.e., the NSDS definition of defensive gun use). In this regard, much of the debate on the number of defensive gun uses may stem from an ill-defined question, rather than measurement error per se.

Response Problems in Firearms Use Surveys

Questions about the quality of self-reports of firearms use are inevitable. Response problems occur to some degree in nearly all surveys but are arguably more severe in surveys of firearm-related activities in which some individuals may be reluctant to admit that they use a firearm, and others may brag about or exaggerate such behavior.4 If some sampled individuals give incorrect answers (inaccurate response) and others fail to answer the survey at all (nonresponse), investigators may draw incorrect conclusions from the data provided by a survey.

3  

A number of scholars have made explicit recommendations for collecting detailed narratives on the nature of the event. See, for example, recommendations made by Cook and Ludwig (1998), Smith (1997), and Kleck (2000). Hemenway and Azrael (2000) and Hemenway et al. (2000) collected and analyzed detailed narratives on gun use incidents that reveal that they are often complex and difficult to categorize.

4  

These same measurement problems were discussed in a report by the National Research Council (2001) that explored the data problems associated with monitoring illicit drug consumption.

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Inaccurate Response

In fact, it is widely thought that inaccurate response biases the estimates of defensive gun use. Self-report surveys on possibly deviant behaviors invariably yield some false reports. Responses are miscoded, and respondents may misunderstand the questions or may not correctly remember or interpret the event. In addition to these unintentional errors, respondents may also exaggerate or conceal certain information.

The literature speculates widely on the nature of reporting errors in the firearms use surveys.5 Some argue that reporting errors cause the estimates derived from the NCVS to be biased downward.6 Kleck and Gertz (1995) and Kleck (2001a), for example, speculate that NCVS respondents doubting the legality of their behaviors or more generally fearing government intrusion may be inclined to provide false reports to government officials conducting nonanonymous interviews. Furthermore, Smith (1997) notes that NCVS respondents are not directly asked about firearms use but instead are first asked whether they defended themselves, and then they are asked to describe in what ways. Indirect questions may lead to incomplete answers.

Others argue that the estimates from the NSDS and other firearms use surveys are upwardly biased. Cook and Ludwig (1998), Hemenway (1997a), and Smith (1997), for example, suggest that the firearms use surveys do not effectively bound events that occur in prior interviews and thus may result in “memory telescoping.” That is, respondents in the NSDS are more likely to report events that occurred prior to the observation window of interest. Furthermore, McDowall et al. (2000) speculate that preemptive uses recorded in the NSDS but not generally covered in the NCVS (which focuses on victims) are susceptible to a greater degree of subjectivity and thus inaccurate reporting.

A number of other general arguments have been raised as to why these surveys might be inaccurate. Some suggest that respondents may forget or conceal events that do not lead to adverse outcomes (Kleck and Gertz, 1995; Kleck, 2001a), while others suggest that respondents may exaggerate or conceal events due to social stigma. Some have even suggested that respondents may strategically answer questions to somehow influence the ongoing public debate (Cook et al., 1997). Finally, Hemenway (1997b) raises what amounts to a mechanical, rather than behavioral, concern

5  

See Kleck (2001a) for a detailed review of the various hypotheses about inaccurate reporting in gun use questionnaires.

6  

Kleck argues that the NCVS is well designed and uses state-of-the-art survey sampling techniques for measuring victimization, but for exactly those reasons it is not well designed for measuring defensive gun use.

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

regarding why the DGU estimates may be generally biased upward. For any rare event, in fact for any event with less than 50 percent probability, there are more respondents who can give false positive than false negative reports. Suppose, for example, in a sample of 1,000 respondents, the true prevalence rate is 1 percent; that is, 10 respondents used a gun defensively. Then 990 may provide false positive reports, while only 10 may provide false negative reports. Even small fractions of false positive reports may lead to substantial upward biases. Cook et al. (1997) further suggest that by focusing on victims, the NCVS reduces the scope of the false positive problem.

Although the rare events problem may be well known and documented in epidemiological studies of disease, it is uncertain whether this same phenomena affects inferences on defensive gun uses as well. People may have reasons to conceal or exaggerate defensive gun uses that may not apply when studying rare diseases. In fact, what is known about accurate reporting of other crime-related activities provides some evidence to the contrary. Validation studies on the accuracy of self-reports of illicit drug use among arrestees, for example, suggest that for this somewhat rare but illegal activity, the numbers of false reports of use are far less than the numbers of false reports of abstinence: self-reports of drug use are biased downward (Harrison, 1995).

Although theories abound, it is not possible to identify the prevalence of defensive gun use without knowledge on inaccurate reporting. Kleck and Gertz (1995) and others suggest that estimates from the NCVS are biased downward, arguing that respondents are reluctant to reveal information to government officials, and that indirect questions may yield inaccurate reports. Hemenway (1997a) and others suggest that estimates from the NSDS are biased upward, arguing that memory telescoping, self-presentation biases, and the rare events problem more generally lead the numbers of false positive reports to substantially exceed the numbers of false negative reports. It is not known, however, whether Kleck’s, Hemenway’s, or some other assumptions are correct. The committee is not aware of any factual basis for drawing conclusions one way or the other about reporting errors.

Nonresponse

While inaccurate response has received a great deal of speculative attention, the problem of nonresponse has hardly been noticed.7 Nonresponse is a problem in survey sampling, but it is especially problematic in the firearms use phone surveys like the NSDS. Although not completely re-

7  

Both Duncan (2000b) and Hemenway (1997a) recognize the potential problems created by nonresponse in the firearms use surveys.

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

vealed by Kleck and Gertz (1995), the response rate in the NSDS appears to lie somewhere between 14 and 61 percent.8 The response rate in the NCVS survey is substantially higher, at around 95 percent.

Survey data are uninformative about the behavior of nonrespondents. Thus, these data do not identify prevalence unless one makes untestable assumptions about nonrespondents. A simple example illustrates the problem. Suppose that 1,000 individuals are asked whether they used a firearm defensively during the past year but that 500 do not respond, so the nonresponse rate is 50 percent. If 5 of the 500 respondents used guns defensively during the past year, then the prevalence of defensive gun use among respondents is 5/500 = 1 percent. However, true prevalence among the 1,000 surveyed individuals depends on how many of the nonrespondents used a firearm. If none did, then true prevalence is 5/1,000 = 0.5 percent. If all did, then true prevalence is [(5 + 500)/1,000] = 50.5 percent. If between 0 and 500 nonrespondents used a firearm defensively, then true prevalence is between 0.5 and 50.5 percent. Thus, in this example, nonresponse causes true prevalence to be uncertain within a range of 50 percent.

Prevalence rates can be identified if one makes sufficiently strong assumptions about the behavior of nonrespondents. In the DGU literature, nonresponse is assumed to be random, thus implying that that prevalence among nonrespondents is the same as prevalence among respondents. The committee is not aware of any empirical evidence that supports the view that nonresponse is random or, for that matter, evidence to the contrary.

External Validity

A number of scholars have suggested that results from the NSDS and other firearms use surveys are difficult to reconcile with analogous statistics

8  

Kleck and Gertz report that 61 percent of contacts with persons for the NSDS resulted in a completed interview. Presumably, however, there were also many households in the original sampling scheme that were not contacted. For example, using data from the National Study of Private Firearms Ownership (NSPFO), a national phone survey designed to elicit information about firearms ownership and use, Cook and Ludwig (1998) report that 29,917 persons were part of the original sampling scheme, of which 15,948 were determined to be ineligible (phones not working, not residential, etc.), 3,268 were determined to be eligible, and the remaining 10,701 were unknown (e.g., no answer, answering machine, busy, etc.). Of the 3,268 that were known to be eligible, 2,568 provided complete interviews, for a response rate of 79 percent among contacted households. The 10,701 with unknown eligibility status must also be accounted for. If none of these households was actually eligible, than the true response rate would be 79 percent. If, however, all of these are eligible, then the true rate would be 18 percent [2,568/(10,701 + 3,268)]. Thus, the response rate in the NSPOF lies between 18 and 79 percent. If the response rates are consistent across the two surveys, the lower bound response rate for the NSDS would be 14 percent [ (0.61/0.79)*0.18].

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

on crime and injury found in other data. For example, Hemenway (1997a) points out that results from the NSDS imply that firearms are used defensively in every burglary committed in occupied households and in nearly 60 percent of rapes and sexual assaults committed against persons over 18 years of age; that defensive gun users thought they wounded or killed offenders in 207,000 incidents, yet only 100,000 people are treated in emergency rooms for nonfatal firearms injuries; and that hundreds of thousands of persons almost certainly would have been killed if they had not used a firearm defensively, implying that nearly all potentially fatal attacks are successfully defended against (Cook and Ludwig, 1998). Cook and Ludwig (1998), Hemenway (1997a), and others argue that these and other similar comparisons lead to “completely implausible conclusions” and go on to suggest that these inconsistencies “only buttress the presumption of massive overestimation” of defensive gun uses in the NSDS (Hemenway, 1997a:1444).

Although potentially troubling, the strong conclusion drawn about the reliability and accuracy of the DGU estimates seems premature. In some cases, it may be that the comparison statistic is subject to error. The reported prevalence of rape in the NCVS, for example, is believed to be biased substantially downward (National Research Council, 2003). More importantly, however, evidence on the apparent biases of the estimated incident rates, wounding rates, and counts of averted injuries does not directly pertain to the accuracy of the DGU estimates. Kleck and Gertz (1995), in fact, note that victimization estimates drawn using the NSDS, a survey designed to measure firearms use rather than victimization, are subject to potential reporting errors in unknown directions. Cook and Ludwig (1998) find evidence of reporting errors of crime in the firearms use surveys, with many respondents reporting that crime was involved on one hand, yet that no crime was involved on the other. Likewise, questions about whether a respondent thought he wounded or killed the offender and those eliciting subjective information on what would have happened had a gun not been used are also subject to substantial reporting biases. As noted by Kleck and Gertz (1998), respondents may be inclined to “remember with favor their marksmanship” and may tend to exaggerate the seriousness of the event.

In addition to invalid response errors, sampling variability may also play an important role in these conditional comparisons. Inferences drawn from the relatively small subsamples of persons who report using firearms defensively (N = 213 in the NSDS) are subject to high degrees of sampling error. Using data from the National Study of Private Firearms Ownership, a survey similar to the NSDS, Cook and Ludwig (1998), for example, estimate that firearms were used defensively in 322,000 rapes (rape, attempted rape, sexual assault) but report a 95 percent confidence interval of

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

[12,000 to 632,000].9 The lower bound interval estimate would imply that firearms are used defensively in less than 3 percent of all rapes and sexual assaults (Kleck, 2001a).

Replication and Recommendations

As indicated above, the estimated numbers of defensive gun uses found using the NSDS have been reproduced (i.e., are statistically indistinguishable) in many other surveys. Kleck (2001a:270) suggests that replication provides ample evidence of the validity of the findings in the NSDS survey:

The hypothesis that many Americans use guns for self-protection each year has been repeatedly subjected to empirical test, using the only feasible method for doing so, survey of representative samples of the populations. The results of nineteen consecutive surveys unanimously indicate that each year huge numbers of Americans (700,000 or more) use guns for self-protection. Further, the more technically sound the survey, the higher the defensive gun use estimates. The entire body of evidence cannot be rejected based on the speculation that all surveys share biases that, on net, cause an over estimation of defensive gun use frequency because, ignoring fallacious reasoning, there is no empirical evidence to support this novel theory. At this point, it is fair to say that no intellectually serious challenge has been mounted to the case for defensive gun use being very frequent.

Certainly, the numerous surveys reveal some phenomena. In light of the differences in coverage and potential response errors, however, what exactly these surveys measure remains uncertain. Ultimately, the committee found no comfort in numbers: the existing surveys do not resolve the ongoing questions about response problems and do not change the fact that different subpopulations are queried. Mere repetition does not eliminate bias (Rosenbaum, 2001; Hemenway, 1997a).

However, the committee strongly agrees with the main sentiment expressed by Kleck and others. Evidence from self-reported surveys will invariably be subject to concerns over reporting errors and other biases. Still, we can hope to have a greater degree of confidence in the survey results by relying on replications and survey sampling experiments that serve to effectively reduce the degree of uncertainty about the true prevalence rate. The objective of these experiments should be consistency of results in a variety of sampling designs. Replications and experiments should disrupt aspects of the original study to check whether the prevalence estimate is reproduced or altered under different survey designs. Effective replications will vary the

9  

Kleck and Gertz (1995) do not report confidence intervals for these conditional estimates.

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

nature of the potential biases in order to explicitly reduce, rather than increase, the prospects of reproducing the original results (Rosenbaum, 2001).

These ideas are not new to this controversial literature. McDowall et al. (2000) do exactly this type of experimental evaluation by holding certain factors constant—namely, the sampling methodology—but varying the content of the questionnaire. Other similar experiments or replications or both could be used to vary the nature of memory telescoping, social presentation bias, and other plausible factors that might influence reporting behaviors. In fact, Cook and Ludwig (1998), Smith (1997), Kleck (2000), and many others make numerous recommendations for experiments or replications.

The committee strongly believes that these types of studies can and should be undertaken. Without reliable information, researchers will continue to be forced to make unsubstantiated assumptions about the validity of responses and thus about the prevalence of defensive gun use.

The committee recommends a systematic and rigorous research program to (1) clearly define and understand what is being measured, (2) understand inaccurate response in the national use surveys, and (3) develop methods to reduce reporting errors to the extent possible. Well-established survey sampling methods can and should be brought to bear to evaluate the response problems. Understanding response will be useful for not only explaining the striking gap in DGU estimates but, more importantly, understanding defensive gun use.

EFFICACY OF SELF-DEFENSE WITH A FIREARM

Accurate measurement on the extent of firearms use is the first step for beginning a constructive dialogue on how firearms are used in American society. Invariably, however, attention will turn to the more important and difficult questions about the consequences of using a firearm for self-defense. How effective are firearms at preventing injury and crime? Would gun users have been better off (on average) using alternative defensive strategies? How does the efficacy of self-defense vary by circumstance (e.g., abilities of victim and perpetrator, location of crime, weaponry)?

Answering these questions is essential for evaluating the costs and benefits of firearms to society. For example, if using a firearm defensively is no more effective than basic avoidance techniques, then defensive gun use would have no relative benefit. In contrast, if firearms are more effective at resisting crime and injury than alternative methods, then civilian ownership and the use of firearms may play a vital role in the nation’s ability to deter and fight crime. Of course, the benefits of defensive gun use must ultimately be weighed against the potential costs that may arise if firearms are involved in the final stages of violent criminal encounters: defensive gun use may lead to relatively higher risks of injury and death to victims or offend-

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

TABLE 5-2 Probability of Injury and Loss Among Victims by Means of Self-Protection

Method

Robbery

Assault

Injury

Loss

Injury

With gun

12.8

15.2

27.9

All self-protection

34.0

52.8

58.1

No self-protection

23.6

83.6

55.2

All incidents

30.2

69.9

57.4

 

SOURCE: Adapted from Kleck (2001b:289, Table 7.1).

ers. Finally, both the benefits and costs must be evaluated within the context of offender weaponry. If criminals were not armed, would firearms be more or less useful for protecting potential victims? If the efficacy of self-defense depends on the number of firearms in society, then partial equilibrium analyses that hold offender weaponry fixed may not answer the right questions.

Empirical Evidence

While the literature on self-defense has been preoccupied with the basic measurement questions, a handful of studies assess the efficacy of defensive gun use.10 Using data from the NCVS, Kleck (2001b) compares the probability of injury and crime by different defensive actions. The results, summarized in Table 5-2, suggest that respondents who use firearms are less likely to be injured and lose property than those using other modes of protection. For example, while the overall rate of injury in robbery is 30.2, only 12.8 percent of those using a firearm for self-protection were injured. Ziegenhagen and Brosnan (1985) draw similar conclusions about the efficacy of armed (although not firearm) resistance when summarizing 13 city victim surveys. Using a multivariate regression analysis, Kleck and DeLone (1993) confirm these basic cross-tabular findings.11 Defense with a firearm is associated with

10  

A number of studies use samples of data collected from crimes reported to police. Police records are presumed to understate resistance in general and defensive gun use in particular (Kleck, 2001a; Kleck and DeLone, 1993). More importantly, these surveys cannot reveal successful forms of resistance that are not reported to the police at all.

11  

The committee is not aware of other multivariate analyses of the effects of resistance with a firearm on crime and injury. Researchers have, however, evaluated the effects of armed resistance. Using data from the NCVS, Kleck and Sayles (1990) conclude that rapes are less likely to be completed if the victim uses armed resistance. Lizotte (1986) draws similar conclusions using data from city victim surveys.

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

fewer completed robberies and less injury. Two forms of self-defense, namely using force without a weapon and trying to get help or attract attention, are associated with higher injury rates than taking no self-protective action.

The results suggest interesting associations: victims who use guns defensively are less likely to be harmed than those using other forms of self-protection. Whether these findings reflect underlying causal relationships or spurious correlations remains uncertain. Much of the existing evidence reports simple bivariate correlations, without controlling for any confounding factors. Kleck and DeLone (1993) rely on multivariate linear regression methods that implicitly assume that firearms use, conditional on observed factors, is statistically independent of the unobserved factors influencing the outcomes, as would be the case in a classical randomized experiment.12 Is this exogenous selection assumption reasonable? Arguably, the decisions to own, carry, and use a firearm for self-defense are very complex, involving both individual and environmental factors that are related to whether a crime is attempted, as well as the outcomes of interest.13 The ability of a person to defend himself or herself, attitudes toward violence and crime, emotional well-being, and neighborhood characteristics may all influence whether a person uses a firearm and the resulting injury and crime. Thus, in general, it is difficult to be confident that the control variables account for the numerous confounding factors that may result in spurious correlations. Furthermore, the committee is not aware of any research that considers whether the finding is robust to a variety of methodological adjustments. Without an established body of research assessing whether the findings are robust to the choice of covariates, functional form, and other modeling assumptions, it is difficult to assess the credibility of the research to date.

The most obvious and fundamental limitation, however, is that the data on defensive gun uses are, as described above, potentially error ridden. Without reliable information on the prevalence of defensive gun use, researchers are forced to make implausible and unsubstantiated assumptions about the accuracy of self-reported measures of resistance. For example, Kleck, one of the most vocal critics of DGU estimates derived from the NCVS, assumes these data are fully accurate when measuring the efficacy of resistance (Kleck, 2001b; Kleck and DeLone, 1993).

12  

Kleck and DeLone (1993) account for basic demographic characteristics of the victim (e.g., race, gender, age, income, and education) and some details on the event (e.g, whether the offender had a gun).

13  

Not only does the potential of unobserved factors create biases of unknown magnitude, but it is also difficult to determine the direction of these biases. If, as suggested by the National Research Council (1993:266), persons who use firearms were better prepared in general to defend against crime, then the estimated associations would be biased upward. In contrast, if firearms are used in more dangerous situations, then the estimated associations would be biased downward (Kleck, 2001b:292).

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

The response problems described above, however, cannot be ignored. To the contrary, these measurement problems may lead to substantial biases in unknown directions. If, for example, respondents are inclined to report being victimized when a crime is “successful” but conceal unsuccessful crimes, the estimated efficacy of resistance will be biased downward. In contrast, if respondents, concerned about being perceived as inept, are inclined to report successful forms of resistance but conceal ineffective forms, the estimated efficacy of self-defense will be biased upward. Without better information on the nature and extent of response problems, it is impossible to know whether and how the estimated associations between defensive gun use, crime, and injury are biased. If, as Kleck and Gertz (1995) suggest, the NCVS misses over 2 million defensive uses per year, then biases caused by reporting errors may be substantial.

Subjective Assessments

Subjective assessments on the efficacy of defensive gun use have been elicited in both the NCVS and the NSDS. Data from the 1994 NCVS, for example, reveal that 65 percent of victims felt that self-defense improved their situation, while 9 percent thought that it worsened their situation (Kleck, 2001a). More direct counterfactual questions were asked in the NSDS survey, in which respondents who reported using a firearm were asked (Kleck and Gertz, 1995:316):

If you had not used a gun for protection in this incident, how likely do you think it is that you or someone else would have been killed? Would you say almost certainly not, probably not, might have, probably would have, or almost certainly would have been killed?

Nearly half of respondents perceived that someone might, probably, or almost certainly would have been killed.

Although intriguing, these assessments are of limited value. Certainly, there are obvious concerns about inaccurate reporting associated with subjective questions. Victims may be inclined to view their actions as effective regardless and may exaggerate counterfactual outcomes. Even if victims report truthfully, the existing questionnaires provide little guidance. What does a respondent mean when he states that someone might have been killed? Are all respondents using consistent criteria to interpret these questions?

Firearms and Fatalities

A number of researchers have attempted to infer the defensive utility of firearms by examining the firearms deaths that occur in or near the victim’s

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

home. Both Kellermann and Reay (1986) and Rushforth et al. (1974) compare fatalities caused by self-defense and other motivations. Both studies find that people using guns in self-defense account for a small fraction of fatalities in the home. Kellermann and Reay find that there were nearly 5 times as many homicides and 37 times as many suicides as perpetrators killed in self-defense. They go on to conclude, “The advisability of keeping a firearm in the home for protection must be questioned.” Rushforth et al. (1974) found similar results and drew similar conclusions.

Although the facts are in no doubt, the conclusions do not seem to follow. Certainly, effective defensive gun use need not ever lead the perpetrator to be wounded or killed. Rather, to assess the benefits of self-defense, one needs to measure crime and injury averted. The particular outcome of an offender is of little relevance. It might be, as Kleck (2001b) suggests, that the ratio of firearm-caused fatalities to fatalities averted because of defensive gun use is a more relevant comparison. Answering this question, however, requires researchers to address the fundamental counterfactual questions regarding the effects of both defensive and offensive uses of firearms that have been the subject of much of this report and have generally proved to be elusive. Simple death counts cannot answer these complex questions.

Case-control sampling schemes matching homicide victims to non-victims with similar characteristics have also been used to infer whether owning a firearm is a risk factor for homicide and the utility of firearms for self-defense (see Chapter 7 for a discussion of the case-control methodology). Kellermann et al. (1993) found that persons who had a firearm in the home were at a greater risk for homicide in their home than persons who did not have a firearm (adjusted odds ratio of 2.7). Cummings et al. (1997) found that persons who purchased a handgun were at greater risk for homicide than their counterparts who had no such history (adjusted odds ratio of 2.2).

In light of these findings, Kellermann et al. (1993) ultimately conclude that owning firearms for personal protection is “counterproductive,” (p. 1087) and that “people should be strongly discouraged from keeping guns in the home” (p. 1090). This conclusion rests on the implicit assumption that the decision to own a firearm is random or exogenous with respect to homicide in the home (after controlling for various observed factors, including whether a household member has been hurt in a fight, has been arrested, or has used illicit drugs). Cummings and his colleagues (1997) do not draw such strong causal conclusions, but instead simply describe the observed positive association between firearms and homicide.

In the committee’s view, the exogenous selection assumption and the resulting conclusions are not tenable. While these observed associations between firearms ownership and homicide may be of interest, they do little to reveal the impact of firearms on homicide or the utility of firearms for

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×

self-defense. As noted by the authors, even small degrees of misreporting on ownership by either the cases or the controls can create substantial biases in the estimated risk factors (see Kleck, 1997, for an illustration of these biases). A more fundamental inferential problem arises from the fact that ownership is not likely to be random with respect to homicide or other forms of victimization. To the contrary, the decision to own a firearm may be directly related to the likelihood of being victimized. People may, for instance, acquire firearms in response to specific or perceived threats, and owners may be more or less psychologically prone toward violence. Thus, while the observed associations may reflect a causal albeit unspecified path-way, they may also be entirely spurious. As Kellermann and his colleagues note (1993:1089), “it is possible that reverse causation accounted for some of the association we observed between gun ownership and homicide.”

Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 102
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 103
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 104
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 105
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 106
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 107
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 108
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 109
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 110
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 111
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 112
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 113
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 114
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 115
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 116
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 117
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 118
Suggested Citation:"5 The Use of Firearms to Defend Against Criminals." National Research Council. 2005. Firearms and Violence: A Critical Review. Washington, DC: The National Academies Press. doi: 10.17226/10881.
×
Page 119
Next: 6 Right-to-Carry Laws »
Firearms and Violence: A Critical Review Get This Book
×
Buy Hardback | $54.95 Buy Ebook | $43.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

For years proposals for gun control and the ownership of firearms have been among the most contentious issues in American politics. For public authorities to make reasonable decisions on these matters, they must take into account facts about the relationship between guns and violence as well as conflicting constitutional claims and divided public opinion. In performing these tasks, legislators need adequate data and research to judge both the effects of firearms on violence and the effects of different violence control policies.

Readers of the research literature on firearms may sometimes find themselves unable to distinguish scholarship from advocacy. Given the importance of this issue, there is a pressing need for a clear and unbiased assessment of the existing portfolio of data and research. Firearms and Violence uses conventional standards of science to examine three major themes - firearms and violence, the quality of research, and the quality of data available. The book assesses the strengths and limitations of current databases, examining current research studies on firearm use and the efforts to reduce unjustified firearm use and suggests ways in which they can be improved.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!