Executive Summary

Since the late 1980s computerized imaging technology has been used to assist forensic firearms examiners in finding potential links between images of ballistics evidence gathered from crime scene investigations, namely, cartridge cases and bullets from fired guns. To support this effort, the Bureau of Alcohol, Tobacco, Firearms, and Explosives (ATF) in 1997 formed the National Integrated Ballistic Information Network (NIBIN). Law enforcement agencies participating in NIBIN contribute to a database of images of bullet and cartridge case evidence recovered from (or test-fired from weapons linked to) crime scenes. This system facilitates rapid comparison with archived evidence and with evidence gathered at other crime sites; when matches look promising, the physical evidence can be retrieved for direct examination and confirmation by an examiner. NIBIN was designed as a tool for search, not for verification, which is always done by an examiner.

The rapid development of computerized ballistic imaging technology has led to speculation about its future potential. A particularly interesting proposal is to create a national reference ballistic image database (RBID) that would house images from firings of all newly manufactured or imported firearms. Proponents of this proposal argue that such a database could provide a quick investigative lead from evidence recovered at a crime scene to the underlying firearm’s original point of sale. State RBIDs already exist in Maryland and New York, and wide attention was drawn to the issue when California studied the feasibility of creating its own RBID.

In 2004 the National Institute of Justice (NIJ) of the U.S. Department of Justice requested that the National Academies appoint a committee of

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Executive Summary Since the late 1980s computerized imaging technology has been used  to  assist  forensic  firearms  examiners  in  finding  potential  links  between  images  of  ballistics  evidence  gathered  from  crime  scene  investigations,  namely, cartridge cases and bullets from fired guns. To support this effort,  the Bureau of Alcohol, Tobacco, Firearms, and Explosives (ATF) in 1997  formed  the  National  Integrated  Ballistic  Information  Network  (NIBIN).  Law enforcement agencies participating in NIBIN contribute to a database  of  images  of  bullet  and  cartridge  case  evidence  recovered  from  (or  test- fired  from  weapons  linked  to)  crime  scenes.  This  system  facilitates  rapid  comparison  with  archived  evidence  and  with  evidence  gathered  at  other  crime  sites;  when  matches  look  promising,  the  physical  evidence  can  be  retrieved for direct examination and confirmation by an examiner. NIBIN  was designed as a tool for search, not for verification, which is always done  by an examiner.  The  rapid  development  of  computerized  ballistic  imaging  technology  has  led  to  speculation  about  its  future  potential.  A  particularly  interest- ing  proposal  is  to  create  a  national  reference  ballistic  image  database  (RBID) that would house images from firings of all newly manufactured or  imported firearms. Proponents of this proposal argue that such a database  could provide a quick investigative lead from evidence recovered at a crime  scene to the underlying firearm’s original point of sale. State RBIDs already  exist  in  Maryland  and  New  York,  and  wide  attention  was  drawn  to  the  issue when California studied the feasibility of creating its own RBID.  In 2004 the National Institute of Justice (NIJ) of the U.S. Department  of Justice requested that the National Academies appoint a committee of  

OCR for page 1
 BALLISTIC IMAGING experts to address the issues raised by the computerized ballistic imaging  technology. The Committee to Assess the Feasibility, Accuracy, and Tech- nical Capability of a National Ballistics Database was asked to “assess the  feasibility, accuracy and reliability, and technical capability of developing  and  using  a  national  ballistics  database  as  an  aid  to  criminal  investiga- tions.” To accomplish this, the panel’s charge is to:   (1)   ssess the technical feasibility, through analysis of the uniqueness  A of ballistic images, the ability of imaging systems to capture unique char- acteristics and to parameterize them, the algorithmic and computational  challenges of an imaging database, the reproducibility of ballistic impres- sions and the ability of imaging systems to extract reproducible informa- tion from ballistic impressions.    (2)   ssess the statistical probabilities that ballistics evidence presented  A would lead to a match with images captured in a database, whether and  how the base rate can be estimated for those crimes that present bullet or  casing evidence that do in fact come from a gun that produced a database  entry, and the probabilities and consequences of false positives and false  negatives.    (3)   ssess  the  operational  utility  of  ballistics  evidence  in  criminal  A investigations—that  is  the  extent  to  which  it  is  used  or  can  be  used  to  identify crime guns and suspects and to solve specific crimes.   (4)   ssess  the  sources  of  error  in  ballistics  database  matching  (from  A examination, digitization, computer matching, chain of custody and docu- mentation of tests, and expert confirmation), how they may be quantified,  and how these errors interact. The charge continues:  The committee’s work will provide scientific and technical knowledge to  inform the government’s deliberations on three policy options with regard  to ballistics databases:   (1)   aintain  the  National  Integrated  Ballistic  Information  Network  M (NIBIN)  on  ballistics  recovered  from  crime  scenes.  It  is  operated  by  the  Bureau of Alcohol, Tobacco, and Firearms.   (2)   nhance the NIBIN system so that it can be used to match crime  E scene evidence with the gun used.    (3)   stablish a national ballistics database of images from bullets fired  E from all, or nearly all, newly manufactured or imported guns for the pur- pose of matching ballistics from a crime scene to a gun and information  on its initial owner. Addressing  the  issues  raised  by  the  tasks  of  the  charge  permitted  the  committee to provide guidance to NIJ on the three federal policy options.  Specifically,  for  option  2,  enhancing  the  NIBIN  system,  we  address  how 

OCR for page 1
 EXECUTIVE SUMMARY to  increase  its  effectiveness  as  a  search  tool,  including  changes  to  the  basic imaging standard used by the system, and improving procedures for  working with the existing hardware and software. For option 3, establish- ing a national RBID, the committee considers it a counterpart to NIBIN,  containing  images  of  ballistic  samples  from  all  newly  manufactured  and  imported weapons. The committee also considered the feasibility of alter- native technologies that could achieve the same goal as a national RBID.  These alternative technologies include microstamping to imprint a known,  unique  marker  on  firearms  parts  or  ammunition:  analysis  of  such  marks  would  complement  or  perhaps  replace  the  need  to  examine  the  currently  used toolmarks.  Underlying  the  specific  tasks  with  which  the  committee  was  charged  is the question of whether firearms-related toolmarks  are  unique:  that  is,  whether  a  particular  set  of  toolmarks  can  be  shown  to  come  from  one  weapon to the exclusion of all others. Very early in its work the committee  found that this question cannot now be definitively answered. Finding: The validity of the fundamental assumptions of uniqueness and reproducibility of firearms-related toolmarks has not yet been fully demonstrated. Notwithstanding  this  finding,  we  accept  a  minimal  baseline  standard  regarding ballistics evidence. Although they are subject to numerous sources  of  variability,  firearms-related  toolmarks  are  not  completely  random  and  volatile; one can find similar marks on bullets and cartridge cases from the  same gun.  A  significant  amount  of  research  would  be  needed  to  scientifically  determine  the  degree  to  which  firearms-related  toolmarks  are  unique  or  even  to  quantitatively  characterize  the  probability  of  uniqueness.  Assess- ing uniqueness at, say, a submicroscopic level, though probably technically  possible, would be extremely difficult and time consuming compared with  less  definitive  but  more  practical  and  generally  available  methods  at  the  macroscopic level. It is an issue of policy and of economics as to whether  doing  so  would  be  worthwhile.  The  committee  did  not  and  could  not  undertake such research, nor does it offer any conclusions about undertak- ing such research. Although it appears to the committee that the needs for  research are extensive, specifying the nature of that research was not part of  the committee’s charge. We also note that the committee does not provide  an overall assessment of firearms identification as a discipline nor does it  advise  on  the  admissibility  of  firearms-related  toolmark  evidence  in  legal  proceedings: these topics are not within its charge.  The  committee’s  charge  is  to  determine  the  extent  to  which  the  tool- marks  left  on  bullets  and  cartridge  casings  after  firing  a  weapon  can  be 

OCR for page 1
 BALLISTIC IMAGING captured by imaging technology. It is also to assess whether a ballistic image  database—particularly a national RBID containing images of exhibits fired  from  all  newly  manufactured  and  imported  guns—would  be  feasible  and  operationally  useful,  by  which  we  mean  capable  of  generating  leads  for  f   ollow-up and further investigation. Whether or not toolmarks are unique  to  a  given  weapon  does  not  preclude  the  committee  from  addressing  this  charge. Indeed, in many situations a sufficient level of toolmark reproduc- ibility can be picked up by imaging or other measurement systems to be use- ful for narrowing a search down to a set of possible weapons, as is currently  done. The final determination of a “match” is made by a human examiner. FEASIbILITy OF A NATIONAL REFERENCE bALLISTIC IMAgE DATAbASE Independent of the reliability and effectiveness of the technology used  in making comparisons of images in a national RBID, there would be sig- nificant limitations in the usefulness of such a database. Most importantly,  there is a huge existing supply of weapons and ammunition that would not  be entered into the database. In addition, revolvers do not eject cartridge  cases at crime scenes as do other handguns. Consequently, even under the  best of circumstances, when random variability is kept to a minimum, the  database itself would be incomplete. Finally, to implement a national RBID,  national protocols would have to be created for the test firing of new and  imported  guns;  ensuring  that  test-fired  cartridge  cases  or  bullets  are  cor- rectly packaged with their corresponding firearm and maintaining a chain  of custody for the exhibits after they are imaged would create a formidable  logistical challenge. In our detailed assessment, three additional points regarding the imple- mentation of a national RBID have particular salience.  First, the current technology in use for automated toolmark compari- son,  based  on  two-dimensional  greyscale  images,  is  useful  for  gross  cat- egorization and sorting of large quantities of evidence. However, it is less  reliable  for  distinguishing  extremely  fine  individual  marks  that  would  be  necessary to make successful matches in RBIDs in which large numbers of  exhibits on file would share gross class and subclass characteristics. Second,  basic  probability  calculations  under  reasonable  assumptions  suggest  that  the  process  of  identifying  a  subset  of  possible  matches  that  contains the true match with a specified level of certainty depends critically  on as-yet underived measures of similarity between and within gun types.  This process is very likely to return too large a subset of candidates to be  practically useful for investigative purposes. Third, the large influence of ammunition type and variability introduces  a significant source of error in identification. A standard, protocol type of 

OCR for page 1
 EXECUTIVE SUMMARY ammunition could be specified in an RBID (as it is in NIBIN), but it is likely  not to correspond with the ammunition actually used in a crime; the choice  of protocol ammunition, or a requirement to use multiple ammunition types,  would have significant financial implications for both ammunition and fire- arm manufacturers, as well as on the information systems involved. Conclusion: A national reference ballistic image database of all new and imported guns is not advisable at this time. MAINTAIN OR ENHANCE NIbIN By  facilitating  access  by  state  and  local  law  enforcement  agencies  to  ballistic imaging technology, the NIBIN program provides a valuable ser- vice in helping to solve gun-related crimes. However, agencies differ in the  degree to which they use the NIBIN resources and, consequently, they differ  markedly in the benefits they derive in establishing links between crimes and  investigative leads. The committee’s principal task includes offering guid- ance on either maintaining NIBIN as it currently operates or enhancing it in  various ways to improve its effectiveness. The former is not really a viable  option:  there  are  always  opportunities  for  improvement  in  any  program,  particularly one as broad as NIBIN.  Conclusion: NIbIN can and should be made more effective through operational and technological improvements. To  this  end,  the  committee  offers  15  specific  recommendations  to  improve NIBIN’s performance and effectiveness. Seven of the recommenda- tions are oriented principally at the operation of the NIBIN program itself  and the practices of NIBIN partner agencies, and they address: •  priority for NIBIN entry of cartridge casings collected from crime  scenes;  •  ballistic imaging as a part of the criminal investigation process for  state and local agencies;  •  cross-jurisdictional tally of hits using the NIBIN system;  •  streamlining of the ballistic image acquisition process and reporting  requirements; •  development of “best practices” in using NIBIN; •  a protocol for the entry of more than one exhibit from the same  crime scene or test firing; and •  allocation of NIBIN system technology.

OCR for page 1
 BALLISTIC IMAGING We also offer eight specific recommendations for enhancing the current  technical platform for the NIBIN program, the Integrated Ballistics Identifi- cation System (IBIS), and the hardware and software system developed by  Forensic Technology WAI, Inc. These eight recommendations address: •  research on the distributions of comparison scores; •  an  “audit  trail”  in  the  NIBIN  system’s  hardware  and  software  systems; •  ammunition brand information in NIBIN; •  the  capacity  for  national  or  cross-regional  searches  against  the  NIBIN database; •  NIBIN’s database partition structure; •  enhancements to the NIBIN interface; •  side-light imagery of breech face impressions; and •  the 20 percent threshold used in the IBIS. In support of this study, the National Institute of Standards and Tech- nology (NIST) was separately contracted by NIJ to perform experimental  work  at  the  committee’s  request.  This  experimental  work  considered  the  value of one major technical enhancement to the current NIBIN system: a  change in imaging standard from two-dimensional, greyscale photography  to  three-dimensional  surface  measurement  using  noncontact  microscopy.  NIST’s  work  included  analysis  of  an  extract  of  cartridges  from  one  of  the  major  existing  studies  of  ballistic  imaging  performance  as  well  as  a  new dataset of test-fired cartridges designed by the committee. The work  highlights  the  promise  of  three-dimensional  surface  measurement,  which  performs comparably with—and, for some cartridge markings, often better  than—the current two-dimensional methodology. However, there are major  substantive challenges—among them the reduction of data collection time  and  the  refinement  of  image  comparison  algorithms  that  make  use  of  three-dimensional information but are still compatible with existing two- d   imensional imagery—that need to be addressed before full consideration  can be given to adopting the new standard. ALTERNATIvE TECHNOLOgIES The goal of a national reference ballistic image database is to provide  an  investigative  link  from  ballistics  evidence  to  the  point  of  sale  of  the  weapon or ammunition used in a crime. The same goal could be achieved  through an entirely different approach, microstamping, which is to place a  known, unique, and unalterable identifier on gun parts, cartridge cases, or  bullets at the time of manufacture. These uniquely microstamped products  could then be associated with their purchaser when sold. Microstamping, if 

OCR for page 1
 EXECUTIVE SUMMARY feasible and practical, would have the advantage of imposing uniqueness as  a characteristic of ballistics evidence, substituting known and fixed mark- ings for microscopically fine, individualizing characteristics that result from  random processes in manufacture and weapon firing.  A  distinct  advantage  of  microstamping  is  that  the  marks  could  be  examined at a crime scene using equipment no more sophisticated than a  magnifying glass, vastly simplifying and speeding up the process of devel- oping investigative leads. The state of California recently passed a law, to  take effect in 2010, which requires microstamping on internal parts of new  semiautomatic  pistols.  However,  the  committee  believes  that  for  such  a  technology to be implemented successfully, in-depth investigations on sev- eral topics are needed. These topics include the reliability and durability of  the marks in a variety of firing conditions, their susceptibility to tampering  and countermeasures, whether it would be best to place them on guns or  ammunition or both, and the cost implications and feasibility of adding a  microstamping process to established manufacturing processes.  PROCESS FOR IMPROvINg COMPuTER-ASSISTED FIREARMS IDENTIFICATION The  current  technology  used  in  automated  examination  of  images  of  ballistics  evidence  is  produced  and  maintained  by  a  single  vendor.  As  a  result,  it  does  not  benefit  from  the  improvements  that  could  be  gained  through competition and vetting among the broader research community,  and its potential for advancement and innovation is limited. The committee  suggests that improvements in matching ballistics evidence be made through  government procurement efforts that demonstrate best practices.  Two  recent  examples  of  government-mandated  large-scale  imaging  system  developments  based  on  initially,  nonmature  technologies  include  s   ystems for fingerprint identification and for facial recognition. Both sys- tems  required  the  creation  of  dedicated  pattern  recognition  algorithms,  similar to the requirements of NIBIN. Instead of relying on a single system  produced by a single vendor, both systems were organized as competitions  between vendors with the goal of advancing the technology as quickly as  possible. Both competitions required that well-vetted datasets from several  sources be made available to researchers so that the correct features could  be identified for extraction. Finally, the results of both competitions were  subjected  to  independently  administered  evaluations,  using  well-defined  and  published  evaluation  methodologies  that  allowed  for  a  direct  quan- titative  assessment  of  the  relative  strengths  and  weaknesses  of  different  approaches. This  approach  to  procurement—removing  strict  dependence  on  a  sole-source provider and ensuring government ownership of and access to 

OCR for page 1
 BALLISTIC IMAGING result data—should be applied for all work related to the improvement in  b   allistics  evidence  analysis,  including  large-scale  two-dimensional  image  search  engines,  three-dimensional  topographical  techniques,  and  micro- stamping processes.