National Academies Press: OpenBook

The Practicality of Pulsed Fast Neutron Transmission Spectroscopy for Aviation Security (1999)

Chapter: 3 Laboratory Tests of Pulsed Fast Neutron Transmission Spectroscopy

« Previous: 2 Principle of Bulk Explosives Detection
Suggested Citation:"3 Laboratory Tests of Pulsed Fast Neutron Transmission Spectroscopy." National Research Council. 1999. The Practicality of Pulsed Fast Neutron Transmission Spectroscopy for Aviation Security. Washington, DC: The National Academies Press. doi: 10.17226/6469.
×

3
Laboratory Tests of Pulsed Fast Neutron Transmission Spectroscopy

PFNTS technology was tested for its capability to meet both the Pd (probability of detection) and the probability of a false alarm (Pfa) specified in the EDS certification standard for the range of explosive classes. The FAA conducted blind tests on two PFNTS-based explosives-detection devices, one developed by the University of Oregon and the other by Tensor Technology. The blind tests measured detection performance (coupled Pd and Pfa) but not bag throughput. The results described below can be used to set the demonstrated PFNTS detection performance level.

University of Oregon Blind Tests

The FAA conducted a series of blind tests (Chmelik et al., 1997) at the University of Oregon in September 1996 using the PFNTS B-matrix approach (Lefevre and Overley, 1998) and two spatial correlation algorithms, a contiguous-pixel test and a shape test. The software used to implement the B-matrix detection algorithm along with the blind test database have been archived (Lefevre, 1998). The University of Oregon detection system examined 3.2 x 3.2 cm2 (1.3 x 1.3 in.2) pixels of a suitcase. The test involved 134 suitcases and eight different nitrogen-based explosives. The FAA tester placed explosives in 75 of the bags. In six instances, two or more different explosives were placed in a single bag. Attempts were made to elicit false alarms by placing a variety of available materials in the bags. No attempt was made to combine materials along the neutron path to reproduce the elemental densities or ratios expected of explosives in cluttered bags. Attempts were made to conceal explosives by placing them in iron or aluminum pipes, behind books, wood, and other objects, and in radios or video cassettes.

Attempts to conceal explosives were mostly unsuccessful. In the initial blind test, using the automated contiguous-pixel test, 67 of the 75 explosive-containing bags were correctly identified, for a Pd of 89.3 percent. Eight of the 59 bags that did not contain explosives did set off an alarm, for a Pfa of 13.6 percent. Using the automated shape test, the presence of explosives in 70 of the 75 explosive-containing bags was detected, for a Pd of 93 percent; seven of the 59 bags that did not contain explosives set off alarms, for a Pfa of 11.8 percent. With operator intervention, the shape test operator, relying primarily on past experience, correctly identified 71 explosive-containing bags out of 75. However, in this test only 46 of the 59 bags that did not contain explosives were correctly identified. These results correspond to a Pd of 94.7 percent and a Pfa of 22 percent. Blind tests conducted at the University of Oregon, unlike EDS certification tests, distinguished between "true detections" and "false detections," that is, false alarms were registered if the explosive was not located in the portion of the bag identified by the algorithm. None of the 59 benign bags registered a false alarm. Most of the false alarms were produced by large areas of only slightly elevated explosive probabilities, which corresponded to the location of large books or blocks of wood. Table 3-1 summarizes the detection performance for the various algorithms used during the blind tests.

In four of the six cases where multiple explosives were placed in the bag, the system was able to detect the presence of each explosive. In the other two cases involving multiple explosives, the explosives were in close proximity and were identified as only one explosive. In 38 of the 75 explosives-

TABLE 3-1 Performance of the University of Oregon Explosives-Detection Algorithm in Blind Tests

Detection Algorithm

Pda

Pfa

Contiguous pixel

89.3%

13.6%

Shape test

93%

11.8%

Operator intervention

94.7%

22%

Post-test algorithm adjustment

93.3%

4.5%

a Pd indicates a "true detection," the correct identification of the region of the bag containing the explosive.

Suggested Citation:"3 Laboratory Tests of Pulsed Fast Neutron Transmission Spectroscopy." National Research Council. 1999. The Practicality of Pulsed Fast Neutron Transmission Spectroscopy for Aviation Security. Washington, DC: The National Academies Press. doi: 10.17226/6469.
×

TABLE 3-2 Performance of the Tensor Explosives-Detection Algorithm in Blind Tests

Detection algorithm

Explosives Classa

Pd

Pfa

Operator assisted

B

94.3%

25%

Operator assisted

A

51.8%

25%

Operator assisted

A-1

83.3%

25%

Operator assisted

A-2

43.2%

25%

Automated neural net (with 10 scans eliminated because of possible interference)

all classes

88%

24%

Automated neural net detection on loose cargo when trained on bags

all classes

70%

see note

Automated neural net detection on loose cargo when trained on bags

A

40%

see note

a Explosive classes A-1 and A-2 are subclasses of explosive class A.

Note: Although no false alarms were recorded, only 4 of the 60 cargo scans did not contain an explosive. Therefore, Pfa is not statistically meaningful in this case.

containing bags, the PFNTS detection algorithm not only indicated the presence of an explosive but also successfully identified the explosives class. In another 16 cases, the actual test explosive corresponded with the second most likely explosives class predicted by the detection algorithm.

When the blind test pixel data were used to modify the B-matrix method, the error rate was significantly reduced. This post-processed blind test analysis missed five out of the 75 explosives and produced no false alarms in the 59 benign bags. If the six cases where the wrong region of the bag was identified as containing an explosive (when in fact there was an explosive in a different region of the bag) were considered to be false alarms, the Pd was 93.3 percent, and the Pfa was 4.5 percent (Algorithm 4 in Table 3-1). The performance for this algorithm must be treated as an indicator of the "potential" detection performance (Pd coupled with Pfa) for the PFNTS and not as a valid blind test result because post-processing of blind test data using detection algorithms optimized to the test data can produce misleading results.

The explosives the University of Oregon expected would be tested, which were specified in the "FNTS Developmental Test Plan," had a defined configuration. However, different than expected explosive configurations were used in the actual blind tests. The test final report (Overley, 1998) indicates that seven out of the eight missed explosives involved smaller amounts of explosive than the University of Oregon researchers expected to detect.

Tensor Technology Blind Tests

In blind tests at Tensor Technology, Inc., in late September 1997, 150 suitcases were scanned at two angles, 0° (broadside) and 60° in azimuth (Gibson et al., 1997; Tensor Technology, 1998a). The initial determination was performed by an operator and was "somewhat subjective" but was "based on a combination of explosive size, general level of attenuation, atomic number density analysis, and the neural net analysis." Table 3-2 shows the Tensor detection performance. The Pd was 94.3 percent for Class B explosives and 51.8 percent for Class A explosives (83.33 percent for Class A-1 and 43.18 percent for Class A-2). The Pfa was 25 percent. There was no significant difference in detection performance between the two scan angles.

An automated explosive detection algorithm is under development using the regression neural net analysis. A preliminary version incorporating the neutron attenuation magnitude and the nitrogen content was used to analyze the blind test scans. The details of the blind test results from this automated algorithm are not clear from Tensor's final report to the FAA (Tensor Technology, 1998b). However, the report suggests that the operator response was somewhat better than the initial automated response. Some questions have been raised about 10 test scans in which, because of attempts to limit interference from a table used to hold the test articles, the explosives may not have been in the detector array's field of view. If these 10 scans are omitted, the automated response algorithm had an overall Pd (for Class A and Class B) of 88 percent and a Pfa of 24 percent.

The Tensor blind tests included tests of the system's potential for screening loose cargo consisting of boxes with varied contents. Thirty loose cargo items were scanned twice each for a total of 60 scans. Explosives appeared 56 times during the testing, including four cases in which two explosives were detected in a single cargo item. A neural network detection algorithm trained on data acquired from luggage (not loose cargo) was used. The overall Pd was 70 percent. The Pd for Class A explosives was only 40 percent. No false alarms were recorded during the cargo scanning.

Some of the loose cargo containers were fairly dense. Six of the 17 missed explosives were in containers with a neutron transmission fraction of less than 0.003, which is comparable to noise levels of background scattered neutrons during screening. In other words, transmitted neutrons could not be differentiated from background noise caused by scattered neutrons at this low transmission level.

Detection of Class a Explosives

The PFNTS method has problems detecting Class A explosives (required by the FAA for certification). This may

Suggested Citation:"3 Laboratory Tests of Pulsed Fast Neutron Transmission Spectroscopy." National Research Council. 1999. The Practicality of Pulsed Fast Neutron Transmission Spectroscopy for Aviation Security. Washington, DC: The National Academies Press. doi: 10.17226/6469.
×

be an intrinsic limitation for explosives-detection approaches that use a large pixel size (> 0.2 cm [> 0.08 in.]). Figures 3-1 and 3-2 show the resolution available to the detection algorithms used in the blind tests. Except for Class A explosives, the PFNTS detection performance (Pfa and Pd) has the potential to meet the FAA's EDS certification requirements. Because the Tensor and University of Oregon tests used very different detection algorithms and because both provided good detection levels for Class B explosives, the PFNTS elemental densities seem to provide a very robust set of measurements for explosives detection. Refinements in the detection algorithm or coupling with other technologies may be necessary, however, to overcome the limitations of PFNTS for the detection of Class A explosives. Once this limitation has been better quantified, it should be possible to determine the potential role for PFNTS in commercial aviation security.

The unreliable detection of Class A explosives is a serious deficiency of PFNTS, particularly for Class A-1 explosives. However, according to reports by the University of Oregon (Overley, 1998), some subclasses of Class A explosives can be reliably detected. Approaches to improving the capability of PFNTS to detect Class A explosives are listed below:

  • reducing the pixel dimensions at the bag location
  • using multiple scans at different angles and a tomographic analysis of the data
  • improving spatial correlation algorithms so that large areas with a low probability of being an explosive are identified as an explosive

Determining the efficacy of these approaches will require further laboratory testing. Some of these approaches (e.g., tomographic analysis) will reduce the bag throughput rate compared to a single-view radiographic method.

FIGURE 3-1 

Neural net values during Tensor blind testing for a  slurry sample at an angle in a suitcase. 

Source: Tensor Technology, 1998a.

FIGURE 3-2 

Gray-scale maps from B-matrix during University  of Oregon blind tests of a bag containing an explosive  in an iron pipe sloping up to the right. 

Source: University of Oregon.

Assessment of Detection Performance

The post-processing of blind test data using detection algorithms optimized to the test data can produce misleading results. For example, if the detection algorithm is optimized to specific test data, it is possible that when new data are analyzed (i.e., from a new set of baggage), the algorithm will not perform as well. Although the 4.5 percent Pfa and 93.3 percent Pd described above for post-processed data from the University of Oregon PFNTS tests indicate a detection performance level that may exceed the explosives-detection potential of x-ray-based CT approaches, these data must be treated with a healthy skepticism. Furthermore, the low Pfa attained during the University of Oregon tests—with a 16-element linear array detector—may not hold for a full two-dimensional array. The University of Oregon acknowledges that changing to a two-dimensional array detector could degrade the performance of PFNTS for the following reasons (Lefevre, 1998):

  • As the detector geometry is expanded, the background noise level becomes much more severe (i.e., more scattered neutrons enter the scanning area).
Suggested Citation:"3 Laboratory Tests of Pulsed Fast Neutron Transmission Spectroscopy." National Research Council. 1999. The Practicality of Pulsed Fast Neutron Transmission Spectroscopy for Aviation Security. Washington, DC: The National Academies Press. doi: 10.17226/6469.
×
  • More collimator area is visible to each detector, which affects the line shapes for monoenergetic neutrons by increasing long flight-time tails.
  • Neutron in-scattering from luggage items increases.
  • Detector cross talk increases.

For all of these reasons, the panel cannot confidently state that the PFNTS system has the potential for a very low false alarm rate under conditions that meet the FAA's required throughput rate. The problems listed above will have to be addressed through more laboratory experimentation or detailed radiation transport modeling. The Tensor two-dimensional 99-element array detector exhibited a much higher false alarm rate during blind testing (compared to the University of Oregon tests), which may be associated with the background from a different neutron source configuration or the use of a different explosives-detection algorithm rather than intrinsic characteristics of the two-dimensional detector array.

Suggested Citation:"3 Laboratory Tests of Pulsed Fast Neutron Transmission Spectroscopy." National Research Council. 1999. The Practicality of Pulsed Fast Neutron Transmission Spectroscopy for Aviation Security. Washington, DC: The National Academies Press. doi: 10.17226/6469.
×
Page 13
Suggested Citation:"3 Laboratory Tests of Pulsed Fast Neutron Transmission Spectroscopy." National Research Council. 1999. The Practicality of Pulsed Fast Neutron Transmission Spectroscopy for Aviation Security. Washington, DC: The National Academies Press. doi: 10.17226/6469.
×
Page 14
Suggested Citation:"3 Laboratory Tests of Pulsed Fast Neutron Transmission Spectroscopy." National Research Council. 1999. The Practicality of Pulsed Fast Neutron Transmission Spectroscopy for Aviation Security. Washington, DC: The National Academies Press. doi: 10.17226/6469.
×
Page 15
Suggested Citation:"3 Laboratory Tests of Pulsed Fast Neutron Transmission Spectroscopy." National Research Council. 1999. The Practicality of Pulsed Fast Neutron Transmission Spectroscopy for Aviation Security. Washington, DC: The National Academies Press. doi: 10.17226/6469.
×
Page 16
Next: 4 Baseline Characteristics of Explosives-Detection Systems Based on X-Ray-Computed Tomography »
The Practicality of Pulsed Fast Neutron Transmission Spectroscopy for Aviation Security Get This Book
×
Buy Paperback | $47.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

A major goal of the Federal Aviation Administration (FAA), and now the Transportation Security Administration (TSA), is the development of technologies for detecting explosives and illegal drugs in freight cargo and passenger luggage. One such technology is pulsed fast neutron analysis (PFNA). This technology is based on detection of signature radiation (gamma rays) induced in material scanned by a beam of neutrons. While PFNA may have the potential to meet TSA goals, it has many limitations. Because of these issues, the government asked the National Research Council to evaluate the potential of PFNA for airport use and compare it with current and future x-ray technology. The results of this survey are presented in "Assessment of the Practicality of Pulsed Fast Neutron Analysis for Aviation Security."

A broad range of detection methods and test results are covered in this report. Tests conducted as of October 2000 showed that the PFNA system was unable to meet the stringent federal aviation requirements for explosive detection in air cargo containers. PFNA systems did, however, demonstrate some superior characteristics compared to existing x-ray systems in detecting explosives in cargo containers, though neither system performed entirely satisfactorily. Substantial improvements are needed in the PFNA detection algorithms to allow it to meet aviation detection standards for explosives in cargo and passenger baggage.

The PFNA system currently requires a long scan time (an average of 90 minutes per container in the prototype testing in October 2000), needs considerable radiation shielding, is significantly larger than current x-ray systems, and has high implementation costs. These factors are likely to limit installation at airports, even if the detection capability is improved. Nevertheless, because PFNA has the best potential of any known technology for detecting explosives in cargo and luggage, this book discusses how continued research to improve detection capabilities and system design can best be applied for the airport environment.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!