4
Baseline Characteristics of Explosives-Detection Systems Based on X-ray-Computed Tomography

A legitimate comparison of x-ray CT-based EDSs and PFNTS-based explosives-detection techniques must be based on their baseline performance. Laboratory test results for PFNTS were presented in Chapter 3. In this chapter, both the laboratory performance and the operational performance of the FAA-certified CTX-5000 SP are reviewed. To date, most of the performance data on deployed explosives-detection equipment has been generated from tests conducted at the FAA Technical Center. However, operational (field) test data on false alarm rates are also reviewed in this chapter.

Test Data from the FAA Technical Center

Most of the performance data available for x-ray CT-based EDSs are from certification tests of the InVision CTX-5000 SP in 1996. Data from the certification tests of the recently certified InVision CTX-5500 DS and L3 Communications 3DX-6000 are not available yet but should be included in future analyses. The InVision CTX-5500 DS has been certified for two different inspection modes: SURE98 Mode, which has a lower false alarm rate than the CTX-5000 SP but a lower throughput rate, and CERT98 Mode, which has a similar false alarm rate to the CTX-5000 SP but a much higher throughput rate. The panel's analysis of performance data focuses on the CTX-5000 SP, although performance data for the CTX-5500 DS are also presented for reference. Table 4-1 shows the performance factors for the CTX-5000 SP and CTX-5500 DS, including the Pd, Pfd, and the bag throughput rate. Because the actual Pd and Pfa numbers are classified,1 Pd is given as a percentage of the overall Pd required for certification (X), and Pfa is given as a percentage of the Pfa required for certification (Y).

Operational Demonstration

In 1995, the FAA initiated the Airport Operational Demonstration Project (FAA, 1995) to determine the operational performance of the InVision CTX-5000 SP in the field as compared to its performance in certification tests. Three test sites were selected for the project: San Francisco International Airport (United Airlines); Atlanta Hartsfield International Airport (Delta Airlines); and Manila International Airport (Northwest Airlines).

TABLE 4-1 Performance Test Results for the InVision CTX-5000 SP and CTX-5500 DS

Equipment

Detection Rate

False Alarm Rate

Throughput (bags/hour)

 

Sample Size

Pd(% X)a

Sample Size

Pfa(% Y)a

 

CTX-5000 SP

600b

106

1,000c

90

245

CTX-5500 DS/SURE98 mode

600b

106

1,000c

60

264

CTX-5500 DS/CERT98 mode

600b

100

1,000c

95

362

a Percentage of the classified value required for certification.

b A total of 150 bags were used containing improvised explosive devices (25 samples of six explosives, detonators, timers, and wires) and four different orientations to get a sample size of 600.

c The 1,000 bags used to acquire the Pfa differed from the 600 bags used to acquire the Pd. The 1,000-bag set was also used to obtain throughput rates.

1  

The actual values required for certification are recorded in classified FAA documents (FAA, 1992).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 17
4 Baseline Characteristics of Explosives-Detection Systems Based on X-ray-Computed Tomography A legitimate comparison of x-ray CT-based EDSs and PFNTS-based explosives-detection techniques must be based on their baseline performance. Laboratory test results for PFNTS were presented in Chapter 3. In this chapter, both the laboratory performance and the operational performance of the FAA-certified CTX-5000 SP are reviewed. To date, most of the performance data on deployed explosives-detection equipment has been generated from tests conducted at the FAA Technical Center. However, operational (field) test data on false alarm rates are also reviewed in this chapter. Test Data from the FAA Technical Center Most of the performance data available for x-ray CT-based EDSs are from certification tests of the InVision CTX-5000 SP in 1996. Data from the certification tests of the recently certified InVision CTX-5500 DS and L3 Communications 3DX-6000 are not available yet but should be included in future analyses. The InVision CTX-5500 DS has been certified for two different inspection modes: SURE98 Mode, which has a lower false alarm rate than the CTX-5000 SP but a lower throughput rate, and CERT98 Mode, which has a similar false alarm rate to the CTX-5000 SP but a much higher throughput rate. The panel's analysis of performance data focuses on the CTX-5000 SP, although performance data for the CTX-5500 DS are also presented for reference. Table 4-1 shows the performance factors for the CTX-5000 SP and CTX-5500 DS, including the Pd, Pfd, and the bag throughput rate. Because the actual Pd and Pfa numbers are classified,1 Pd is given as a percentage of the overall Pd required for certification (X), and Pfa is given as a percentage of the Pfa required for certification (Y). Operational Demonstration In 1995, the FAA initiated the Airport Operational Demonstration Project (FAA, 1995) to determine the operational performance of the InVision CTX-5000 SP in the field as compared to its performance in certification tests. Three test sites were selected for the project: San Francisco International Airport (United Airlines); Atlanta Hartsfield International Airport (Delta Airlines); and Manila International Airport (Northwest Airlines). TABLE 4-1 Performance Test Results for the InVision CTX-5000 SP and CTX-5500 DS Equipment Detection Rate False Alarm Rate Throughput (bags/hour)   Sample Size Pd(% X)a Sample Size Pfa(% Y)a   CTX-5000 SP 600b 106 1,000c 90 245 CTX-5500 DS/SURE98 mode 600b 106 1,000c 60 264 CTX-5500 DS/CERT98 mode 600b 100 1,000c 95 362 a Percentage of the classified value required for certification. b A total of 150 bags were used containing improvised explosive devices (25 samples of six explosives, detonators, timers, and wires) and four different orientations to get a sample size of 600. c The 1,000 bags used to acquire the Pfa differed from the 600 bags used to acquire the Pd. The 1,000-bag set was also used to obtain throughput rates. 1   The actual values required for certification are recorded in classified FAA documents (FAA, 1992).

OCR for page 17
Two InVision CTX-5000s were installed in Atlanta and one each in San Francisco and Manila. The demonstration project included four open tests and one blind (red team) test using improvised explosive devices to determine Pd. The Pfa was acquired routinely throughout the project on real passenger bags. Only the data from the San Francisco and Atlanta operational deployments have been documented in final reports (FAA, 1997a, 1997b, 1997c). Some of the performance data for the CTX-5000 SP installed at San Francisco International Airport are given in Table 4-2. The automated explosives-detection capability of the CTX-5000 SP during open testing was about the same as the capability measured at the FAA Technical Center. However, operator intervention to resolve alarms measurably reduced Pd. This tendency was also observed during blind testing at San Francisco and Atlanta (FAA, 1997a, 1997b). During the operational demonstration project at San Francisco, the automated Pfa was 113 percent to 150 percent higher than the certification standard. Operational data reviewed by the inspector general of the U.S. Department of Transportation suggested that false alarm rates were as much as 169 percent higher than the certification standard. Data from the first of the four open tests show an average of 50 seconds for alarm resolution using the CTX-5000 SP. Although the resolution time was lower during subsequent tests, the combination of a high Pfa and a long alarm resolution time can have a significant impact on throughput rates. In fact, it was determined to be the limiting factor for throughput rate. Although the CTX-5000 SP exhibited a significantly higher automated (i.e., without operator intervention) Pfa in operation at airports than during certification testing, the TABLE 4-2 Summary of Open Testing of CTX-5000 SP at San Francisco International Airport Tests Sample Size Pd(% X) Pfa(% Y)a Machine (automated) 131 102 150 Machine + operator 131 89 5 a Pfa was obtained by measuring the Pfa for regular passenger baggage. image produced by the system is used by trained operators to assist in resolving the alarms. Thus, the actual Pfa (with operator intervention) is much lower than the automated Pfa measured during certification testing. Nevertheless, the time required for alarm resolution is a factor that must be taken into consideration when evaluating the performance of this EDS. The FAA's Security Equipment Integrated Product Team (SEIPT) has deployed more than 40 InVision CTX-5000 SP EDSs since 1997. Thus, the FAA now has an outstanding opportunity to collect performance data (e.g., Pd and Pfa) as well as operational performance data (e.g., down time, mean time between failure, and mean time to repair). Unfortunately, this information was not available to the panel, which substantially compromised the panel's ability to assess the performance of deployed equipment and to make comparisons between deployed equipment and alternative technologies, such as PFNTS. Credible future assessments of the performance of deployed explosives-detection equipment will require both performance and operational data.