Page 41
7—
Trace Explosives Detection
Trace explosives-detection devices (TEDDs) represent a significant portion of the current FAA/SEIPT deployment of explosives-detection equipment. Configured first as vapor detectors, and more recently as trace (e.g., minute particulates) detectors, these relatively low cost, portable, sensitive, chemically specific devices are sometimes used for screening carry-on baggage. TEDDs are being used in the current deployment for the resolution of alarms of checked baggage from bulk explosives-detection equipment and for checking suspicious carry-on hand baggage.
The detection of trace amounts of explosive does not necessarily reveal the presence of a bomb. An alarm from a TEDD signifies the presence of vapors from or particles of explosive material that can only be connected inferentially to the presence of an explosive. Although TEDDs can be used to determine the type of explosive material, they cannot be—and are not—used to determine the quantity of explosive material. Furthermore, trace-detection techniques may be vulnerable to unsophisticated countermeasures, and the absence of trace amounts of an explosive does not guarantee that no explosive is concealed in the bag.
The purchase and deployment of TEDDs in the United States was mandated before a TEDDs-certification program was developed on the assumption that their deployment would improve aviation security. Since then, at least one TEDD test and evaluation plan has been drafted by the FAA that includes an evaluation of the detection of traces of explosive materials on carry-on passenger baggage by TEDDs and associated procedures (DOT, 1997). The plan is intended to establish a long-term method of collecting operational data for the purpose of assessing the reliability, performance, and costs associated with deployed TEDDs.
Unfortunately, standards and procedures for evaluating the performance of TEDDs and TEDD operators were not available at the time of this writing. In this chapter, the panel introduces the principles of trace detection, discusses the current deployment of TEDDs, outlines a short-term test and
| ||||||||||||||||||||
evaluation program, and suggests a course of action for developing a certification process for TEDDs and TEDD operators.
Principles of Trace Detection
The detection of trace amounts of explosives on suspect articles is done in steps (Figure 7-1). First the suspect article is wiped with a sampler,1 which is either a patch of material or a vacuuming device that draws air through the material for concentration of the sample (Table 7-1). For example, an operator might wipe the handle of a suitcase, the outside of a case, a computer switch, or a suspected area with a sampler. The sampler is then manually put into the TEDD where the collected material is thermally desorbed from the wiping substrate. The vapor is then transferred to a detection system in the device for chemical and quantitative characterization.
1 Wiping is not the best method of sampling for all explosives. The optimum sampling technique varies by explosive type.
Page 42
Figure 7-1
Operational steps of a trace explosives-detection device.
The detection system separates chemical species and creates a signature characteristic of the chemicals present. The signature is then compared to the signature of actual explosive materials, and if the combination meets the trace alarm threat criteria, the operator is notified of an alarm.
Several types of physical and chemical techniques are used in the TEDD to identify explosive materials (Box 7-1). The rationale for using TEDDs is only valid if it is assumed that the bomb maker will contaminate his or her hands with explosives and subsequently contaminate the bomb container (e.g., a radio) during fabrication of the explosive device.
Research conducted on the fabrication of explosive devices under controlled conditions has shown that it is very difficult, but not impossible, to avoid contaminating the surface of the primary container housing the explosive device. Fingerprints with sufficient material to be detected by commercial TEDDs are usually left. Plastic explosive materials tend to be "sticky" and are transferred to anything that is touched. Tests using C4 explosives as a model material suggest that, even with only 10 percent yield (i.e., the sampler only collects 10 percent of the material available for sampling), enough material is left from tenth-generation fingerprints to be detected by current TEDDs (Greshem et al., 1994). It should be noted, however, that the sampling yield is a function of the surface sampled (e.g., telephone, radio, computer), the sampling technique, and the sampler material. The ease of removing particles from a surface depends on several factors, such as the amount of explosive present, molecular and electrostatic attraction between the particle and the surface, adsorbed surface films, particle size and shape, degree of particle agglomeration, surface roughness, and the duration of contact. For a TEDD to be effective, the surface of interest must be adequately sampled, and the TEDD must be operating within known limits of detection and chemical selectivity.
Deployment
As a part of the congressional mandate to deploy explosives-detection equipment, the SEIPT has scheduled the deployment of 631 TEDDs operated by 30 carriers at a cost of approximately $100,000 per deployed device. As of January 31, 1999, 366 TEDDs had been deployed in 39 airports, including all 19 Category X airports and one-third of Category I airports. The TEDDs are being used to resolve alarms of checked baggage screened by bulk explosives-detection equipment and to screen electronic devices and other carry-on items not cleared by x-ray at airport security checkpoints. The status of the current deployment is shown in Table 7-2.
The deployed TEDDs are being evaluated separately and not as part of an overall system, such as the TAAS. A study of how TEDDs are being used in airports would provide a much better picture of their utility as part of an overall security system, but the operational performance of TEDDs cannot be evaluated because the detection sensitivity and
|
Page 43
| |||||||||||||||||||||||||||||||||||||||||||||
chemical selectivity of the installed units were not previously determined in the laboratory. Because the performance capabilities of the TEDDs are not known, operator performance cannot be measured either. Furthermore, no standards or procedures for testing and data collection have been established. Thus, Pd, Pfa, sampler efficiency, and operator proficiency cannot be measured.
Testing and Evaluation
Quantifying the improvement in aviation security by explosives-detection equipment is difficult. The detection and interception of an explosive before it is loaded onto or carried aboard an aircraft would be a clear measure of efficacy. However, to date, no incidents of this type of detection have been recorded in the United States. In fact, the only statistics available on aircraft bombing attempts are on attempts that actually resulted in onboard explosions.2 The number of bombings and typical baggage-flow rates suggest that there is less than one bombing attempt against U.S. aircraft every 10 years. Thus, only one bag out of several billion contains an explosive. Therefore, it is almost impossible to measure directly the efficacy of TEDDs in preventing actual bombing attempts, and their effectiveness can only be estimated through comprehensive testing and evaluation, which should include the steps shown in Figure 7-2. Ultimately, the improvement afforded to the overall system by TEDDs can be evaluated by their impact on the SEF.
The most practical way to assess the contributions of TEDDs to overall aviation security is to establish performance requirements. Until a certification standard is established, the FAA could initiate a short-term (e.g., six-month) program to assess currently deployed TEDDs on the basis of their operational effectiveness. The data from operational testing could then be used to assess their effect on overall security (e.g., the proposed SEF).
Establishing the Trace Threat Amount
The amount of each threat material a TEDD must be able to detect (the "trace threat amount") must be established, not as a range but as a specific amount for each explosive and specified as a function of area because sampling is area dependent. FAA studies to establish trace threat amounts are ongoing.
Testing Equipment
Testing a TEDD involves introducing a known amount of explosive material from a sampler used by the manufacturer (e.g., a cloth wipe). The amount on the surface of the sampler—which is then introduced into the detector—must be a known amount not a random amount from wiping an area on a suitcase where explosive material may or may not be present or that may be difficult to transfer to the wipe. The amount used to test the TEDD could be determined experimentally. For example, if the TEDD's sampling of a defined threat amount (e.g., 1 ng from a specific area of a contaminated article) with the vendor's sampler is determined to be 10 percent efficient, the TEDD would have to alarm on 100 picograms of explosive placed directly on the sampler and into the detector to be considered effective. This test would determine the ability of the TEDD to detect explosives and the utility of the sampler for transferring the explosive to the detector. This standard could then be defined as a TEDD calibration standard (TEDDCS) and could be shared with the TEDD vendors.
The TEDDCS should consist of explosive material dissolved in a volatile solvent to produce a known concentration prepared under the direction and control of the FAA, not the vendor. Currently, TEDDs are tested on a daily basis by vendor-supplied standards, some of which are not even
2 No bombings of U.S. commercial aircraft have been confirmed since the initiation of the congressionally mandated deployment of explosives-detection equipment.
Page 44
Figure 7-2
A process for measuring the effectiveness of the operator/TEDD system.
quantitative. The panel was informed that the FAA is preparing standards, but it is not clear whether the standards are slurries or explosive materials dissolved in a volatile solvent (suspension solutions) (Fox, 1998). Slurries or suspension solutions pose significant difficulties for establishing known quantities and should not be used for this purpose (INEL, 1998).
Testing the Sampling Mechanism
Testing the sampler will involve defining a standard surface or surfaces (e.g., a suitcase handle) and standard (known) contamination level (e.g., the amount left from a tenth-generation fingerprint) and measuring how well a sampler collects a sample from a known amount of explosive material deposited on a known area (e.g., of fingerprint size). No test protocol has yet been developed to determine the efficacy of a sampler.
Testing the Equipment Operator
Once it can be shown that a TEDD sampler can collect an adequate amount by "wiping" a known area in an operationally compatible mode, the operator can then be tested. Unlike the automated EDS called for in the Aviation Security Improvement Act of 1990 (PL 101-604), TEDDs will only work if the operator samples a surface correctly. Given their dependence on operator performance, it is incumbent upon the FAA to ensure that TEDD operators sample baggage and other objects properly. This will probably require a protocol for certifying TEDD operators, as well as a protocol for maintaining their performance at a certified level.
Measuring the performance of a TEDD operator can be done using blind and double-blind tests, which require test objects that will not be recognizable as test objects by the operator. If, for example, an operator is handed vinyl luggage handles, floppy discs, or zippers (the current test items for experimental verification and validation of TEDDs), the operator will probably recognize them as test objects. Thus, this procedure will not serve as a blind test. Furthermore, there is evidence that the current test suspension is visible when deposited on test substrates (INEL, 1997). For blind tests, normal hand-carried items on which the trace threat has been deposited in a concealed manner must be used. So far, no procedures have been developed for placing a TEDDCS on realistic bags, for performing blind and double-blind tests with such objects, or for systematically evaluating TEDD operators in an airport environment. Blind testing is essential to measuring the performance of the operator-TEDD combination.
Evaluating False Positives
False positives and their impact on aviation security and airline/airport operations were not evaluated in this report. The Pfa of deployed TEDDs has been reported to be on the order of about 2 percent (INEL, 1997). Because the sensitivity of TEDDs (i.e., the amount of explosive required to set off an alarm) is not known, identifying the source of false alarms is problematic. Once the devices are operating at a known and specified detection level, the causes of false alarms can be identified. That is, the cause of all cleared (false) alarms should be documented so that a list of interferants can be tabulated. These data would be invaluable for future development and deployments.
Evaluating False Negatives
False negatives are exceedingly difficult to measure and assess. A false negative could be the result of a TEDD being out of specification or an operator not following proper sampling procedures, or it could be the result of a very careful bomb maker. In other words, a very "clean" bomb maker who uses a very good concealment technique could conceivably defeat a TEDD even if it is operating within specifications and the operator is performing flawlessly. Thus, false negatives are partly dependent on the terrorist and cannot be completely eliminated by improvements in TEDD technology or operational protocols.
Conclusions and Recommendations
Trace explosives-detection techniques detect traces of explosive materials, which may or may not indicate the presence of a concealed explosive. The best way to ensure that TEDDs are contributing to the security of commercial aviation is to ensure that they meet yet-to-be-defined FAA certification requirements. However, because sampling is a dominant factor in determining the effectiveness of the
Page 45
technique, separate certification and operational testing will be necessary for measuring the performance of equipment operators. The certification of TEDDs will be necessarily independent of the certification of TEDD operators to ensure that both are working properly. In the panel's opinion, the FAA has not provided a viable quantitative primary standard, and the current method of deposition on test objects does not mimic fingerprints.
Recommendation
The FAA should establish separate certification protocols for trace explosives-detection devices (TEDDs) and TEDD operators.
Recommendation
The FAA should establish a specific threat trace amount that trace explosives-detection devices must be able to detect to be certified.
Recommendation
The FAA should develop a trace explosives-detection device (TEDD) calibration standard (TEDDCS)—based on an established threat trace amount—for measuring the capability of a TEDD to detect explosives introduced directly onto the sampler and into the instrument. This standard should be used for certification testing, as well as for performance verification in the field. The TEDDCS should also be made available to TEDD manufacturers for developmental purposes.
Recommendation
The trace explosives-detection device calibration standard should be dissolved in a volatile solvent to produce a known concentration. Slurries or suspension solutions should not be used for this purpose.
Recommendation
The FAA should develop a test to measure the trace explosives-detection devices sampling mechanism (sampler). The test should measure how well a sampler collects a known amount (trace explosives-detection device sampling standard) of explosive material deposited on a known area in a manner consistent with the trace explosives-detection device's operational protocol.
Recommendation
The FAA should develop a certification test for operators of trace explosives-detection devices that measures their ability to screen baggage and other objects. In addition, the FAA should periodically conduct blind and double-blind tests to monitor the performance of certified operators.
Recommendation
The panel recommends that the FAA continue to monitor false alarms of deployed trace explosives-detection devices (TEDDs). The false alarms should be documented, including the causes, and a list of interferants compiled to be used for the future development and deployment of TEDDs.
