• Blind administration and blind evaluation of the technique, Whoever administers and scores tests based on the technique must do so in the absence of any information on whether the examinee is truthful or deceptive.

• Adequate sample sizes. Most of the studies examined [in the National Research Council 2003 polygraph report] were based on relatively small sample sizes that were sometimes adequate to allow for the detection of statistically significant differences but were insufficient for accurate assessment of accuracy. Changing the results of only a few cases might dramatically affect the implications of these studies.

• Appropriate comparison conditions and experimental controls. These conditions and controls will vary with the technique. A suggestion of what may be involved is the idea in polygraph research of comparing a polygraph examination with a bogus polygraph examination, with neither the examiner nor the examinee knowing that the test output might be bogus.

• Cross-validation of any exploratory data analytic solution on independent data. Any standardized or computerized scoring system for measurements from a technique cannot be seriously considered as providing accurate detection unless it has been shown to perform well on samples of examinees different from those on whom it was developed.

• Examinees masked to experimental hypotheses if not to experimental condition. It is important to sort out precisely what effect is being measured. For example, the results of a countermeasures study would be more convincing if examinees were instructed to expect that the examiner is looking for the use of countermeasures, among other things, rather than being instructed explicitly that this is a study of whether countermeasures work and can be detected.

• Standardization. An experiment should have sufficient standardization to allow reliable replication by others and should analyze the results from all examinees. It is important to use a technique in the same way on all the examinees, which means: clear reporting of how the technique was administered; sharply limiting the examiner’s discretion in administering the technique and interpreting its results; and using the technique on all examinees, not only the ones whose responses are easy to classify. If some examinees are dropped from the analysis, the reasons should be stated explicitly. This is a difficult test for a procedure to pass, but it is appropriate for policy purposes.

• Analysis of sensitivity and specificity or their equivalents. Data should be reported in a way that makes it possible to calculate both the sensitivity and specificity of the technique, preferably at multiple thresholds for diagnostic decision making or in a way that allows comparisons of the test results with the criterion on other than binary scales.

SOURCE: National Research Council, The Polygraph and Lie Detection, The National Academies Press, Washington, D.C., 2003, pp. 222-224.