Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS 37 APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS Classical psychophysical methods have as their goal the determination of a stimulus threshold. Thresholds measured can be detection, discrimination, recognition, and identification. The concept of threshold actually has two meanings, one empirical and one theoretical. Empirically a threshold is the stimulus energy that will allow the observer to perform a task (detection, discrimination, recognition, or identification) at some criterion level of performance (75 percent correct, for example). Sensitivity is defined as the reciprocal of the threshold value. The classical concept of detection threshold, as represented in the high threshold model of detection, hypothesizes that there is a stimulus level below which the stimulus has no effect (as if the stimulus were not there) and above which the stimulus is perceived. The classical psychophysical methods (the method of limits, the method of adjustment, and the method of constant stimuli) developed by G.T. Fechner (1860) were deigned to infer the stimulus value corresponding to the theoretical threshold from the observed detection performance data. In this sense, the stimulus threshold is the stimulus energy that exceeds the theoretical threshold with a probability of 0.5. Until the 1950s the high threshold model of detection dominated conceptualization of the detection process and provided the theoretical basis for the psychophysical measurement of thresholds. In the 1950s a major theoretical advance was made by combining detection theory with statistical decision theory. Actual detection performance was conceived to be based on two separate and independent processes: a sensory process and a decision process. The sensory process transforms the physical stimulus energy into some sort of internal representation, and the decision process makes a decision based on this representation to say âyes, the stimulus was presentâ or âno, the stimulus was not presentâ (in the simplest case). Each of these separate processes is characterized by at least one parameter: the sensory process by a sensitivity parameter and the decision process by a response criterion or response bias parameter. It was further realized that estimates of thresholds made using any of the three classical psychophysical methods confounded the sensitivity of the sensory process with the response criterion of the decision process. In order to measure these two separate characteristics, one needs two measures of detection performance. Not only must one measure the probability that the observer says âyesâ when a stimulus is present (the hit rate: HR) but also one must measure the probability that the observer says âyesâ when a stimulus is not present (the false alarm rate: FAR). Under certain assumptions, these two performance measures, the hit rate and the false alarm rate, may then be used to estimate detection sensitivity and decision criterion.
APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS 38 HIGH THRESHOLD MODEL OF DETECTION The specific way in which the hit rate and the false alarm rate are used to derive detection sensitivity and response criterion depends on the specific model one adopts for the sensory process and the decision process. Some of these different models and how to distinguish among them are discussed by Krantz (1969). Assuming the high threshold model leads to the following measures of sensory process sensitivity and decision process response criterion: g = FAR (2) where p is the probability that the stimulus will exceed the hypothetical threshold and g is the response bias (called guessing rate in the high threshold model). Equation 1 is the widely used correction-for-guessing formula. Extensive research testing the validity of the prediction of the high threshold model has led to its rejection as an adequate description of the detection process and the conclusion that neither Equations 1 nor 2 succeeds in separating the effects of sensitivity and response bias (Swets, 1961; Swets et al., 1961; Krantz, 1969; Green and Swets, 1974). One important characteristic of any detection model is its prediction of the relationship between the hit rate and the false alarm rate as the observer changes the response criterion. This plot of HR against FAR is called an ROC curve (receiver operating characteristic). By algebraic rearrangement of Equation 1, the high threshold model of detection predicts a linear relationship between HR and FAR in the ROC curve: HR = p + (1-p) * FAR (3) where p is the sensitivity parameter of the high threshold sensory process. This predicted ROC curve is shown in Figure 17. When one actually measures the HR and FAR pairs in a detection experiment using different degrees of response bias, one obtains a bowed-shaped ROC curve shown by the filled circles in Figure 17. This curve, which one actually obtains in experiments is quite different from the straight line relationship predicted by the high threshold model and is one of the bases for rejecting that model. SIGNAL DETECTION THEORY A widely accepted alternative to the high threshold model is the signal detection model. This model does not contain the concept of a sensory threshold (Swets, 1961). It assumes that the sensory process has a continuous output based on random Gaussian noise and that when a signal is present, it adds to that noise. The sensitivity of the sensory
APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS 39 process is expressed as the difference between the mean output under no signal condition and that under signal condition: d⢠(d-prime). The decision process is assumed to hold a single decision criterion (in more elaborate versions of this model, multiple criteria are possible). This decision criterion is based on the output of the sensory process. If the output of the sensory process equals or exceeds the decision criterion, the observer says âyes, the signal was present.â If the output of the sensory process is less than this criterion, the observer says âno, the signal was not present.â The decision criterion may be expressed in several ways. One is beta, the ratio formed by the likelihood that the observed output of the sensory process was due to a signal being present divided by the likelihood that the output was due to the signal being absent. Another measure is xc, the critical value of the sensory process output used as the decision criterion. If one assumes that the probability distributions describing the output of the sensory process are normal Gaussian distributions of equal variance, then d⢠and xc are calculated from the HR and FAR in the following way: FIGURE 17 Hit rate as a function of false alarm rate. The filled circles are HR-FAR pairs from a detection experiment, forming a bowed-shaped ROC curve. The straight line is the ROC curve predicted by the high threshold model of detection.
APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS 40 d⢠= ZHR â ZFAR (4) xc = ZFAR (5) where ZHR and ZFAR are the z-score transforms, based on the normal distribution, of the HR and FAR probabilities. The ROC curve predicted by the signal detection model is shown in Figure 18 along with the empirical data shown in the previous figure. The signal detection prediction is in accord with the observed data. The data shown in Figure 18 correspond to d⢠= 1.0. All ROC curves predicted by this model are anchored at the 0,0 and 1,1 points on the graphs. Each different value of d⢠generates a different ROC curve. For d⢠= 0, the ROC curve is the positive diagonal extending from 0,0 to 1,1. For d⢠greater than 0, the ROC curves are bowed. As d⢠increases, so does the bowing of the corresponding ROC curve. An algebraic rearrangement of Equation 4 leads to this relationship between HR and FAR: ZHR = d⢠+ ZFAR (6) FIGURE 18 Hit rate as a function of false alarm rate. The filled circles are the same data as in Figure 18. The smooth curve is the ROC curve predicted by the equal-variance Gaussian signal detection theory.
APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS 41 Equation 6 predicts that when HR and FAR are plotted as z-scores instead of probabilities, the bowed- shaped ROC curve shown in Figure 18 will be a straight line, because equation 6 is in the form of a linear equation. This predicted ROC curve is shown in Figure 19, along with the data from the previous figures. Sensitivity is generally a relatively stable property of the sensory process, but the decision criterion used by an observer can vary widely from task to task and from time to time. The decision criterion used is influenced by three factors: (1) the instructions to the observer; (2) the relative frequency of signal and no-signal trails (the a priori probabilities); and (3) the payoff matrix, the relative cost of making the two types of errors (false alarms and misses) and the relative benefit of making the two types of correct responses (hits and correct rejections). These three factors can cause the observer to use quite different decision criteria at different times, and, if the proper index of sensitivity is not used, changes in decision criteria will be incorrectly interpreted as changes in sensitivity. Figure 20 shows the high threshold sensitivity index, p, for different values of decision criteria, for an observer having constant sensitivity. The detection FIGURE 19 Z-score of the hit rate as a function of the Z-score of the false alarm rate. The same data from Figure 19 replotted in Z-score coordinates. The predicted ROC curve and the data form a straight line.
APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS 42 sensitivity, p, calculated from Equation 1, is not constant but changes as a function of decision criteria. FIGURE 20 Detection sensitivity, p, calculated under the assumptions of the high threshold model (equation 1), as a function of response bias, g (equation 2). Note that p is not independent of g, contrary to the assumption of the high threshold model of detection. Also plotted is dⲠ(equation 4) as a function of response bias. Note that it is unaffected by shifts in response bias. A widely used psychophysical procedure is the forced-choice paradigm, especially the two-alternative, forced-choice (2AFC) paradigm. Because only one performance index is obtained from this paradigm, the percentage correct, it is not possible to calculate both a detection sensitivity index and a response criterion index. It is now understood, however, that detection performance in the 2AFC paradigm is equivalent to an observer using an unbiased decision criterion and that the percentage correct performance can be predicted from signal detection theory. Specifically, the percentage correct in a 2AFC detection experiment corresponds to the area under the ROC curve obtained if the same stimulus were used in the yes-no signal detection paradigm (Green and Swets, 1974; Egan, 1975). Calculation of d⢠from the 2AFC percent correct is also straightforward:
APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS 43 d⢠= 2.00.5 * Zpc (7) where Zpc is the z-score transform of the 2AFC percentage correct (Simpson and Fitter, 1973). The area under the ROC curve for d⢠= 1.0, illustrated above, is 0.76 (the maximum area of the whole graph is 1.0). SUMMARY The classical psychophysical methods of limits, of adjustment, and of constant stimuli provide procedures for estimating sensory thresholds. These methods, however, are not able to separate the independent factors of sensitivity and decision criterion. Furthermore, there is no evidence to support the existence of sensory thresholds, at least in the form these methods were designed to measure. There are today two methods that allow one to measure an observer's detection sensitivity relatively uninfluenced by changes in decision criteria. One method is the forced-choice paradigm, which forces all observers to adopt the same decision criterion. The second method is based on signal detection theory and requires that there be two types of detection trials: some containing the signal and some containing no signal. Both detection sensitivity and response criterion may be calculated from the hit rates and false alarm rates resulting from the performance in these experiments. Either of these methods may be used to measure the contrast sensitivity function. âThresholdâ contrast corresponds to the stimulus contrast giving rise to a certain level of detection performance. A d⢠of 1.0 or a 2AFC detection of 0.76 is often used to define threshold, but other values may be chosen as long as they are made explicit. A comparison of contrast sensitivity functions measured by means of the method of adjustment and the two- alternative, forced-choice method is reported by Higgins et al. (1984). The variability of the 2AFC measurements is less than half of those made with the adjustment method. This reduction of measurement variability will increase the reliability of the threshold measures and increase its predictive validity. Although there are clear benefits of reducing the variability due to differences in decision criterions, the cost effectiveness of these benefits must be evaluated on a case by case basis. Factors such as testing time, ease of administration, ease of scoring, and cost must be carefully considered in relation to the desired reliability, accuracy, and ultimate use to which the measurements will be put. Finally, it must be recognized that no psychophysical method is perfect. Observers may make decisions in irrational ways; some may try to fake a loss of sensory capacity. Care must be taken, regardless of the psychophysical method used to measure capacity, to detect such behavior. A properly administered, conceptually rigorous psychophysical procedure will ensure the maximum predictive validity of the measured sensory capacity.