National Academies Press: OpenBook

Emergent Techniques for Assessment of Visual Performance (1985)

Chapter: APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS

« Previous: APPENDIX A: BASIC FACTORS IN SPATIAL CONTRAST SENSITIVITY
Suggested Citation:"APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS." National Research Council. 1985. Emergent Techniques for Assessment of Visual Performance. Washington, DC: The National Academies Press. doi: 10.17226/916.
×
Page 37
Suggested Citation:"APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS." National Research Council. 1985. Emergent Techniques for Assessment of Visual Performance. Washington, DC: The National Academies Press. doi: 10.17226/916.
×
Page 38
Suggested Citation:"APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS." National Research Council. 1985. Emergent Techniques for Assessment of Visual Performance. Washington, DC: The National Academies Press. doi: 10.17226/916.
×
Page 39
Suggested Citation:"APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS." National Research Council. 1985. Emergent Techniques for Assessment of Visual Performance. Washington, DC: The National Academies Press. doi: 10.17226/916.
×
Page 40
Suggested Citation:"APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS." National Research Council. 1985. Emergent Techniques for Assessment of Visual Performance. Washington, DC: The National Academies Press. doi: 10.17226/916.
×
Page 41
Suggested Citation:"APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS." National Research Council. 1985. Emergent Techniques for Assessment of Visual Performance. Washington, DC: The National Academies Press. doi: 10.17226/916.
×
Page 42
Suggested Citation:"APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS." National Research Council. 1985. Emergent Techniques for Assessment of Visual Performance. Washington, DC: The National Academies Press. doi: 10.17226/916.
×
Page 43

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS 37 APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS Classical psychophysical methods have as their goal the determination of a stimulus threshold. Thresholds measured can be detection, discrimination, recognition, and identification. The concept of threshold actually has two meanings, one empirical and one theoretical. Empirically a threshold is the stimulus energy that will allow the observer to perform a task (detection, discrimination, recognition, or identification) at some criterion level of performance (75 percent correct, for example). Sensitivity is defined as the reciprocal of the threshold value. The classical concept of detection threshold, as represented in the high threshold model of detection, hypothesizes that there is a stimulus level below which the stimulus has no effect (as if the stimulus were not there) and above which the stimulus is perceived. The classical psychophysical methods (the method of limits, the method of adjustment, and the method of constant stimuli) developed by G.T. Fechner (1860) were deigned to infer the stimulus value corresponding to the theoretical threshold from the observed detection performance data. In this sense, the stimulus threshold is the stimulus energy that exceeds the theoretical threshold with a probability of 0.5. Until the 1950s the high threshold model of detection dominated conceptualization of the detection process and provided the theoretical basis for the psychophysical measurement of thresholds. In the 1950s a major theoretical advance was made by combining detection theory with statistical decision theory. Actual detection performance was conceived to be based on two separate and independent processes: a sensory process and a decision process. The sensory process transforms the physical stimulus energy into some sort of internal representation, and the decision process makes a decision based on this representation to say “yes, the stimulus was present” or “no, the stimulus was not present” (in the simplest case). Each of these separate processes is characterized by at least one parameter: the sensory process by a sensitivity parameter and the decision process by a response criterion or response bias parameter. It was further realized that estimates of thresholds made using any of the three classical psychophysical methods confounded the sensitivity of the sensory process with the response criterion of the decision process. In order to measure these two separate characteristics, one needs two measures of detection performance. Not only must one measure the probability that the observer says “yes” when a stimulus is present (the hit rate: HR) but also one must measure the probability that the observer says “yes” when a stimulus is not present (the false alarm rate: FAR). Under certain assumptions, these two performance measures, the hit rate and the false alarm rate, may then be used to estimate detection sensitivity and decision criterion.

APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS 38 HIGH THRESHOLD MODEL OF DETECTION The specific way in which the hit rate and the false alarm rate are used to derive detection sensitivity and response criterion depends on the specific model one adopts for the sensory process and the decision process. Some of these different models and how to distinguish among them are discussed by Krantz (1969). Assuming the high threshold model leads to the following measures of sensory process sensitivity and decision process response criterion: g = FAR (2) where p is the probability that the stimulus will exceed the hypothetical threshold and g is the response bias (called guessing rate in the high threshold model). Equation 1 is the widely used correction-for-guessing formula. Extensive research testing the validity of the prediction of the high threshold model has led to its rejection as an adequate description of the detection process and the conclusion that neither Equations 1 nor 2 succeeds in separating the effects of sensitivity and response bias (Swets, 1961; Swets et al., 1961; Krantz, 1969; Green and Swets, 1974). One important characteristic of any detection model is its prediction of the relationship between the hit rate and the false alarm rate as the observer changes the response criterion. This plot of HR against FAR is called an ROC curve (receiver operating characteristic). By algebraic rearrangement of Equation 1, the high threshold model of detection predicts a linear relationship between HR and FAR in the ROC curve: HR = p + (1-p) * FAR (3) where p is the sensitivity parameter of the high threshold sensory process. This predicted ROC curve is shown in Figure 17. When one actually measures the HR and FAR pairs in a detection experiment using different degrees of response bias, one obtains a bowed-shaped ROC curve shown by the filled circles in Figure 17. This curve, which one actually obtains in experiments is quite different from the straight line relationship predicted by the high threshold model and is one of the bases for rejecting that model. SIGNAL DETECTION THEORY A widely accepted alternative to the high threshold model is the signal detection model. This model does not contain the concept of a sensory threshold (Swets, 1961). It assumes that the sensory process has a continuous output based on random Gaussian noise and that when a signal is present, it adds to that noise. The sensitivity of the sensory

APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS 39 process is expressed as the difference between the mean output under no signal condition and that under signal condition: d• (d-prime). The decision process is assumed to hold a single decision criterion (in more elaborate versions of this model, multiple criteria are possible). This decision criterion is based on the output of the sensory process. If the output of the sensory process equals or exceeds the decision criterion, the observer says “yes, the signal was present.” If the output of the sensory process is less than this criterion, the observer says “no, the signal was not present.” The decision criterion may be expressed in several ways. One is beta, the ratio formed by the likelihood that the observed output of the sensory process was due to a signal being present divided by the likelihood that the output was due to the signal being absent. Another measure is xc, the critical value of the sensory process output used as the decision criterion. If one assumes that the probability distributions describing the output of the sensory process are normal Gaussian distributions of equal variance, then d• and xc are calculated from the HR and FAR in the following way: FIGURE 17 Hit rate as a function of false alarm rate. The filled circles are HR-FAR pairs from a detection experiment, forming a bowed-shaped ROC curve. The straight line is the ROC curve predicted by the high threshold model of detection.

APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS 40 d• = ZHR − ZFAR (4) xc = ZFAR (5) where ZHR and ZFAR are the z-score transforms, based on the normal distribution, of the HR and FAR probabilities. The ROC curve predicted by the signal detection model is shown in Figure 18 along with the empirical data shown in the previous figure. The signal detection prediction is in accord with the observed data. The data shown in Figure 18 correspond to d• = 1.0. All ROC curves predicted by this model are anchored at the 0,0 and 1,1 points on the graphs. Each different value of d• generates a different ROC curve. For d• = 0, the ROC curve is the positive diagonal extending from 0,0 to 1,1. For d• greater than 0, the ROC curves are bowed. As d• increases, so does the bowing of the corresponding ROC curve. An algebraic rearrangement of Equation 4 leads to this relationship between HR and FAR: ZHR = d• + ZFAR (6) FIGURE 18 Hit rate as a function of false alarm rate. The filled circles are the same data as in Figure 18. The smooth curve is the ROC curve predicted by the equal-variance Gaussian signal detection theory.

APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS 41 Equation 6 predicts that when HR and FAR are plotted as z-scores instead of probabilities, the bowed- shaped ROC curve shown in Figure 18 will be a straight line, because equation 6 is in the form of a linear equation. This predicted ROC curve is shown in Figure 19, along with the data from the previous figures. Sensitivity is generally a relatively stable property of the sensory process, but the decision criterion used by an observer can vary widely from task to task and from time to time. The decision criterion used is influenced by three factors: (1) the instructions to the observer; (2) the relative frequency of signal and no-signal trails (the a priori probabilities); and (3) the payoff matrix, the relative cost of making the two types of errors (false alarms and misses) and the relative benefit of making the two types of correct responses (hits and correct rejections). These three factors can cause the observer to use quite different decision criteria at different times, and, if the proper index of sensitivity is not used, changes in decision criteria will be incorrectly interpreted as changes in sensitivity. Figure 20 shows the high threshold sensitivity index, p, for different values of decision criteria, for an observer having constant sensitivity. The detection FIGURE 19 Z-score of the hit rate as a function of the Z-score of the false alarm rate. The same data from Figure 19 replotted in Z-score coordinates. The predicted ROC curve and the data form a straight line.

APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS 42 sensitivity, p, calculated from Equation 1, is not constant but changes as a function of decision criteria. FIGURE 20 Detection sensitivity, p, calculated under the assumptions of the high threshold model (equation 1), as a function of response bias, g (equation 2). Note that p is not independent of g, contrary to the assumption of the high threshold model of detection. Also plotted is d′ (equation 4) as a function of response bias. Note that it is unaffected by shifts in response bias. A widely used psychophysical procedure is the forced-choice paradigm, especially the two-alternative, forced-choice (2AFC) paradigm. Because only one performance index is obtained from this paradigm, the percentage correct, it is not possible to calculate both a detection sensitivity index and a response criterion index. It is now understood, however, that detection performance in the 2AFC paradigm is equivalent to an observer using an unbiased decision criterion and that the percentage correct performance can be predicted from signal detection theory. Specifically, the percentage correct in a 2AFC detection experiment corresponds to the area under the ROC curve obtained if the same stimulus were used in the yes-no signal detection paradigm (Green and Swets, 1974; Egan, 1975). Calculation of d• from the 2AFC percent correct is also straightforward:

APPENDIX B: DETECTION SENSITIVITY AND RESPONSE BIAS 43 d• = 2.00.5 * Zpc (7) where Zpc is the z-score transform of the 2AFC percentage correct (Simpson and Fitter, 1973). The area under the ROC curve for d• = 1.0, illustrated above, is 0.76 (the maximum area of the whole graph is 1.0). SUMMARY The classical psychophysical methods of limits, of adjustment, and of constant stimuli provide procedures for estimating sensory thresholds. These methods, however, are not able to separate the independent factors of sensitivity and decision criterion. Furthermore, there is no evidence to support the existence of sensory thresholds, at least in the form these methods were designed to measure. There are today two methods that allow one to measure an observer's detection sensitivity relatively uninfluenced by changes in decision criteria. One method is the forced-choice paradigm, which forces all observers to adopt the same decision criterion. The second method is based on signal detection theory and requires that there be two types of detection trials: some containing the signal and some containing no signal. Both detection sensitivity and response criterion may be calculated from the hit rates and false alarm rates resulting from the performance in these experiments. Either of these methods may be used to measure the contrast sensitivity function. “Threshold” contrast corresponds to the stimulus contrast giving rise to a certain level of detection performance. A d• of 1.0 or a 2AFC detection of 0.76 is often used to define threshold, but other values may be chosen as long as they are made explicit. A comparison of contrast sensitivity functions measured by means of the method of adjustment and the two- alternative, forced-choice method is reported by Higgins et al. (1984). The variability of the 2AFC measurements is less than half of those made with the adjustment method. This reduction of measurement variability will increase the reliability of the threshold measures and increase its predictive validity. Although there are clear benefits of reducing the variability due to differences in decision criterions, the cost effectiveness of these benefits must be evaluated on a case by case basis. Factors such as testing time, ease of administration, ease of scoring, and cost must be carefully considered in relation to the desired reliability, accuracy, and ultimate use to which the measurements will be put. Finally, it must be recognized that no psychophysical method is perfect. Observers may make decisions in irrational ways; some may try to fake a loss of sensory capacity. Care must be taken, regardless of the psychophysical method used to measure capacity, to detect such behavior. A properly administered, conceptually rigorous psychophysical procedure will ensure the maximum predictive validity of the measured sensory capacity.

Next: APPENDIX C: BASIC CONCEPTS IN FOURIER ANALYSIS »
Emergent Techniques for Assessment of Visual Performance Get This Book
×
Buy Paperback | $40.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Recent vision research has led to the emergence of new techniques that offer exciting potential for a more complete assessment of vision in clinical, industrial, and military settings. Emergent Techniques for Assessment of Visual Performance examines four areas of vision testing that offer potential for improved assessment of visual capability including: contrast sensitivity function, dark-focus of accommodation, dynamic visual acuity and dynamic depth tracking, and ambient and focal vision. In contrast to studies of accepted practices, this report focuses on emerging techniques that could help determine whether people have the vision necessary to do their jobs. In addition to examining some of these emerging techniques, the report identifies their usefulness in predicting performance on other visual and visual-motor tasks, and makes recommendations for future research.

Emergent Techniques for Assessment of Visual Performance provides summary recommendations for research that will have significant value and policy implications for the next 5 to 10 years. The content and conclusions of this report can serve as a useful resource for those responsible for screening industrial and military visual function.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!