DIFFICULTIES IN MEASURING PUBLIC UNDERSTANDING, PERCEPTIONS AND VALUES

The standard method for learning what the public knows and thinks is survey research. This method has several problems, the two most serious of which are differentiating what the respondents actually know from what they infer from the content of the questions they are asked and framing effects, which can lead to different responses depending on how the question is posed (e.g., number of lives lost vs. numbers of deaths prevented).

Standard methods for learning public values include contingent valuation and utility elicitation. The first involves posing questions about willingness to pay, either to avoid having some condition occur or to make it go away once it has occurred. The second involves asking questions about the importance that the respondent attaches to various valued attributes so as to be able to construct a normative model of choice among outcomes that involve different levels of the valued attributes. Both of these approaches suffer from the difficulty that they assume that people have well-articulated values on the questions of interest and that the problem is simply one of measuring those values. In many instances relevant to a topic such as industrial ecology, this is most unlikely (Fischhoff, 1993).

Two examples illustrate the types of problems that arise. The first has to do with the way questions are framed. A few years ago, the U.S. Environmental Protection Agency (EPA) decided to do an internal study that looked across the agency and asked. ''Are we paying attention to the right set of environmental risks?" It was an internal risk-ranking exercise, but the agency also commissioned a Roper survey that gave respondents a long list of scary sounding things and asked how serious they thought each one was. The New York Times summarized the results as follows: "The American public and the EPA rank environmental threats quite differently with the public's fear focused most sharply on hazardous wastes sites which the government views much less serious" (Stevens, 1991). EPA's Frederick W. Allen wrote, "The most obvious reasons for the difference is that the general public simply does not have the information." EPA has argued that it is these public perceptions that have driven the agency to focus on things like Superfund, the program designed to clean up hazardous waste sites (Stevens, 1991).

The way framing works is best illustrated by the results from a different survey on the same topic that was conducted a few years later. The survey instructions asked respondents to take 7 minutes to list as many risks as they could think of. When the questionnaire was first piloted, participants voiced concerns such as "I'm worried about the risk of loosing my job", "I'm worried about the risks of my love life going on the rocks", "I'm worried about the risks of my kids flunking out of school", "I'm worried about the risks of eternal damnation", and so on. That seems like a pretty reasonable set of concerns, but it was not the set



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement