Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 263
16
Challenges in Evaluating Voluntary
Environmental Programs
Kathryn Harrison
Novel environmental policy instruments, including tradable permits, vol-
untary programs, and more flexible approaches to regulation, have been
embraced by governments throughout the world in recent years. Rigor-
ous evaluation of these diverse approaches is clearly essential if we are to draw
any conclusions about which approaches offer the most effective basis for future
policy. The focus of this chapter is on the particular challenges associated with
evaluation of voluntary programs. Because it is not possible to provide a sum-
mary of the effectiveness of the wide variety of voluntary programs that have
emerged in recent years within the scope of this brief chapter, I propose instead
to focus on some of the challenges that arise in evaluation of voluntary pro-
grams, illustrated with three case studies. The discussion that follows focuses on
conceptual issues in evaluation of voluntary programs, setting aside for the mo-
ment questions of political will and adequacy of administrative resources to
conduct program evaluations (Gormley, 2000~.
Policy evaluation is a difficult task, one that is too seldom undertaken for
traditional regulatory as well as novel voluntary programs. However, rigorous
evaluation of voluntary programs is arguably particularly important, because the
historical failure of markets and voluntarism to address environmental problems
and resource depletion creates a heavy burden of proof for those who advocate
voluntary alternatives to regulation (Andrews, 1998; Leiss and Associates, 1996~.
Unfortunately, evaluation of voluntary programs is simultaneously especially
difficult, because it is plagued by problems of data availability, credibility, and
self-selection. I will argue that as a result of these factors, there has been a
tendency to overstate the effectiveness of voluntary programs.
In the chapter that follows, I discuss three questions that arise in any evalu-
263
OCR for page 264
264 CHALLENGES IN EVALUATING VOLUNTARY ENVIRONMENTAL PROGRAMS
ation: (1) What is being evaluated? (2) What criteria should be used in evalua-
tion? (3) What baseline should be used for the evaluation? Then I illustrate
issues that arise in that discussion with case studies of the U.S. 33/50 program,
the Canadian Accelerated Reduction/Elimination of Toxics (ARET) program,
and the Canadian National Pollutant Release Inventory.
WHAT IS BEING EVALUATED? A TYPOLOGY OF
VOLUNTARY PROGRAMS
The Organization for Economic Co-operation and Development (OECD)
(1999) distinguishes among three types of voluntary programs: negotiated vol-
untary agreements, public voluntary programs (also sometimes referred to as
"voluntary challenge" programs), and unilateral (i.e., nongovernmental) pro-
grams. Among these, voluntary agreements are closest to regulation. Voluntary
agreements are characterized by strong expectations on the part of government
that nongovernmental parties to the negotiated agreements (typically industry)
will comply, and such agreements often are prompted by an explicit or implied
threat of regulation should the voluntary approach fail. Voluntary agreements
typically are negotiated by government and industry, whether an individual firm
or an entire sector, though other stakeholders also may be involved. Although
most voluntary agreements are nonbinding, some take the form of legally bind-
ing contracts.)
In contrast to voluntary agreements, governmental efforts to persuade target
groups to change their behavior via voluntary challenges or public voluntary
programs typically involve less arm twisting. Although voluntary agreements
have been prominent in Europe in recent years,2 voluntary challenge programs
are more common in North America. Examples include the U.S. Environmental
Protection Agency's (EPA's) 33/50 program and various Partners for the Envi-
ronment programs, and, in Canada, the ARET Challenge and the Voluntary Chal-
lenge and Registry for greenhouse gases.3 The central difference between a
voluntary agreement and a voluntary challenge is that the latter is an open-ended
public challenge that applies widely, but that no particular actor is expected to
accept. Thus, the EPA 33/50 program called on any and all firms releasing 17
specified toxic substances to reduce their discharges of those substances. In
practice, requirements of participation in challenge programs tend to be very
flexible. For instance, in both the 33/50 and ARET programs, participants were
expected to commit to make some reductions of the listed substances, but they
were not required in practice to meet any particular performance standard.
Information dissemination is often central to voluntary challenge programs,
which typically offer participants an incentive of public recognition. A related
approach, and one that has buttressed both the 33/50 and ARET programs, is the
mandatory pollutant release or "community right-to-know" inventory (discussed
also by Herb and colleagues, this volume, Chapter 15~. Such inventories require
OCR for page 265
KATHRYN HARRISON
265
that polluters publicly report their discharges, but do not require reductions of
those discharges per se. Rather, the intent of the approach is to facilitate public
pressure on firms to make voluntary reductions. Inventories can also document
changes in releases reported to complementary voluntary challenge programs.
In addition to these government-sponsored voluntary programs, a wide vari-
ety of nongovernmental or unilateral programs exist (many of which are re-
viewed by Nash, this volume, Chapter 14~. In many respects, nongovernmental
programs parallel the state-based voluntary programs described earlier, but with
someone other than government doing the coercing or encouraging. Participants
in nongovernmental voluntary programs vary from trade associations to nonpar-
tisan third parties such as the International Organization for Standardization
(ISO) to environmental groups. A critical distinction among nongovernmental
programs (to be discussed) is whether they focus directly on environmental per-
formance objectives or rather on management systems as a tool to improve envi-
ronmental performance.
No single unidimensional typology can capture all the relevant differences
among voluntary programs. There is thus substantial variation within each of
the categories listed.4 However, accepting OECD's three-way typology as a
useful starting point, it is apparent that the kind of voluntary program being
evaluated has several implications for the evaluation. The first issue concerns
the credibility of the evaluation, which can be especially problematic in the
case of nongovernmental programs sponsored by firms or trade associations.
In such cases, evaluations by the program's sponsors understandably may be
viewed with the same skepticism that would greet students not only grading
their own exam papers, but also writing the exam questions in the first place.
Verification by independent third parties is thus essential to lend credibility to
the evaluation.
That is not to suggest that program evaluations by government are immune
to credibility challenges. Government evaluations of voluntary agreements ne-
gotiated bilaterally by industry and the state may be met with particular skepti-
cism because as Ayres and Braithwaite (1992:55) note, "The very conditions
that foster the evolution of cooperation are also the conditions that promote the
evolution of capture and indeed corruption."5 Openness of the evaluation pro-
cess and, ideally, participation by environmentalists and other third parties in
negotiating voluntary programs in the first place will be critical in establishing
credibility (Gunningham and Grabosky, 1998~.
Distinctions between voluntary programs also are relevant to program evalua-
tion because conclusions about the effectiveness of one voluntary program may
not apply to another very different program. In this regard, the typology offered
earlier is at best a starting point, given the diversity of voluntary approaches within
each category. For example, lessons learned from legally binding Dutch cove-
nants negotiated to achieve environmental targets previously set by the state may
not apply to nonbinding "gentlemen's agreements" in which the goals themselves
OCR for page 266
266 CHALLENGES IN EVALUATING VOLUNTARY ENVIRONMENTAL PROGRAMS
may be subject to negotiation, let alone to voluntary challenges or nongovernmen-
tal programs.6
CRITERIA FOR EVALUATION
An obvious question for any program evaluation is what evaluative criteria
should be used. Although it is clear that evaluation of any environmental policy
instrument should consider its effectiveness in achieving environmental objec-
tives, policymakers also routinely consider other societal objectives in devising
and evaluating public policies. Evaluations of voluntary programs thus have
tended to apply multiple criteria, including environmental effectiveness, cost-
effectiveness for both government and business, equity, democratic accountabil-
ity and public participation, promotion of innovation, and adaptability to future
challenges (European Environmental Agency, 1997; Davies and Mazurek, 1996;
Harrison, 1998~. Many studies of voluntary approaches also have emphasized
"soft effects," such as improvement of the business-government relationship,
building trust among stakeholders, and promotion of a more environmentally
sensitive business culture (European Environment Agency, 1997; European
Commission, 1996; Nash and Ehrenfeld, 1997~.
Among these criteria, it is useful to draw a distinction between ultimate objec-
tives, such as equity, efficiency, and reduction of environmental impacts, and in-
termediate or instrumental objectives, such as fostering innovation and improving
business-government relationships. It is striking that even ambitious attempts to
evaluate voluntary programs often have been able to say little about attainment of
objectives such as reduction of environmental impacts and cost-effectiveness, and
thus have focused on instrumental goals (e.g., European Environment Agency,
1997~. Indeed, some analysts of voluntary programs focus almost exclusively on
these "soft" objectives.7 This emphasis on intermediate objectives is not unique to
voluntary programs. Gormley (2000) reports that in evaluating regulatory pro-
grams, government agencies also tend to focus on easier-to-measure "outputs"
(such as inspections and permitting activity) to the exclusion of "outcomes" (such
as emissions) or "impacts" on human health or the environment.
The focus on intermediate objectives arguably is more problematic with
respect to voluntary than regulatory programs, however. The emphasis on soft
effects often has been necessitated by a lack of data on environmental outcomes.
This can be particularly problematic with respect to industry-sponsored pro-
grams, where if data on environmental performance do exist, they tend to be
proprietary. However, it also can be an issue for government-sponsored volun-
tary programs which, unlike regulatory programs, typically do not rely on coer-
cion to compel disclosure of monitoring nor threaten inspections to verify firms'
reports. The extent of this problem was demonstrated by a study of 137 volun-
tary agreements by the European Commission in 1997, which found that 118 had
no requirement for firms to report the results of their compliance monitoring,
OCR for page 267
KATHRYN HARRISON
267
and 47 had no requirements for monitoring at all (European Commission Direc-
torate, 1997~. Based on a review of 88 industry-sponsored voluntary initiatives
worldwide, Borkey and Levesque (1998) concluded that industry initiatives typ-
ically made no provisions for monitoring or public reporting. In other words,
evaluations of voluntary codes of practice often have focused on intermediate
objectives simply because they have no data with which to assess performance
relative to environmental objectives. Although lack of environmental quality
data is often problematic for evaluation of regulatory programs as well, at least
the latter typically require monitoring and reporting by industry and also include
provisions for independent verification by the state.
That might be acceptable if there was a sufficient basis for the assumption that
"soft effects," such as business-government cooperation, will in fact deliver "hard"
environmental benefits. As Mazurek speculates (this volume, Chapter 13), coop-
eration may not only foster success in voluntary programs, but may spill over and
improve business-government relations in traditional regulatory programs. How-
ever, in the absence of evidence, it seems premature to draw that conclusion. I
have argued elsewhere that business-government cooperation also can be detri-
mental to the environment (Harrison, 1995), while the fine line between coopera-
tion and capture already has been noted (Ayres and Braithwaite, 1992:55~.
Closely related to this distinction between ultimate and intermediate criteria
is the distinction between voluntary programs that establish environmental per-
formance objectives and "voluntary environmental management systems," which
leave the setting of performance objectives up to individual participants and
focus instead on management approaches to ensure that those goals are achieved.
The latter approach is most common among industry-sponsored programs (Nash,
this volume, Chapter 14~. For example, the Canadian Responsible Care codes of
practice commit participants to "be aware of all effluents and emissions to the
environment, monitor those for which it is necessary, and implement plans for
their control when necessary," but do not specify which emissions should be
monitored nor what degree of control should be applied. Although the program
provides for independent audits of firms' compliance, the audit focuses exclu-
sively on whether the facility has all the elements of the environmental manage-
ment system in place, not on its environmental performance per se.8
It might be argued that this emphasis on intermediate objectives is quite
appropriate for nongovernmental codes of practice intended for use as internal
management tools and that it is thus unfair to hold such programs to the broader
societal standards appropriate for public policy. That is certainly true to the
extent that environmental management systems are limited to use as internal
tools. However, the complaint seems disingenuous in light of firms' efforts to
publicize to customers and regulators their compliance with management sys-
tems such as ISO 14001 and Responsible Care. More importantly, to the extent
that nongovernmental voluntary programs are being considered as a complement
to or substitute for government policies, it is entirely appropriate for the state to
OCR for page 268
268 CHALLENGES IN EVALUATING VOLUNTARY ENVIRONMENTAL PROGRAMS
hold them to the same standards, even if firms set less ambitious objectives for
themselves.
BENCHMARKS FOR EVALUATION
A critical question in evaluating voluntary programs concerns the appropri-
ate baseline against which to measure performance. One can, in fact, envision
several different levels of analysis of environmental effectiveness (European
Environment Agency, 1997~. The first, and simplest to achieve, is a measure of
change (e.g., in environmental performance, costs to government, costs to the
private sector, and so on) relative to a reference year. A second and more rigor-
ous standard is to ask to what degree the observed changes are attributable to the
program in question, acknowledging that actors' behavior might have changed
anyway in response to other factors, including market forces and other concur-
rent government policies. This standard necessitates the difficult task of devel-
oping a counterfactual "business-as-usual" baseline that would have prevailed in
the absence of the voluntary program.
Finally, the third and most difficult standard of analysis is to compare the
impact of the voluntary program to what would have happened if an alternative
approach had been pursued, a level of analysis even more challenging than es-
tablishing the "business-as-usual" baseline. Although a full-blown comparison
is obviously very difficult, greater attention to this issue is nonetheless warrant-
ed. Coercive regulatory programs often tend to be criticized against a yardstick
of complete compliance and economic efficiency. In contrast, there is a tenden-
cy for proponents of voluntary programs to hail any accomplishments above the
current baseline as a success.9 This is appropriate if the voluntary program is the
only realistic alternative to the status quo. However, to the extent that voluntary
and regulatory programs are being considered as substitutes, the relevant ques-
tion is how they compare to each other in practice.
In constructing counterfactuals (i.e., "what would have happened if" scenar-
ios), a critical issue that typically arises concerns the motivation for voluntary
action by business. Voluntary environmental improvements may be undertaken
by a profit-oriented firm for three reasons. First, the firm may be motivated to
change its behavior to save money, such as by reducing energy costs or by
recovering valuable reactants, regardless of any associated environmental bene-
fits. Second, a firm may be motivated by market pressures for environmental
improvements from investors, lenders, insurers, customers, or workers.~° Third,
a firm might opt for voluntary measures beyond those prompted by cost savings
and "green" market pressures in order to forestall or avoid mandatory regulation
or legal liability. A critical question in that case is whether the incentive for
voluntary action lies in reducing costs by accomplishing the same environmental
objective with greater flexibility than allowed by traditional regulation, or rather
in reducing costs by meeting less demanding environmental objectives.
OCR for page 269
KATHRYN HARRISON
269
Which of these three motives applies in a given case has implications both for
the kind of voluntary program that is likely to be effective and the kinds of issues
that will arise in program evaluation. When there are direct financial savings to be
realized by reducing environmental losses, businesses have strong incentives to
correct their behavior even in the absence of government intervention. However,
government-sponsored information dissemination and technology transfer pro-
grams nonetheless may help firms recognize and take advantage of waste reduc-
tion opportunities and energy efficiency improvements (Storey, 1996~. Similarly,
firms facing green market pressures also have incentives to change their behavior
regardless of the existence of governmental voluntary programs. Nonetheless,
government verification of voluntary efforts, such as through certification and
awards programs, may enhance those market forces by lending greater credibility
to firms' claims with consumers. Similarly, government programs to disseminate
information about all firms' behavior can help environmentally motivated con-
sumers and investors identify leaders and laggards alike.
With respect to voluntary actions motivated either by environmental market
pressures or direct cost savings, it is important to bear in mind that firms have
incentives to change their behavior regardless of the existence of a voluntary
program. Indeed, voluntary programs invite self-selection and thus prompt ques-
tions of whether participating firms are simply the ones that were already in-
clined to comply anyway. The challenge is thus to assess the marginal impact of
the program on firms' behavior. A program evaluation that simply assesses the
change in behavior relative to a reference year rather than a business-as-usual
baseline will tend to overstate program effectiveness.
A similar issue of self-selection also arose in the context of energy conser-
vation programs adopted in the 1970s. Some analysts evaluating those programs
argued that failure to control for self-selection inevitably would result in over-
statement of program benefits, because individuals or firms undertaking volun-
tary behavioral changes tend to be those facing the lowest costs of compliance
(Weinstein et al., 1989~. In contrast, Keating (1989) argued that self-selection
could result in understatement of program benefits, because those who opt to
participate tend to be those who perceive the greatest benefits from participation.
In that case, comparison of any extra actions undertaken by participants relative
to actions by nonparticipants would not provide an accurate assessment of the
program's benefits because, in the absence of the program, participants actually
would have done less than the "control" nonparticipants. However, as Keating
noted, understatement of effectiveness seemed to be more problematic with re-
spect to programs that offered subsidies for energy conservation, and indeed for
those who had already invested in desired energy conservation measures would
not be eligible. In contrast, overstatement was more problematic in the absence
of such inducements, where those who had already assumed compliance costs
not only would be allowed, but also would be most likely to participate. The
latter is more consistent with contemporary voluntary challenge programs.
OCR for page 270
270 CHALLENGESIN EVALUATING VOLUNTARY ENVIRONMENTALPROG^MS
The situation is quite different when firms are motivated to undertake volun-
tary actions by a threat of regulation. In this case, the challenge for government
is not merely to enhance existing market incentives, but to create those incen-
tives in the first place in the form of a credible and sustained threat of regulation
(Glachant, 1994:47; Gibson, 1999:244~. There is reason to believe that govern-
ment coercion is still a powerful influence on business environmental behavior.
A survey by the European Commission found that the potential to forego or
postpone regulation was cited as the most important benefit of voluntary envi-
ronmental agreements by roughly two thirds of industry respondents (European
Commission, 1997~. Similarly, a Canadian business survey reported that 95
percent of firms cited compliance with regulations as one of the top five factors
motivating their environmental improvements. One implication is that in eval-
uation of voluntary agreements in particular, the applicable comparison is not
necessarily the status quo or even the business-as-usual baseline, but rather what
would have happened had regulations been adopted instead. The significance of
coercion in motivating firms' behaviors also suggests a need to look at whether
concurrent regulatory policies have contributed to any changes in behavior ob-
served relative to the reference year.
The problem is that those conducting an evaluation of a voluntary program
typically do not know what is motivating firms targeted by the program. Differ-
ent firms are likely to have different motivations, and the same firm may be
motivated to varying degrees by each of the three incentives noted earlier. Firms
have incentives to understate their environmental releases and impacts whether
they face voluntary or regulatory programs. However, it may be particularly
difficult to unpack the degree to which progress is attributable to a voluntary
program as compared to regulation. Once a regulatory program is established
requiring a comparable level of compliance for all facilities affected, there is
little reason for a firm to overstate what it "would have done anyway." In
contrast, firms participating in a voluntary program in order to forestall regula-
tion have an interest in convincing regulators not only of how much they have
done but also how they have sufficient market incentives to maintain that behav-
ior in the absence of regulation. As a result, although the need to establish a
credible baseline requires that the evaluator understand what is going on in the
heads of the target population, it is by no means clear that one can get a credible
answer simply by asking.
CASE STUDIES
Three prominent voluntary programs the U.S. 33/50 program, the Canadi-
an Accelerated Reduction/Elimination of Toxics (ARET) program, and the Ca-
nadian National Pollutant Release Inventory (NPRI) are reviewed in this sec-
tion. No claim is offered that these three programs represent the universe of
voluntary programs, though it is noteworthy that they are often advanced as
OCR for page 271
KATHRYN HARRISON
271
particular success stories. Rather, the case studies are used to illustrate some of
the issues already introduced, including the issue of credibility in self-reporting,
the importance of establishing a common reference year, and the importance of
questioning the degree to which progress relative to that reference year can be
attributed to the program.
33/5012
In 1991, the EPA challenged the business community to voluntarily reduce
its releases and transfers of 17 high-priority chemicals by 33 percent by the end
of 1992 and by 50 percent by the end of 1995, dubbing the program "33/50."
Consistent with the previous discussion of voluntary challenges, requirements
for participation were very flexible. A firm needed only to write to the EPA
pledging some degree of reduction of its discharges of any of the 33/50 chemi-
cals. In turn, the EPA would provide a certificate of appreciation and recognize
33/50 participants in its annual report on the Toxics Release Inventory (TRI).
Although only 13 percent of firms contacted by the EPA agreed to partici-
pate, the goal of a 33-percent reduction relative to the reference year was achieved a
year early, by the end of 1991, and the 50-percent reduction goal also was
achieved early, by the end of 1994 (EPA, 1999~. By the target year of 1995,
reductions of 55 percent had been achieved. That has prompted many, including
the EPA, to cite the 33/50 program favorably as an example of the potential of
voluntary approaches. However, when one considers whether reductions were
achieved relative to a "business-as-usual" baseline, the benefits of the 33/50
program are not as clear.
As noted in Table 16-1, the first concern is that when the program was
launched in 1991, the EPA chose 1988 as the reference year, because it was the
most recent year for which discharge data were available. As a result, more than
one quarter of the reported reductions occurred before the program' s inception.
TABLE 16-1 Analysis of Reductions of Discharges and Transfers of 17
Chemicals Reported to the 33/50 Program
1988 Reference Year
1990 Reference Year
Goal
Total reduction by 1995 target year
Reductions after start of program
Reductions by participants
Excess reductions by participants relative to
nonparticipants
50%
55%
40%
27%
9%
NA
NA
47%
32%
11%
Source: EPA (1999).
OCR for page 272
272 CHALLENGESIN EVALUATING VOLUNTARY ENVIRONMENTALPROGRAMS
Excluding these reductions, the total reductions reported to the 33/50 program
were 40 percent relative to the 1988 reference year or 47 percent relative to a
more meaningful 1990 reference year.
The significant downward trend in releases even before the inception of the
program leads one to ask what was motivating those reductions and to what
degree those trends continued during the 33/50 years. In fact, even firms that
elected not to participate in the program made substantial reductions in their
discharges and transfers of 33/50 chemicals over the course of the program.
Indeed, more than one quarter of the reductions reported by the 33/50 program
were made by nonparticipating firms. Although it is conceivable that some
nonparticipating firms were motivated by the 33/50 program to reduce their
releases, perhaps because they interpreted the program as a subtle threat of regu-
lation rather than an opportunity for credit, the fact that firms were making deep
reductions even before the program was introduced would lead one to look first
for other explanations. Excluding nonparticipants, reductions relative to a 1990
reference year fell by 32 percent, as shown in Table 16-1.
Circumstantial evidence does indicate that the 33/50 program encouraged
firms to make reductions over and above what they would have made otherwise.
Firms participating in the 33/50 program reduced their discharges of 33/50 chem-
icals by 55 percent from 1990 to 1995, compared to 36 percent for nonparticipat-
ing firms. This 19 percent "extra" reduction by participating firms translates to
an 11 percent reduction relative to the total releases and transfers of 33/50 chem-
icals in the 1990 reference year.
The question of self-selection remains, however. The fact that partici-
pants in the 33/50 program made greater reductions than nonparticipants does
not necessarily indicate that those reductions were prompted by the 33/50 pro-
gram. Firms already inclined to make substantial reductions of 33/50 chemi-
cals, whether in response to negative publicity associated with mandatory re-
porting of discharges to TRI, market forces, cost savings, or concurrent
regulatory requirements, simply may have been the ones inclined to sign on for
credit. This is supported by Arora and Cason's (1996) finding that the larger a
firm's releases and transfers, the more likely it was to participate in 33/50
because these are the firms that would be expected to be subject to the greatest
pressure in response to the release of TRI data, even in the absence of the 33/
50 program. It is also problematic that none of the analyses of the 33/50
program conducted to date systematically have controlled for the effects of
concurrent regulations. Yet the EPA (1999) reports that reductions of two
Montreal protocol substances included in the 33/50 program alone account for
27 percent of reductions from 1990 to 1995.~3 Moreover, O'Toole et al. (1997)
found that stringency of state regulations was one of the two most important
factors in accounting for state-level reductions of 33/50 chemicals as reported
to TRI. In summary, although it is not possible to conclude with confidence
the precise benefits of the 33/50 program, it appears to have prompted dis-
OCR for page 273
KATHRYN HARRISON
273
charge reductions of the specified chemicals of less than 11 percent, well be-
low the 55 percent often attributed to the program.
ARET
The Canadian ARET Challenge, launched in 1994, is similar in many re-
spects to the 33/50 program, though more ambitious. After the details of the
challenge were negotiated by the government and some industry representatives,
a broader challenge was issued to all firms to reduce discharges of some 30
chemicals considered to be toxic, persistent, and bioaccumulative by 90 percent
by the year 2000, and of 87 others with one or more of these characteristics by
50 percent by the same year. Characteristic of a voluntary challenge program,
there was no threat of penalties for failure to achieve those goals. Indeed, as in
33/50, firms that choose to participate were not required to commit to the full 90-
percent and 50-percent reductions.
Preliminary assessments of the impact of ARET based on the first 4 years of
participant reports are promising. By the end of 1998, action plans had been
received from 316 industry and government facilities (ARET,2000~. Discharges
of all ARET substances had been reduced 67 percent relative to base year levels.
Reduction levels of the class of substances targeted for 50-percent reductions
were surpassed 4 years ahead of schedule, with further reductions promised.
As with 33/50, however, the degree to which these reductions are attribut-
able to the ARET program is unclear. The base year problem is exacerbated in
the ARET case, because each participating facility can pick its own base year
anytime after 1987. This option allows firms to claim credit toward the ARET
program for discharge reductions they made as much as 6 years before the pro-
gram's inception, and to strategically choose a year with particularly high dis-
charges to maximize apparent reductions (Gallon, 1998~. In fact, ARET pro-
gram figures indicate that roughly half of the 67-percent reductions claimed by
the end of 1998 had been achieved before the program was even launched.~4
As with 33/50, there are also questions about whether the reductions attrib-
uted to ARET are in fact voluntary, and if so, whether they are attributable to the
program. The ARET program notes that one quarter of base year emissions
were substances regulated under the Canadian Environmental Protection Act
(CEPA), and a further 11 percent were substances undergoing evaluation for
potential regulation under CEPA. i5 Thus one might question whether the ARET
reductions are an artifact of regulation. Harrison and Antweiler (2001) found
that facilities reporting on-site releases of regulated substances to the Canadian
National Pollutant Release Inventory reported greater reductions of those sub-
stances over time than of other substances. There is also anecdotal evidence that
significant reductions by individual sources can be attributed to regulation.
Moreover, as with 33/50 and TRI, voluntary reductions may have been driven
less by the positive publicity associated with the ARET challenge than the nega-
OCR for page 274
274 CHALLENGESIN EVALUATING VOLUNTARY ENVIRONMENTALPROG^MS
five publicity associated with public reporting of discharges of the half of the
ARET chemicals covered by the NPRI. Finally, the absence of any provisions
for third-party verification of firms' own claims of discharge reductions is trou-
bling, particularly for the half of ARET chemicals not covered by the regulatory
NPRI program (Leiss and Associates, 1996~.
NPRI
The discussion of the 33/50 and ARET programs notes the potentially con-
founding influence of toxic pollutant release inventories in both countries. Firms
may have been motivated less by the positive incentives in the form of public
recognition offered by the voluntary challenge programs as discussed earlier,
and motivated more by negative incentives (shame) generated by the pollutant
release inventories that underpinned them. Such inventories have increased in
popularity since the EPA established its TRI in 1988. The Canadian government
created its NPRI in 1993, and similar inventories have since been established in
Australia and the European Union. It is noteworthy that such inventories are not
exclusively voluntary programs in that all facilities leaders and laggards alike-
are required by law to report their toxic releases to the state. However, the
raison d'etre for those regulations compelling disclosure is to promote voluntary
action. Pollutant inventories are predicated on the assumption that, armed with
more complete information about firms' environmental practices, consumers,
workers, and investors will be empowered to use markets and social networks to
pressure firms to voluntarily reduce their releases (Herb and colleagues, this
volume, Chapter 15~.
The impact of such inventories, measured in terms of the least challenging
baseline of change relative to the base year, has been highly encouraging. Total
releases reported to the TRI declined by 46 percent from 1988 to 1995 (Natan
and Miller, 1998~. Releases reported to the Canadian NPRI similarly declined
by 36 percent in the first 3 years of the program, from 1993 to 1996. This
success has led some commentators to conclude that discharge inventories that
prompt voluntary action are more effective than regulation (Gunningham and
Grabosky, 1998:64; Organization for Economic Cooperation and Development
[OECD], 1996; Fung and O'Rourke, 2000~. However, a more critical look at
these inventories raises a number of questions for program evaluation.
First, it is important to acknowledge uncertainty about the extent to which
the reductions relative to reference years are real (Hearne, 1996~. Although
accurate reporting is legally mandated, individual facilities prepare their own re-
ports to both TRI and NPRI with minimal oversight by regulators. Indeed, facili-
ties are not required to measure their own discharges; they can estimate them using
techniques of varying reliability. Thus, an apparent decline in releases from a
facility from one year to the next may simply represent adoption of alternative
estimation methods. There is also concern that reported reductions often reflect a
OCR for page 275
KATHRYN HARRISON
275
shell game, in which facilities merely recategorize waste streams so they no longer
have to be reported (Natan and Miller, 1998~. This could be especially problemat-
ic for the Canadian NPRI, which did not require reporting of recycling and reuse in
the first few years of the program and thus may have invited facilities to reclassify
their waste streams as "recycling" to evade reporting.
There has been a tendency to assume that the reductions revealed by TRI
and NPRI have been voluntary responses to market mechanisms (e.g., Tieten-
berg, 1998:593; Fung and O'Rourke, 2000~. However, the picture becomes less
clear when one attempts to assess the degree to which the observed changes
relative to base years are attributable to information dissemination. With respect
to the U.S. TRI, few studies have considered the impact of regulatory programs
on TRI discharges. Shapiro (1998, 1999) found state regulatory efforts to be
among the most important determinants of reductions of TRI releases over time.
Similarly, Khanna and Damon (1999) found that liability under the federal Su-
perfund legislation and anticipation of new hazardous air pollutant regulations
under the U.S. Clean Air Act were among the most significant factors in explain-
ing firms' TRI reports for 33/50 program chemicals. Finally, Santos, et al.
(1996) found that regulatory compliance was one of the two reasons most fre-
quently cited by facilities (the other being employee health) for reduction of their
TRI releases and transfers. These studies suggest a need for greater attention to
regulation and liability in accounting for the rapid progress apparent in both
discharge inventories and the voluntary challenge programs predicated on them.
With respect to Canada's NPRI, analyses performed by Harrison and Ant-
weiler (2001) are summarized in Figure 16-1. The top line in Figure 16-1 repre-
sents the sum of reported releases of all NPRI substances by all facilities that
reported to NPRI in one or more of the years between 1993 and 1998. Consis-
tent with experience with the TRI, there was a dramatic 36-percent decrease in
the first 3 years from 1993 to 1996, though that has since been counteracted
somewhat by growth in releases, resulting in an overall reduction of 30 percent
from 1993 to 1998.
These reductions of both on-site releases and off-site transfers may be un-
derstated as a result of a growing number of facilities reporting to NPRI. A1-
though some of these are new facilities contributing real increases in waste pro-
duction, others are older facilities that either belatedly learned of the requirement
to report to NPRI or were affected by a minor adjustment to NPRI reporting
requirements in 1995. The exclusion of these older facilities from the totals in
previous years thus may understate reductions over time. One way to control for
this is to focus on the subset of facilities that reported in both 1993 and 1998
(though this has the disadvantage of excluding genuinely new facilities). As
indicated by the middle line in Figure 16-1, these facilities are doing somewhat
better than the totals for all facilities.
Among this subset of continuous reporters, a relatively small number of
facilities account for a substantial fraction of the total reduction in onsite releas-
OCR for page 276
276
250
225
200
175
150
125
100
75
50
25
o
CHALLENGES IN EVALUATING VOLUNTARY ENVIRONMENTAL PROGRAMS
AC
1993 1994 1995 1996 1997 1998
FIGURE 16-1 Trends in on-site releases (kilotons).
Note: A: total on-site releases from all facilities reporting to NPRI each year. B: total on-
site releases from the "continuous reporter" subset of facilities, which reported to NPRI in
each year 1993 through 1998. C: total on-site releases from "continuous reporters" ex-
cluding the pulp and paper industry and the Kronos facility in Varenne, Quebec.
Sources: Harrison and Antwiler (2000~. Reprinted with permission of Centre for Economic
and Social Policy, University of British Columbia.
es (Harrison and Antweiler, 2001~. Indeed, a single Quebec facility, Kronos
Canada, which adopted process changes in response to regulatory enforcement
actions undertaken by both the federal and provincial governments in the early
l990s (Picard, 1992; Hamilton, 1993), accounts for more than half of the total
reductions. In addition to Kronos, many of the facilities that contributed the
greatest reductions to NPRI were pulp and paper mills. It is noteworthy that the
pulp and paper industry is the only industry that faced new discharge regulations
at the national level in Canada during the past decade, and it also was subject to
extensive reform of regulations and permits at the provincial level in the early
l990s (Harrison, 1996~.
Because there are compelling reasons to conclude that reductions by these
facilities were driven by regulation (not least among them these industries' resis-
tance to behavioral change before regulatory enforcement), the final series in
Figure 16-1 reports trends among the continuous reporters, with Kronos and the
OCR for page 277
KATHRYN HARRISON
277
pulp and paper industry excluded. It is striking that once these most obvious
impacts of national regulation are excluded, the impressive 37-percent reduction
of on-site releases by continuous reporters evaporates, leaving a net increase of
7 percent. Although we cannot know whether an even greater increase in releas-
es might have been observed in the absence of NPRI, this figure does suggest
that the dramatic reductions of toxic discharges typically attributed to NPRI are
largely the result of traditional regulation, rather than a voluntary response by
firms to information disclosure.
A final question concerns the impact on human health and the environment
of these emissions reductions. Pollutant release inventories publicly report the
volume of releases, but do not provide an estimate of the risks such releases
might pose to human health or the environment. Harrison and Antweiler (2001)
take one step closer to such an evaluation by adjusting reported NPRI releases
for toxicity using the EPA's Chronic Human Health Indicators.~7 After adjust-
ing for toxicity, the 30-percent reduction in total on-site releases between 1993
and 1998 translates to a 9-percent increase. In other words, although the total
weight of releases has declined, the toxicity of those releases apparently has
increased, and has done so to a degree that this substitution effect may actually
outweigh the benefits of declining releases. These findings, although far from
constituting a complete risk assessment, reinforce the call for program evaluations
to go beyond mere "outputs" and "outcomes" to consideration of "impacts."
CONCLUSIONS
The one clear area of consensus among students of voluntary approaches is
that there has been too little attention to evaluation of either the economic or
environmental benefits of voluntary programs (Storey, 1996; Davies and Ma-
zurek, 1996; European Environment Agency, 1997; Beardsley, 1996; National
Research Council, 1996; OECD, 1999; Mazurek, this volume, Chapter 13~. In
part, this reflects the novelty of voluntary approaches; it is simply too early to
assess their effectiveness in many cases. However, it also reflects a pathology of
unclear targets and inattention to the kinds of monitoring, verification, and pub-
lic reporting needed to support program evaluation.
At a minimum, systematic evaluation of any program, whether regulatory,
voluntary, or market based, demands a consistent reference year, clear expecta-
tions, monitoring and verification of environmental performance, and public re-
porting. Some of these characteristics present special challenges for voluntary
programs, which all too often have had unclear expectations, inconsistent base-
lines, and few if any monitoring and reporting requirements. In contrast, regula-
tory programs typically have quite explicit and consistent standards, self-moni-
toring requirements, and provisions for independent compliance monitoring by
the state. Nonetheless, these characteristics could in theory become standard
practice for all government-sponsored voluntary programs as well. Nongovern-
OCR for page 278
278 CHALLENGESIN EVALUATING VOLUNTARY ENVIRONMENTALPROGRAMS
mental programs present a greater challenge, because private-sector sponsors
may not be as inclined to conduct rigorous and public evaluations. Moreover,
many nongovernmental programs are environmental management systems, which
specify objectives in the form of management "inputs" rather than environmen-
tal performance "outputs." However, if participants in such programs seek ei-
ther recognition or concessions from regulators, it is reasonable for the state to
establish comparable expectations as for government-sponsored voluntary and
regulatory programs.
Of course, program evaluation also requires political will and administrative
resources. The greater effort made to evaluate the voluntary energy conserva-
tion programs in the 1980s may reflect the fact that many of those programs
were predicated on government subsidies. With less public funding at stake
(recognition is cheap after all), there simply may be less incentive to allocate
resources to evaluation today, even though the opportunity costs of misdirecting
scarce resources to a less effective policy instrument may be substantial. Ironi-
cally, the same impulse to reform regulation to lessen the burden on regulated
interests that has given rise to voluntary programs simultaneously may have
rendered evaluation of the effectiveness of those new programs more difficult.
For instance, in an effort to minimize the burden on the private sector, the U.S.
Paperwork Reduction Act of 1995 establishes a quite elaborate process of Federal
Register notices and Office of Management and Budget approval for "informa-
tion collection requests" should an agency want to contact 10 or more firms or
individuals.
The issue of establishing a baseline for evaluation also remains a special
challenge for voluntary programs. In the context of the literature on program
evaluation, with voluntary programs we typically confront "quasi-experiments"
involving nonequivalent groups (Cook and Campbell, 1979), in which the chal-
lenge is to separate the effects of "treatment" (i.e., the voluntary program) from
differences resulting from lack of comparability between the "treated" and "un-
treated" groups. The tendency to assess environmental progress only relative to
reference years almost certainly exaggerates program effectiveness, because
some fraction of improvements typically would be attributable to market incen-
tives and/or concurrent regulatory requirements in the absence of the voluntary
program (to say nothing of the scenario in which the reference year is several
years prior to the launch of the voluntary program). The problem is that devel-
opment of a "business-as-usual" scenario in the absence of the voluntary pro-
gram requires knowledge of what factors are motivating business behavior. Some
recent studies have undertaken extensive interviews with program participants to
address this issue.~9 Leverage also may be gained by considering trends prior to
introduction of the program, looking for the impact of concurrent governmental
programs, and using regression techniques to control for other factors that might
explain self-selection.20 Ideally, new programs could be introduced (perhaps
initially at a pilot scale) in which a randomly assigned control group is ineligi-
OCR for page 279
KATHRYN HARRISON
279
ble for participation. Although none of these approaches offers a panacea, the
risk of misallocating scarce resources to ineffective programs surely justifies
greater effort.
NOTES
1 Although such contracts are legally binding, they can still be considered voluntary agree-
ments because parties enter into the contract voluntarily.
2 For reviews of European experience, see European Environment Agency (1997), Borkey and
Levesque (1998), and Rennings et al. (1997).
3 Mazurek's chapter in this volume (Chapter 13) and EPA (2001) both offer excellent over-
views of U.S. challenge and information-based programs.
4 See Ehrenfeld, Response to Talking with the Donkey: Cooperative Approaches to Environ-
mental Protection, and Harrison, A Response to Professor Ehrenfeld, Journal of Industrial Ecology,
online letters to the editor, at http://www.yale.edu/jie.
5 Consistent with a close relationship between business and government, Dietz and Rycroft
(1987) reported considerable career mobility between government and industry, but almost none
between government and environmental nongovernmental organizations.
6 On the importance of the distinction between negotiation of goals versus means, see Glachant
(1994).
7 For instance, Nash and Ehrenfeld (1997) evaluate environmental management systems against
the criterion of whether they foster "cultural change" within a firm.
8 It is noteworthy that there is no requirement to make Responsible Care compliance audits
public in the US program (Nash and Ehrenfeld, 1997). Indeed, the U.S. American Chemistry Coun-
cil leaves it to individual member companies to define what constitutes "full implementation" for
their own circumstances (Mazurek, 1998).
9 The EPA Partners for the Environment program's summary of benefits offers an example
(see http://www.epa.gov/partners/partnerships.html). It is also noteworthy that the figures presented
are based on self-reporting by individual "partners" and, in light of the discussion to follow, that no
effort has been made to assess which of those benefits were prompted by the program or whether
some fraction might have occurred even in the absence of the program in question.
10 Among these, second-order "green" pressures from investors, lenders, and insurers can be
distinguished from first-order green pressures from customers, investors, and workers, in that sec-
ond-order market pressures depend on the existence of first-order ones. However, second-order
market pressure from investors, lenders, and insurers that is motivated by fear of civil liability or
regulatory costs, rather than consumer or worker reactions, would fall in the third category of busi-
ness motive, regulatory threat.
11 In contrast, factors such as cost savings, customer requirements, and public pressure were all
cited by less than half the respondents (KPMG, 1994).
12 The analysis presented here was influenced significantly by the approach taken by Davies
and Mazurek (1996).
13 The 1990 Clean Air Act Amendments also mandated regulation of discharges of volatile
organic compounds (in order to achieve ground-level ozone objectives) and hazardous air pollutants,
both of which could be expected to cover all 17 33/50 chemicals.
14 ARET reports total discharges of 39.4 tonnes in participants' various base years, 26.9 tonnes
in 1993, and 13 tonnes in 1998. Thus, at least 12.5 of the 26.4 tonnes of reductions were achieved
prior to the launch of the program in early 1994 (ARET, 2000).
15 The ARET program reports that 26 percent of base year emissions are CEPA "schedule 1"
substances (ARET, 2000). The 11-percent figure was calculated using ARET base year data for the
following substances on "priority substance list 2": chloroform, ethylene oxide, formaldehyde,
acetaldehyde, 1,3-butadiene, phenol, and acrylonitrile.
OCR for page 280
280 CHALLENGES IN EVALUATING VOLUNTARY ENVIRONMENTAL PROGRAMS
16 Gallon (1998) notes that the 90 percent reduction in sulfur dioxide emissions claimed by
INCO's Sudbury smelter under the ARET program were legally mandated.
17 Such an adjustment still does not take into account patterns of dilution or human exposure
leading to chronic human health impacts, nor acute impacts on human health or environmental
impacts.
18 Although such programs do not legally require participation, compliance with monitoring and
reporting requirements can be made a condition for attaining program incentives (whether recogni-
tion or regulatory relief).
19 See, for example, information about the National Database on Environmental Management
Systems (IDEMS) project (http://www.eli.org/isopilots.htm).
20 Obstacles to causal inference and statistical techniques that attempt to address those problems
in the case of quasi-experiments involving nonequivalent groups are discussed in Chapters 3 and 4 of
Cook and Campbell (1979).
REFERENCES
Andrews, R.N.L.
1998 Environmental regulation and business 'self-regulation.' Policy Sciences 31: 177- 197.
Accelerated Reduction/Elimination of Toxics
2000 Environmental Leaders 3 Update. Ottawa: ARET.
Arora, S., and T.N. Cason
1996 Why do firms volunteer to exceed environmental regulations? Understanding participa-
tion in EPA's 33/50 Program. Land Economics 72(4):413-432.
Ayres, I., and J. Braithwaite
1992 Responsive Regulation: Transcending the Deregulation Debate. New York: Oxford
Beardsley, D.
University Press.
1996 Incentives for Environmental Improvement: An Assessment of Selected Innovative Pro-
grams in the States and Europe. Washington, DC: Global Environmental Management
Initiative.
Borkey, P., and F. Levesque
1998 Voluntary Approaches for Environmental Protection in the European Union. Paris:
Organization for Economic Co-operation and Development.
Cook, Thomas D., and D.T. Campbell
1979 Quasi-Experimentation:DesignandAnalysisIssuesforFieldSettings. Boston: Hough-
ton Mifflin.
Davies, J.C., and J. Mazurek
1996 Industry Incentives for Environmental Improvement: Evaluation of U.S. Federal Initia-
tives. Unpublished manuscript prepared for the Global Environmental Initiative, Wash-
ington, DC.
Dietz, T., and R.W. Rycroft
1987 The Risk Professionals. New York: Russell Sage Foundation
European Commission
1996 Communication from the Commission to the Council and the European Parliament on
Environmental Agreements. COM(96) 561.
European Commission Directorate
1997 Study on Voluntary Agreements Concluded Between Industry and Public Authorities in
the Field of the Environment. General III.01 - Industry. Unpublished final report, Brus-
sels, Belg., January.
OCR for page 281
KATHRYN HARRISON
European Environment Agency
281
1997 Environmental Agreements: Environmental Effectiveness. Copenhagen, Den.: European
Environment Agency.
Fung, A., and D. O'Rourke
2000 Reinventing environmental regulation from the grassroots up: Explaining and expanding
the success of the tonics release inventory. Environmental Management 25:115-127.
Gallon, G.
1998 Accuracy is optional in reporting voluntary success. Alternatives 24:12.
Gibson, R.B.
1999 Conclusion. In Voluntary Initiatives: The New Politics of Corporate Greening, R.
Gibson, ed. Peterborough, Ontario, Can.: Broadview.
Glachant, M.
1994 The setting of voluntary agreements between industry and government: Bargaining and
efficiency. Business Strategy and the Environment 3(2):43-49.
Gormley, W.T.
2000 Environmental Performance Measures in a Federal System. Unpublished paper pre-
sented at the Annual Meeting of the Association for Public Policy and Management.
Gunningham, N., and P. Grabosky
1998 Smart Regulation: Designing Environmental Policy. New York: Oxford.
Hamilton, G.
1993 Kronos, two executives charged with polluting St. Lawrence. The Gazette, March 9, p.
A4.
Harrison, K.
1995 Is cooperation the answer? Canadian environmental enforcement in comparative con-
text. Journal of Policy Analysis and Management. 14:221-244.
1996 The regulator's dilemma: Regulation of pulp mill effluents in the Canadian federal
state. Canadian Journal of Political Science 29: 469-496.
1998 Talking with the donkey: Cooperative approaches to environmental protection. Journal
of Industrial Ecology 2(3):51-71.
Harrison, K., and W. Antweiler
2001 Information Dissemination vs. Environmental Regulation: The View from Canada's
National Pollutant Release Inventory. Working Paper. Vancouver: Centre for Research
on Economic and Social Policy, University of British Columbia.
Hearne, S.A.
1996 Tracking tonics: Chemical use end the public's 'right-to-know.' Environment 38:5-9
28-34.
Keating, K.M.
1989 Self-selection: Are we beating a dead horse? Evaluation and Program Planning 12: 137-
142.
Khanna, M., and L.A. Damon
1999 EPA's voluntary 33/50 program: Impact on toxic releases and economic performance
of firms. Journal of Environmental Economics and Management 37: 1-25.
KPMG
1994 Canadian Governmental Management Survey. Toronto, Can.: KPMG.
Leiss W. and Associates
Mazurek, J.
1996 Lessons Learned from ARET: A Qualitative Survey of Perceptions of Stakeholders.
Final Report. Working Paper Series 96-4. Ontario, Can.: Environmental Policy Unit,
School of Policy Studies, Queen's University.
1998 The Use of Unilateral Agreements in the United States: The Responsible Care Initia-
tive. ENV/EPOC/GEEI(98)25/FINAL. Paris: Organization for Economic Co-operation
and Development.
OCR for page 282
282 CHALLENGES IN EVALUATING VOLUNTARY ENVIRONMENTAL PROGRAMS
Nash, J., and J. Ehrenfeld
1997 Codes of environmental management practice: Assessing their potential as a tool for
change. Annual Review of Energy and Environment 22:487-535.
Natan, T., Jr., and C.G. Miller
1998 Are toxic release inventory reductions real? Environmental Science and Technology
3(8):368A-374A.
National Research Council
1996 Fostering Industry-Initiated Environmental Protection Efforts. Washington, DC: Na-
tional Academy Press.
Organization for Economic Co-operation and Development
Storey, M.
1996 Demand Side Efficiency: Voluntary Agreements with Industry. Annex I Expert Group
on the UN FCCC, supported by the Organization for Economic Co-operation and De-
velopment and IEA, December.
Tietenberg, T.
1998 Disclosure strategies for pollution control. Environmental and Resource Economics
1 1 :587-602.
US Environmental Protection Agency
1999 33/SO Program: The Final Record. EPA-745-R-99-004. Washington, DC: U.S. Envi-
ronmental Protection Agency.
2001 The United States Experience with Economic Incentives for Protecting the Environ-
ment. EPA-240-R-01-001. Washington, DC: U.S. Environmental Protection Agency.
Weinstein, R., R. Scott, and C. Jones
1989 Measurement of 'free-riders' in energy conservation programs. Evaluation and Pro-
gram Planning 12:121-130.
1996 Pollutant Release and Transfer Registers (PRTRS): A Tool for Environmental Policy
and Sustainable Development. Guidance Manual for Governments. OCDEIGD(96)32.
Paris: Organization for Economic Co-operation and Development.
1999 Voluntary Approaches for Environmental Policy: An Assessment. Paris: Organization
for Economic Co-operation and Development.
O'Toole, L.J., Jr., C. Yu, J. Cooley, and G. Cowie
1997 Reducing toxic chemical releases and transfers: Explaining outcomes for a voluntary
program. Policy Studies Journal 25(1): 11-26.
Picard, A.
1992 Plant to shut down for ignoring clean-up orders. Globe and Mail, May 30, p.A2.
Rennings, K., L. Brockmann, and H. Bergmann
1997 Voluntary agreements in environmental protection: Experiences in Germany and future
perspectives. Business Strategy and the Environment 6:245-263.
Santos, S.L., V.T. Covello, and D.B. McCallum
1996 Industry response to SARA Title III: Pollution prevention, risk reduction, and risk
communication. Risk Analysis 16:57-66.
Shapiro, M.
1998 The Power of Information: Community Exposure and the Efficacy of State Information
Regulation Policy. Unpublished paper presented at the Annual Meeting of the Ameri-
can Political Science Association, New York, October.
1999 Equity and Information: Risks from Toxic Chemicals, Environmental Justice, and Infor-
mation Regulation Policy. Unpublished paper presented at the Annual Meeting of the
Association for Public Policy Analysis and Management, Washington, DC, November.
Representative terms from entire chapter:
voluntary program
