Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 237
Appendix F
Methods to Support Decision Making
We highlight here four approaches to formalizing decision making and
the rationalization of decisions that have been useful in complex water
resources planning. These are robust decision making, collaborative model-
ing for decision support, decision scaling, and joint fact finding. We present
these as examples of formal frameworks for tracking and understanding
decisions in complex situations. We recommend that these, and others, be
evaluated and some version (or perhaps a hybrid) be adopted for the delta.
These approaches are frameworks that include a transparent procedure
with a series of structured linkages and steps. Some of these steps include
the use of statistical and numerical models, some of which already exist for
the delta and others that would need to be developed should one of these
approaches be adopted.
ROBUST DECISION MAKING
Robust decision making (RDM) (Lempert et al. 2003, 2006, Groves
and Lempert 2007) is a quantitative, scenario-based method for identifying
policies (or strategies) that are relatively insensitive to poorly understood
uncertainty. Instead of developing a single and potentially contested proba-
bilistic forecast and associated optimal solution, RDM evaluates candi-
date solutions against large ensembles of possible outcomes to illuminate
critical vulnerabilities and suggest approaches for increasing the strategies'
robustness. RDM has been applied to problems related to climate change
mitigation or adaptation in a variety of different contexts, including global
sustainability (Lempert et al. 2004, 2006) and long-term water planning
237
OCR for page 238
238 SUSTAINABLE WATER MANAGEMENT IN THE DELTA
(Groves and Lempert 2007, Groves et al. 2008a, 2008b, 2008c, 2008d).
It has been a useful framework for developing robust climate adaptation
strategies for water agencies. Key challenges to deploying RDM include
retooling existing models to be evaluated many more times than is typical,
deploying new and often unfamiliar statistical approaches for identifying
vulnerabilities, and ensuring that decision makers and stakeholders under-
stand the new approach.
RDM proceeds through a series of steps that can be customized de-
pending on the application. In the first step, analysts, often in conjunction
with stakeholders and decision makers, specify the key uncertain exogenous
factors (X) that are likely to be disputed by different parties to the decision,
draw up a list of policy levers (L) that comprise strategies, identify measures
(M) to consider when evaluating policy outcomes, and identify models and/
or relationships (R) that relate the uncertainties and strategies to outcomes.
The resulting information, termed an "XLRM" chart, is used to assemble
the quantitative models to be used to evaluate the performance of strategies
under many alternative scenarios.
The resulting analysis is not used to identify a single "optimal" strategy.
Instead, one or a few strategies are identified for a structured evaluation
of their performance against a wide array of plausible scenarios (steps 2
and 3). In the fourth step, statistical tools are used to identify the key vul-
nerabilities, or sets of assumptions that lead the proposed strategy to fail.
These vulnerabilities thus represent future conditions (or scenarios) that
are critically important to the choice of strategies--they are the conditions
that might lead the promising strategy to perform poorly. Under these
conditions, alternative strategies would be preferred. The trade-offs among
alternatives under these vulnerable conditions can be helpful in identifying
new hedging options that can then be used to develop more robust strate-
gies. These more robust strategies are then evaluated as before. Through
iteration, RDM helps the analyst explore across a broad range of possible
strategies without requiring the contentious specification of uncertain future
parameters. The strategies identified become more robust, thus reducing the
sensitivity of the strategy's performance to the key uncertainties.
In contrast to probabilistic assessments, which typically provide rank-
ings of strategies based on a set of underlying assumptions about climate
change, RDM identifies the key uncertainties relevant to the choice of strat-
egy and then provides trade-off curves that enable decision makers to assess
the implications of different expectations of the key uncertainties to their
choices. This information has been compelling to stakeholders and decision
makers when evaluating climate change impacts on water-management
systems (Groves et al. 2008c).
OCR for page 239
APPENDIX F 239
COLLABORATIVE MODELING FOR DECISION SUPPORT
To evaluate alternative delta scenarios it would be helpful to have a
multifaceted analysis that could address scarcity economics, water market
prices, energy utilization, and alternatives for adaptive management. Collab-
orative modeling for decision support (abbreviated CMDS or COMODES)
is the "generic" (Cardwell 2011) name given to a suite of techniques that
can be used to achieve consensus on complex, contentious issues. Indeed,
Lorie (2010) defined CMDS as "integrating collaborative modeling with
participatory processes to inform natural resource management decisions."
CMDS is an approach to reach consensus and make decisions about com-
plex systems that combines technical skills required to understand the
systems scientifically and stakeholder involvement (Cockerill et al. 2006,
Langsdale et al. 2011). With respect to stakeholder involvement, process
skills such as an appreciation of institutional setting and ability to engage
stakeholders and build their trust are essential (Langsdale et al. 2011).
Various "brand names" of CDMS are Shared Vision Planning (SVP),
the brand of CMDS used by the Institute for Water Resources (IWR) of the
U.S. Army Corps of Engineers (Cardwell et al. 2008); Computer-Assisted
Dispute Resolution or CADRe (Stephenson et al. 2007); or mediated mod-
eling (van den Belt 2004). Although CMDS has been practiced in one form
or anther since the late 1980s (Langsdale et al. 2011), only recently has
there been specification of guiding principles and best practices.
Langsdale et al. (2011) listed eight guiding principles:
1.Collaborative modeling is appropriate for complex, conflict-laden
decision making processes where stakeholders are willing to work
together.
2.All stakeholder representatives participate early and often to ensure
that all their relevant interests are included.
3.Both the analysis and the process remain accessible and transparent
to all participants.
4.Collaborative modeling builds trust and respect among parties.
5.The analysis supports the decision process by easily accommodating
new information and quickly simulating alternatives.
6. The analysis addresses questions that are important to decision mak-
ers and stakeholders.
7. Parties share interests and clarify the facts before negotiating
alternatives.
8.Collaborative modeling requires both modeling and facilitation
skills.
OCR for page 240
240 SUSTAINABLE WATER MANAGEMENT IN THE DELTA
One aspect of CMDS that can perplex sophisticated modelers is the
premise that stakeholders, many of whom have little or no experience with
either the development or application of simulation models, will be active
participants in the modeling process. For a system as complex as the delta,
this may seem to be an impossible situation. Langsdale et al. (2011) offer
some guidance on this.
Often, system dynamics (SD) modeling techniques are applied to con-
duct collaborative modeling studies because they allow participants to
examine complex physical systems that involve social and economic factors
involved (Cockerill et al. 2006).
This section would be incomplete without addressing the prospects for
consensus that collaborative modeling seeks to achieve. Madani and Lund
(2012) have traced changes in the delta conflict in the context of game
theory and suggest that the conflict has evolved with time from cooperation
to "chicken." In the early 20th century, stakeholders agreed to cooperative
solutions; later on, fights over water allocations led to stakeholders com-
peting as opposed to cooperating (Madani and Lund 2012). They do state
that a win-win resolution may be possible but that a cooperative solution
is unlikely without external influence.
Indeed, they conclude their paper with the following:
Including the state of California (or federal government) did not funda-
mentally alter the game. For the cases examined, the Chicken charac-
teristics remained and cooperation was unlikely. Adding the state to the
game suggested that California can be the victim and loser in the conflict,
bearing much of the cost of a Delta failure, due to its past failure so far to
develop reliable mechanisms to enforce cooperation.
Whatever plan is adopted to fix the Delta in the coming decades, the
Delta's sustainability is not guaranteed without powerful mechanisms
which provide incentives for cooperation or penalties for deviation from
cooperation. While recent efforts address symptoms of the problem, they
have not yet solved a main cause - lack of effective and responsive govern-
ing mechanisms. California must "govern" the Delta or pay for absence
of effective governance.
The prospect for achieving consensus, whether by collaborative model-
ing or some other means, is a daunting task.
Collaborative Modeling in the Delta
Two episodes in the recent history of delta management illustrate the
value of collaborative modeling. The first of these was the development of
flow standards by the U.S. Environmental Protection Agency (EPA) in the
period 1992-1994, which began with the 1992 EPA workshops (Schubel
OCR for page 241
APPENDIX F 241
et al. 1993, Kimmerer and Schubel 1994). The key step in translating the
conclusions of this workshop into a workable standard for flow, in this case
based on the position of X2, was modeling used to understand the water-
supply recommendations of the standard. This was done through a collabo-
ration between a regulatory agency (the EPA) and by engineers from the
Contra Costa Water District acting on behalf of the California Urban Water
Agencies (CUWA), an organization of stakeholders who would have been
affected by the regulation (R. Denton, personal communication, 2012). In
the end, the EPA X2 regulations as modified by CUWA were adopted as the
1994 Bay-Delta Accord, an agreement that helped lead to the establishment
of CALFED (Rieke 1996, Hanemann and Dyckman 2009).
The second episode was the gaming carried out to design the Envi-
ronmental Water Account (EWA). In this case, a group of regulators, con-
sultants, and representatives of water agencies and environmental groups
explored the water-supply implications of different-size EWAs using the wa-
ter resources system model CALSIM (Brown et al. 2004, Booher and Innes
2010). Using historical salvage data, this gaming was used to developing
strategies for deploying EWA assets in order to have maximal effect (Brown
et al. 2009). Importantly, as Connick and Innes (2003) write,
it [the EWA] would not have been even imaginable without the trust and
co-operation of the stakeholders. Moreover the details could not have been
worked out without this social capital. Agency personnel and stakehold-
ers from agricultural and urban water interests and environmental groups
spent hundreds of hours working through various scenarios to test how
the approach could be used before recommending that it be part of the
CALFED programme.
Thus, the aims of these collaborative efforts were in some ways modest;
that is, the outcomes of the modeling they used were relatively straightfor-
ward, being focused on water operations and their effect on the physical
environment. Nonetheless, the committee views them as examples worth
emulating in future efforts to manage the delta ecosystem.
Collaborative Modeling in Everglades Restoration
In 1993, the U.S. Army Corps of Engineers (USACE), in partnership
with the South Florida Water Management District (SFWMD) and other
stakeholders, initiated the Comprehensive Review Study of the Central
and Southern Florida (C&SF) Project. This study, commonly called "the
Restudy," was intended to integrate solutions which when implemented will
enhance the ecological values of the Florida Everglades by increasing the
total spatial extent of natural areas, improving the habitat and functional
quality, plant and animal species abundance, and diversity. Another objec-
OCR for page 242
242 SUSTAINABLE WATER MANAGEMENT IN THE DELTA
tive was the enhancement of economic values and social well-being through
increase of the availability of freshwater for agricultural, municipal, and
industrial users, reduction of flood damages, provision of recreational and
navigational opportunities, and protection of cultural and archeological
resources and values.
The Restudy followed a transparent, multiagency, participatory, and
highly iterative process with a strong collaborative modeling component
for the development of the Comprehensive Everglades Restoration Plan
(CERP). The core Restudy team of analysts consisted of multidisciplinary
professionals from numerous federal, state, local, and tribal organiza-
tions, and subteams for modeling, alternatives design, alternative analysis,
and public involvement. The Restudy's success in meeting deadlines and
consensus building required the use of a large team consisting of over 150
individuals from 30 different public entities representing over 20 different
professional disciplines. The modeling team relied heavily on the use of
several hydrologic, ecological, and water-quality simulation models and
expert judgment.
Plan formulation began by developing a list of many different ideas
to achieve goals and objectives. The ideas, called "components," were the
individual building blocks that were combined in various ways to form al-
ternative plans that included both structural and nonstructural features. In
each iteration, alternative plans were formulated by the Alternative Design
Team (ADT) and modeled by the Modeling Team. The designs of the alter-
native plans were built into the South Florida Water Management Model
(SFWMM), a regional-scale hydrologic model, for performance evaluation
and to provide input to other models in the toolbox. The modeling output
was used to produce a large suite of performance measures that had been
developed from conceptual models of the major landscapes and water-
supply planning efforts. Each alternative plan was evaluated by another
multiagency team called the Alternative Evaluation Team (AET), which
incorporated comments from different agencies and the public, together
with their own evaluation to make recommendations to the ADT for the
next iteration. The AET was responsible for evaluating each plan's strengths
and weaknesses, and describing plan shortfalls to the ADT. This repetitive
formulation and evaluation process progressively refined and improved
the performance of subsequent alternative plans. Because of the large and
geographically dispersed number of people involved and interested in the
Restudy, the Internet was used to communicate formulation and evaluation
results. This allowed the Restudy team to solicit comments from a broad
base of the public and permitted people to participate as team decisions
were being made.
The collaborative modeling effort continues today through a newly cre-
ated Interagency Modeling Center (IMC) with key leadership of sponsoring
OCR for page 243
APPENDIX F 243
agencies and participation by others. It is a single point of service for the
modeling needs of CERP projects and programs and provides coordination,
review of other modeling efforts. Through interagency collaboration IMC
acts as a clearinghouse for all project-specific modeling and conducts its
own regional-scale analysis.1
DECISION SCALING
Brown et al. (2011) have recently described an alternative approach
to decision making under climate change that may be applicable to the
delta. Rather than begin with climate change predictions and their asso-
ciated uncertainty downscaled to the problem at hand, the concept is to
turn traditional decision analysis around and start by identifying which
uncertainties are important from the viewpoint of the decision maker. In
the case of climate change, the framework facilitates the identification of
climate information that is critical to the planning decision. As a result,
decision analysis provides an analytic framework that can be exploited to
link bottom-up climate vulnerability analysis with the generation of climate
change projections. The process is entitled "decision scaling."
The key tenet of the approach is that the appropriate orientation for
adaptation planning is one of acceptance of large uncertainties and plan-
ning for a wide variety of possible futures. This runs contrary to the general
scientific orientation of focusing on the reduction of uncertainty and then
planning for the accepted expert characterization of the future. Instead, the
approach emphasizes robustness over a wide range of climate futures. It has
been applied to the development of a regulation plan for the Upper Great
Lakes (Brown et al. 2011). The regulation plan utilizes dynamic responses
to evolving conditions and adaptive management of uncertainties and sur-
prise. However, Brown et al. present a general process for water resources
planning under climate change (or any other uncertainty for which a va-
riety of predictions are possible) based on a decision-analytic approach to
identifying and tailoring the necessary information. The framework links
insight from bottom-up analysis, including performance metrics defined by
stakeholders with the processing of, in the Great Lakes example, climate
change projections to produce decision-critical information.
A key aspect of decision scaling is that the specification of the climate
states, that is, the specific climate information that causes a particular
decision to be favored over another (or an impact to be large enough to
warrant preventative actions, i.e., the identification of thresholds), may al-
low the credibility of climate information derived from GCM projections
(or other sources) to be improved. That is, with the information from the
1 See www.evergladesplan.org. Accessed July 17, 2012.
OCR for page 244
244 SUSTAINABLE WATER MANAGEMENT IN THE DELTA
bottom-up, decision-analytic framework in hand, the generation of climate
information may be tailored to best provide credible information through
the selection of process models, temporal and spatial scales, and scaling
techniques given the time.
The approach begins with stakeholders rather than predictive system
models. Planners ask stakeholders and resource experts what conditions
they could cope with and which would require substantial policy or in-
vestment shifts. This is then formalized within a framework that links the
multiple models needed to relate changes in the physical climate conditions
to performance metrics of interest to stakeholders. After these are estab-
lished, hydrologists and climate scientists estimate the plausibility of the
water conditions that exceed the coping thresholds, taking into account
not only climate change but also natural climate variability and stochastic
variability observed with a stationary climate assumption. While the exist-
ing applications of decision scaling focus on uncertainties associated with
climate change, the approach could be adjusted to consider other uncer-
tainties that are key in the bay delta including consumption patterns and
environmental factors.
Joint Fact-Finding in Bay-Delta Science
The products of the delta science process involve at least three science
efforts: one carried out by wildlife agencies, one by water users, and a
third effort by professional environmental organizations. They each involve
many scientists, including agency staff, academics, consulting firms, and
individual experts with national reputations, and are carried out largely
separately. There are fundamental disagreements. Each attempts to present
the objective truth on a variety of issues. While there are several forms for
collaboration, there does not seem to be a format for resolving professional
scientific differences of opinion.
A process called "joint fact-finding" may be of value. Ehrmann and
Stinson's seminal chapter in the Consensus Building Handbook: A Com-
prehensive Guide to Reaching Agreement, describes the process as follows:
"Joint fact-finding" offers an alternative to the process of "adversary sci-
ence" [what has been, perhaps inappropriately, termed, "combat science"
in this estuary] when important technical or science-intensive issues are at
stake. Joint fact-finding is a central component of many consensus build-
ing processes; it extends the interest-based, cooperative efforts of parties
engaged in consensus building into the realm of information gathering
and scientific analysis. In joint fact-finding, stakeholders with differing
viewpoints and interests work together to develop data and information,
analyze facts and forecasts, develop common assumptions and informed
OCR for page 245
APPENDIX F 245
opinion and, finally, use the information they have developed to reach
decisions together.
Several references describe the important features of joint fact-finding
(see Ehrmann and Stinson 1999, Karl et al. 20072), which can be sum-
marized as
· participation by all parties with interest and scientific contributions
to make;
· use of a neutral, expert facilitator to manage the process;
· identification of key scientific questions to be addressed by the
process;
· development of an agreed-on process for answering the questions;
and
· carrying out that process and jointly evaluating the results.
Although this process might rely in part on outside, independent ex-
perts, it primarily involves the disputing experts, those with the most at
stake, those whose ultimate buy-in is necessary to resolve or narrow scien-
tific disputes. It certainly is true that without joint fact-finding, long-held
positions can change. As Kuhn (1970/1996) observed, "Sometimes the
convincing force is just time itself and the human toll it takes," or as Kuhn
quoted Max Planck, "a new scientific truth does not triumph by convincing
its opponents and making them see the light, but rather because its oppo-
nents eventually die, and a new generation grows up that is familiar with
it." One purpose of joint fact-finding is to speed this process and make its
outcomes relevant to decisions that will be made soon.
Where "joint fact-finding" can run awry is the premise sometimes put
forward by the advocates of that concept that once the facts are on the table
then the scientists can "resolve the issue." It is very important that the goal
of the "jointly evaluating results" segment is clear. Clear "resolution" of a
complex problem is rarely how science works (see Chapter 5). For example,
ranking stressors certainly has many policy benefits, but it is a simplification
that, if resolved by a joint fact-finding panel, would be turned over by the
next panel, ad infinitum. The benefit of properly focused joint fact-finding
is broad involvement of many parties in the scientific dialogue. Adversary
science can be minimized for the purposes of dialogue, if the immediate
discussion of a workshop, for example, is constrained to defining the state
of the science, defining where disagreements exist and what they are, and
2 For additional information on joint fact-finding, see http://ocw.mit.edu/courses/urban-
studies-and-planning/11-941-use-of-joint-fact-finding-in-science-intensive-policy-disputes-
part-i-fall-2003/readings/.
OCR for page 246
246 SUSTAINABLE WATER MANAGEMENT IN THE DELTA
deciding on the path forward and/or what the policy choices are. If the
dialogue is allowed to turn into an argument about who is right and who
is wrong, constructive progress is lost; this is where the court cases have
taken California to today. A dialogue in which public events are focused
on constructive progress, if given time and supported by the policy com-
munity, (a) helps develop at least some commonalities in views of the state
of the science among adversaries; (b) points out where new work is needed
as agreed on by all parties; (c) can smooth the waters of conflict by provid-
ing a nonadversarial forum in which people from different sides can find at
least some subjects on which they can agree; and (d) improves public trust
if the dialogue takes place in public forums. The Science Program of the
Delta Stewardship Council has a history of attempting to build such a dia-
logue. In a speech in 2002 Secretary of Resources Mary Nichols suggested
this approach was gaining traction with policy makers. It appears that
Madani and Lund's game of "chicken" reasserted itself after CALFED was
declared a failure in 2004. But there is still an undercurrent of constructive
scientific dialogue taking place, sponsored by the Science Program of the
Delta Stewardship Council, from which there are opportunities to build if
given support.
A return to an enthusiastic joint, constructive scientific dialogue, per-
haps mediated by independent experts, might possibly be an ingredient that
could help bridge what is now an ever-widening gap between key interest
groups. Seeking points of agreement among adversaries, even if only over
the science, would be a step toward consensus about at least some aspects
of important science-driven policy issues and their uncertainties. This small,
easily implemented change could begin to improve public trust, placing
decision making on firmer ground. It is a process that can provide a more
timely result than that which might occur by waiting for the professional
demise of leading proponents of the opposing viewpoints. It is a process
whose application to the delta science process is long overdue.
REFERENCES
Booher, D. E., and J. E. Innes. 2010. Governance for resilience: CALFED as a complex adap-
tive network for resource management. Ecology and Society 15(3):35. Available at http://
www.ecologyandsociety.org/vol15/iss3/art35/. Accessed June 4, 2012.
Brown, C., W. Werick, W. Leger, and D. Fay. 2011. A decision-analytic approach to manag-
ing climate risks: Application to the upper great lakes. Journal of the American Water
Resources Association 47:524-534.
Brown, L., W. Kimmerer, and R. Brown. 2009. Managing water to protect fish: A review
of California's environmental water account, 2001-2005. Environmental Management
43:357-368, doi:10.1007/s00267-008-9213-4.
OCR for page 247
APPENDIX F 247
Brown, R., T. Cannon, D. Fullerton, and B. Herbold. 2004. California (CALFED) daily en-
vironmental management modeling for fish protection and water supply evaluation. In
Building Partnerships: Proceedings of Joint Conference on Water Resource Engineering
and Water Resources Planning and Management 2000, July 30-August 2, 2000, Min-
neapolis, MN, edited by R. H. Hotchkiss and M. Glade. ASCE Publications.
Cardwell, H., 2011. Introduction. In Converging Waters: Integrating Collaborative Modeling
with Participatory Processes to Make Water Resources Decisions, edited by L. Bourget.
Alexandria, VA: Institute for Water Resources, U.S. Amry Corps of Engineers, 218 pp.
Cardwell, H., S. Langsdale, and K. Stephenson. 2008. The Shared Vision Planning Primer:
How to Incorporate Computer Aided Dispute Resolution in Water Resources Planning.
Alexandria, VA: Institute for Water Resources (IWR) Report No. 08-R--2. www.iwr.
usace.army.mil/docs/iwrreports/2008-R-02.pdf. Accessed July 20, 2011.
Cockerill, K., H. Passell, and V. Tidwell. 2006. Cooperative modeling: Building bridges
between science and the public. Journal of the American Water Resources Association
42:457-471.
Connick, S., and J. E. Innes. 2003. Outcomes of collaborative water policy making: Applying
complexity thinking to evaluation. Journal of Environmental Planning and Management
46(2):177-197.
Ehrmann, J. R., and B. L. Stinson. 1999. Joint fact-finding and the use of technical experts. In
The Consensus Building Handbook: A Comprehensive Guide to Reaching Agreement.
Cambridge, MA: The Consensus Building Institute.
Groves, D. G., M. Davis, R. Wilkinson, and R. Lempert. 2008a. Planning for climate change
in the inland empire: Southern California. Water Resources IMPACT, July.
Groves, D. G., D. Knopman, R. Lempert, S. Berry, and L. Wainfan. 2008b. Presenting Uncer-
tainty About Climate Change to Water Resource Managers--Summary of Workshops
with the Inland Empire Utilities Agency. TR-505. Santa Monica, CA: RAND.
Groves, D.G., and R.J. Lempert. 2007. A new analytic method for finding policy-relevant
scenarios. Global Environmental Change 17:73-85.
Groves, D. G., R. Lempert, D. Knopman, and S. Berry. 2008c. Preparing for an Uncertain
Future Climate in the Inland Empire--Identifying Robust Water Management Strategies.
DB-550. Santa Monica, CA: RAND.
Groves, D. G., D. Yates, and C. Tebaldi. 2008d. Developing and applying uncertain global
climate change projections for regional water management planning. Water Resources
Research 44:W12413, doi:10.1029/2008WR006964.
Hanemann, M., and C. Dyckman. 2009. The San Francisco Bay-Delta: A failure of decision-
making capacity. Environmental Science and Policy 12:710-725.
Karl, H. A., L. E. Susskind, and K. H. Wallace. 2007. A dialogue not a diatribe. Effective
integration of science and policy through joint fact finding. Environment 49(1):20-34.
Kimmerer, W. J., and J. R. Schubel. 1994. Managing freshwater flows into San Francisco Bay
using a salinity standard: Results of a workshop. Pp. 411-416 in Changes in Fluxes in
Estuaries, edited by K. R. Dyer and R. J. Orth. Fredensborg, Denmark: Olsen and Olsen.
Kuhn, T. (1970/1996). The Structure of Scientific Revolutions, 3rd Edition. Chicago: Univer-
sity of Chicago Press.
Langsdale, S., A. Beall, L. Bourget, E. Hagen, S. Kudlas, R. Palmer, D. Tate, and B. Werrick.
2011. Collaborative Modeling for Decision Support in Water Resources: Principles and
Best Practices. Available at http://www.computeraideddisputeresolution.us/bestpractices.
Accessed 21 July 2011.
Lempert, R.J., D.G. Groves, S.W. Popper, and S.C. Bankes. 2006. A general, analytic method
for generating robust strategies and narrative scenarios. Management Sciences 52(4):
514-528.
OCR for page 248
248 SUSTAINABLE WATER MANAGEMENT IN THE DELTA
Lempert, R., N. Nakicenovic, D. Sarewitz, and M. Schlesinger. 2004. Characterizing climate-
change uncertainties for decision-makers. Climatic Change 65:1-9.
Lempert, R. J., S.W. Popper, and S. C. Bankes. 2003. Shaping the Next One Hundred Years:
New Methods for Quantitative, Long-Term Policy Analysis. Santa Monica, CA: RAND.
Lorie, M. 2010. Computer-Aided Dispute Resolution 2nd Workshop: Summary and Strategic
Plan. Alexandria, VA: Institute for Water Resources (IWR) Report No. 2010-R-5.
Madani, K., and J. R. Lund. 2012. California's Sacramento-San Joaquin Delta conflict: From
cooperation to chicken. Journal of Water Resources Planning and Management 138 (2):
90-99, doi:10.1061/(ASCE)WR.1943-5452.0000164.
Rieke, E. A. 1996. The Bay-Delta Accord: A stride toward sustainability. Colorado Law
Review 67:341-69.S.
Schubel, J. R., et al. 1993. Managing Freshwater Discharge to the San Francisco Bay/
SacramentoSan Joaquin Delta Estuary: The Scientific Basis for an Estuarine Standard.
San Francisco Estuary Project, U.S. Environmental Protection Agency, San Francisco,
CA, 109 pp.
Stephenson, K., L. Shabman, S. Langsdale, and H. Cardwell. 2007. Computer Aided Dispute
Resolution: Proceedings from the CADRe Workshop. Alexandria, VA: Institute for Water
Resources, IWR Report No. 07-R6. Available at www.sharedvisionplanning.us/docs/SVP
---2007---R- --06.pdf. Accessed July 22, 2011.
Van den Belt, M., 2004. Mediated Modeling: A System Dynamics Approach to Environmental
Consensus Building. Covelo, CA: Island Press, 296 pp.
