|
Items in cart [0] |
Questions? Call 888-624-8373 |
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 122
4
Macro-Level Formal Models
T
his chapter presents modeling approaches for representing the
behavior of humans in groups and organizations. It discusses system
dynamics models first, followed by a discussion of several approaches
to organizational modeling.
SySTEM DyNAMICS MODELS
What Is System Dynamics Modeling?
System dynamics modeling is a method of modeling the dynamic
behavior of complex systems by breaking down these systems into sim-
pler interconnected components (“blocks”) connected together via links or
“wires” that connect one block’s outputs to another block’s inputs. This
breaking down or recursive modeling continues until simple blocks can
be defined in terms of well-understood interactions between the block’s
inputs, outputs, and its “internal state.” Within any given block, this state
is defined by the associated state variables, which are usually related by a
set of differential equations that underlie the dynamics of that block.1
To provide a quick illustration of the basic concepts involved, if
one were to model the dynamics of two cars traveling down a straight
road, one behind the other, one might specify four blocks: one for each car
and one for each driver. Each car would have (a) two states: a speed and a
1 The use of differential equations reflects the history of system dynamics modeling and its
roots in electrical and mechanical engineering and control systems theory.
OCR for page 123
MACRO-LEVEL FORMAL MODELS
position/location down the road; (b) a single input (or control) of accelera-
tion, determined by the driver’s application of the gas or brake pedal; and
(c) a single output, the position/location down the road.2 Simple differential
equations, based on the laws of physics (and the vehicle acceleration/braking
dynamics) would then be used to define the relation of the input (control)
of the driver’s use of the gas pedal or brake to the car’s output, the position
down the road. The second car would be modeled similarly. The trailing
car driver would be likewise modeled as a block, with perhaps two inputs,
distance and closing speed to the front car, and a single output, gas/brake
pedal usage. The differential equations or “control law” relating driver
inputs to driver outputs would be specified by well-understood manual
control dynamics (see, for example, McRuer and Krendel, 1974). The lead
driver could be modeled in “open-loop” fashion, as a block with no input
but with a randomly varying output of gas pedal pressure, leading to ran-
dom speed behavior. By specifying each individual block’s behavior (via the
inputs, the outputs, and the differential equations underlying the internal
dynamics) and by linking up the appropriate inputs to the appropriate
outputs of the four-block system, one then has a general system dynamics
representation of the dynamics of the two-car, two-driver “system.”
The fundamental power of this approach lies in four areas:
1. System dynamics concepts are tightly bound to the twin notions
of (1) the dynamic behavior of systems over time and (2) feedback
and cross-connectivity between different elements of the system.
Dynamic behavior can evolve simply because of a system’s internal
dynamics and its initial conditions (e.g., a frictionless swing set to
infinite harmonic oscillation by an initial offset from the vertical).
But the dynamic behavior is considerably more interesting when
it is driven by the dynamics of yet some other system (e.g., some-
one pumping the swing ever higher and eliciting nonlinear swing
behaviors), through a cross-coupling or feedback loop involving
real physics or abstract information. And when these loops are
contaminated by noise (an erratic “pumper”), time delays (a slow-
to-respond pumper), and/or distortion in the form of frequency- or
amplitude-selective feedback channels, then the opportunity exists
for often unanticipated and sometime surprising behaviors across
the system as a whole. These are often the characteristics of com-
2 Two states suffice for a simple kinematic representation of the longitudinal (fore-aft)
control of vehicle location; additional states would be added for finer grained representation
of the situation if one were interested in modeling the effect of the detailed dynamics of the
brake calipers, for example. The approach would be the same, however, via the introduction
of yet another block placed between the driver’s brake pedal and the block representing the
vehicle kinematics.
OCR for page 124
4 BEHAVIORAL MODELING AND SIMULATION
plex human-machine and human-human systems that modelers are
dealing with.
2. The use of blocks, which can be made up of subblocks ad infini-
tum, so that any level of detail can be examined in a given model,
within practical computational limits. Literally millions of state
variables can be introduced—in a structured manner—to allow
the finest grained examination of the impact of very small com-
ponents (e.g., O-ring brake failure) on overall system behavior
(e.g., a 20-car pileup on the Los Angeles freeway). In essence, this
approach provides one means of modeling the “butterfly effect,”
as an alternative to chaos theory, which models how small changes
in the initial state (or initial conditions) of a nonlinear system can
lead to large changes of the system state (or system trajectory) at
some later point in time.3 The systems dynamics approach takes
a bottom-up building block approach, which is appealing in its
dependence on well-understood domain-specific theory and laws,4
whereas chaos theory takes a broader systems level view that, if
more abstract, is well grounded mathematically.
3. The use of interconnected blocks ensures that the fundamentals of
feedback are (nearly) always present. In the example above, the
driving behavior of the lead driver clearly will affect the behavior
of the trailing driver.5 Thus, subtle interactions can be accounted
for, as one element of the system accounts for and accommodates
to others. It is often these feedback loops that give rise to unantici-
pated “emergent” behaviors (pilot-induced oscillations in aircraft
handling, stock market crashes, etc.).
4. The use of blocks with “internals” that can be elaborated as the
need arises. Generally, differential equations serve as the basis for
a block’s dynamics, but it is straightforward to elaborate, via either
the addition of subordinate blocks as just described or the addi-
tion of, for example, nonlinear characteristics (e.g., a limit on the
acceleration obtainable via a fully pressed-down gas pedal in the
above example). However, any such nonlinear additions often tend
3 The term “butterfly effect” was introduced by one of the pioneers of chaos theory, Edward
Lorenz, in a paper given by him in 1972 to the American Association for the Advancement of
Science in Washington, D.C., entitled Predictability: Does the Flap of a Butterfly’s Wings in
Brazil Set Off a Tornado in Texas?
4 See later comments on the limits to the system dynamics approach of building, from the
ground up, models that seem plausible at each level, until they are actually run and compared
with dramatically different real-world results.
5 And to explore the impact of the trailing driver’s behavior on the lead driver’s, one would
merely need to add in a rear-view mirror into the model of the lead driver, and postulate the
dynamics of lead driver behavior as a function of, say, trailing driver tailgating activity, thus
fully “closing the loop” between the two drivers.
OCR for page 125
5
MACRO-LEVEL FORMAL MODELS
to make the theoretical analysis of such systems intractable, so that
system dynamics analysts must then rely on simulation execution
and analysis in order to understand or predict system behavior.
A specialized version of system dynamics modeling, and the main focus
of this section, focuses on a fairly explicit representation of the system
states, called “stocks” (entities that accumulate or deplete over time) and
their associated “flows” (the rates of change of stocks) (Forrester, 1968). In
essence, Forrester6 transformed the generic nth order differential equations
characterizing general system dynamics theory into n first-order differential
equations that are intuitively simple to understand and, via the associated
programming language Dynamo, into a transparent graphic representa-
tion of the key interrelationships among variables (Richardson and Pugh,
1981). Using Dynamo to implement these first-order relations, it becomes
a relatively simple exercise in computational model development by the
nonspecialist who may not have been schooled in differential equations
and their specification or solution. Feedback and interconnections are intro-
duced by defining how the level of one stock controls the flow of another.
Nonlinearity is introduced via simple limits on stock levels and flow rates.
A simple example is given in Box 4-1, which illustrates how two states
(birth rate and death rate) define the flow of a third state (net growth rate).
This is a simple open-loop example with no feedback, but it is not a diffi-
cult exercise to close the loop, for example, by postulating how population
growth rate might influence economic growth rate, which could induce
consumer confidence and, through that, cause birth rates to increase.
An example showing this level of loop closure is given in Figure 4-1,
which illustrates one component of a larger system dynamics model of the
spread of an epidemic (Sage and Armstrong, 2000). The three state vari-
ables (stocks) are X1, the population susceptible to infection (susceptible
population), X2, the population that is actually infected (infected popula-
tion), and X3, the population that has developed an immunity to the infec-
tion (immune population). Note that boxes are used to represent these
states graphically. The associated flows are LR (loss of immunity rate), IR
(infection rate), and RR (recovery rate). Note that the valve symbols are
used to indicate how the flows control the stock levels, via the following
intuitive graphic analogy: flow into a block increases the stock level, while
6 Although Jay Forrester’s name is the one most closely associated with the system dynamics
concept, his work owes much to the electrical engineering pioneers at Bell Laboratories work-
ing with feedback circuits and notions of system stability in the 1920s and 1930s (see, e.g.,
Black, 1977); the discipline of cybernetics developed at the Massachusetts Institute of Tech-
nology by Norbert Weiner and colleagues during the 1940s and 1950s (Weiner, 1948); and,
more recently, practitioners who have done much to popularize its application to important
problems in the social sciences, most notably Richardson and colleagues (see, e.g., Richardson
and Pugh, 1981; Richardson, 1991).
OCR for page 126
BEHAVIORAL MODELING AND SIMULATION
BOX 4-1
The Equation, Variables, and Mathematical Representations for
Birth and Death Used in Population Modeling
Description of variables:
b(t) : Average birth rate per unit person in the population at time t
D(t) : Average death rate per unit person in the population at time t
mn(t) : Expected value
Mathematical representation of birth rate, death rate, and average rate of popula-
tion growth:
b(t)mn(t) : Total average birth rate
D(t)mn(t) : Total average death rate
dµn (t )
= [ β (t ) − ∆ (t )] µn (t ) : Average rate of population growth (the difference
dt
between the total average birth rate and death rate)
X1
Susceptible
Population
IR(t)
)
Infection Rate
X22
x LR(t)
)
Infected
Infected Loss of Immunity
Population Rate
population
RR(t)
)
Recovery Rate
X3
Immune
Population
FIguRE 4-1 Example of a system dynamics model that shows the partial system
dynamics description for propagation of 4-1.eps epidemic.
a potential
SOURCE: Adapted from Sage and Armstrong (2000, p. 235).
OCR for page 127
MACRO-LEVEL FORMAL MODELS
flow out decreases it.7 The diagram captures the following qualitative and,
for the mathematically inclined, quantitative notions:8
• For the states:
— The susceptible population X1 will increase as the recovered
lose immunity (LR) and decrease as the susceptibles become
infected (IR). Or9
d(X1)/dt = LR – IR
n
— The infected population X2 will increase as the susceptibles
become infected (IR) and decrease as the infected recover (RR).
Or
d(X2)/dt = IR – RR
n
— The immune population X3 will increase as the infected recover
(RR) and decrease as the immune lose immunity (LR). Or
d(X3)/dt = RR – LR
n
• For the flows (not illustrated for simplicity):
— The infection rate (IR) increases both as the susceptibles (X1)
increase and as the infected (X2) increases, due to the net-
worked nature of spreading infections. Or10
IR = a*X1*X2
n
— The recovery rate (RR) is directly proportional to the infected
(X2). Or
RR = b*X2
n
— Likewise, the loss of immunity rate (LR) is directly propor-
tional to the infected (X3). Or
LR = b*X3
n
Note the complete loop closure relating the three states, and the
potential for continuing growth and decay of an infected population over
time. Note also the potential for nonlinear behavior over time, because of
the fundamental nonlinearity introduced via the infection rate equation
(IR = a*X1*X2).
The structure of system dynamics models can be characterized by four
hierarchical levels, as shown in Figure 4-2.11 All interactions and impacts
7 Not explicitly shown is how the flows are influenced by the stock levels.
8 Note that in this set of equations and in subsequent sets, the asterisk (*) is not meant to
represent a convolution operation or function composition, but rather a simple multiplication,
in line with DYNAMO code conventions, as well as FORTRAN syntax, which was a popular
computational language at the time of DYNAMO’s introduction.
9 d( )/dt is used to denote the first-order derivative of the associated variable.
10 The constants (a,b,c) are chosen on the basis of underlying knowledge of dynamics of
infection, recovery, etc.
11 This description borrows heavily from Sage and Armstrong (2000, p. 237).
OCR for page 128
BEHAVIORAL MODELING AND SIMULATION
Closed Boundary
Around a System
Rate and Level Variables as
Basic Structural Elements
Rate Variables Representing
Level Variables Representing
Activity Within Feedback Loops
Accumulations Within
Feedback Loops
Detection of
Control or
Discrepancy
Goals or Observed Policy Action
Between Goals
Objectives Conditions Based on the
and Observed
Discrepancy
Conditions
FIguRE 4-2 The four hierarchical levels of system dynamics modeling.
4-2.eps
SOURCE: Sage and Armstrong (2000, p. 237).
in the system dynamics model take place inside a boundary. Within this
boundary, variables are chosen to represent the key states that define overall
system behavior. A derivative variable is chosen to control a flow into the
state or level variable, which integrates or accumulates this level. Informa-
tion concerning the level is used to control the rate variable (state feedback
to the same associated state). In other words, we define a rate variable as
the time derivative of a level or state variable and determine rate variables
as functions of level variables.
Some useful readings on system dynamics modeling methodology
are Roberts, Anderson, Deal, Garet, and Shaffer (1983); Sterman (2000);
Ogata (2003); and Karnopp, Margolis, and Rosenberg (2006). A more
detailed description of system dynamics modeling and the equations it uses
is available in Sage (1977) and Sage and Armstrong (2000). Comprehen-
sive approaches to modeling complex projects—including industrial and
military—are described by Williams (2002).
OCR for page 129
MACRO-LEVEL FORMAL MODELS
State of the Art in System Dynamics Modeling
Early History of System Dynamics
Jay W. Forrester created this focused version of system dynamics in
the mid- to late 1950s at the Massachusetts Institute of Technology’s Sloan
School of Management, basing it on the more traditional modeling used
at the time, implementing differential equation models on analog com-
puters. Forrester brought these concepts to the digital domain, codified
them in the stocks and flows paradigm described above, and used this
approach to model highly complex systems such as organizations and
the urban environment (Forrester, 1961; see also Forrester, 1969). This
novel approach of developing computational dynamic models of hitherto
unmodeled phenomena led to the founding of the System Dynamics Group
at the Massachusetts Institute of Technology in the early 1960s (see http://
web.mit.edu/sdg/www/what_is_sd.html).
Forrester wrote several books on system dynamics methodology that
provide the foundations of the field. The first was Industrial Dynamics
(Forrester, 1961), providing a computational foundation for understand-
ing the dynamics of organizations and processes in industry. Forrester
then published Urban Dynamics (1969), which was the first noncorporate
application of system dynamics (Radzicki, 1997). Shortly thereafter For-
rester published World Dynamics (1971) in which he applied system
dynamics methodology to the behavior of the highly interrelated forces of
global dynamics (Sage and Armstrong, 2000). Forrester’s student, Dennis
Meadows, and colleagues expanded on World Dynamics in The Limits to
Growth (Meadows, Meadows, Randers, and Behrens, 1972) and a follow-
up, Beyond the Limits (1992) (Radzicki, 1997). The Malthusian projections
that came from these early models not only alienated the growth-oriented
policy makers of the West, but also brought severe criticism from many
of the academics in the field (e.g., economists), because of the glaring mis-
match between model “predictions” and what was actually occurring on
the world stage. This became more apparent as time went on, and it is fair
to say that this failure to meet empirical validation standards considerably
dampened the initial enthusiasm that met the system dynamics viewpoint
toward understanding the complex interrelations of complex systems.12
12 However, system dynamics modeling has been applied to several other areas, including
software project dynamics (Abdel-Hemid and Madnick, 1991), organizational learning (Senge,
Kleiner, Roberts, Ross, and Smith, 1994; Morecroft and Sterman, 1994), agriculture (Elmahdi,
Malano, and Khan, 2006), health care management (Rohleder, Bischak, and Baskin, 2007),
and transportation (Springael, Kunsch, and Brans, 2002).
OCR for page 130
0 BEHAVIORAL MODELING AND SIMULATION
More Recent Applications of System Dynamics Modeling
More recently, there has been a resurgence of interest in system dynamics
modeling, most particularly in public policy and business areas. Sterman’s
text on Business Dynamics (2000) presents a number of case studies that
demonstrate successful applications across a number of areas, including
global warming, the war on drugs, reengineering the supply chain of a
major computer firm, developing a marketing strategy in the automobile
industry, and planning process improvements in the petrochemicals indus-
try. The Department of Defense (DoD) has also taken a keen interest in this
approach, particularly for modeling diplomatic, information, military, and
economic (DIME) actions, and political, military, economic, social, infor-
mation, and infrastructure (PMESII) interactions. It is not our intent here to
survey all of these efforts, but merely to provide a few illustrative examples
to indicate the potential of system dynamics modeling in this area.
For example, Robbins’ Stabilization and Reconstruction Operations
Model (SROM) (Robbins, Deckro, and Wiley, 2005) analyzes the orga-
nizational hierarchy, dependencies, interdependencies, exogenous drivers,
strengths, and weaknesses of a country’s PMESII systems using a complex
set of interdependent system dynamics representations. SROM models a
country system in a holistic manner as a national model, which, as shown in
Figure 4-3, is then defined in terms of its n regional submodels that interact
with each other and the national model. Each regional submodule contains
six functional submodels: the demographics submodel, the insurgent and
coalition military submodel, critical infrastructure, law enforcement, indig-
enous security institutions, and public opinion. Each submodel is comprised
of approximately 600 model parameters, 90 random variables, 80 states
(stocks), and 190 rates of change (flows).
National
Sub-Model
Region 2
Region 1 Region N
Sub-Module
Sub-Module Sub-Module
FIguRE 4-3 Top-level nation SROM.
SOURCE: Robbins et al. (2005, p. 19).
4-3.eps
OCR for page 131
MACRO-LEVEL FORMAL MODELS
Figure 4-4 shows a portion of the critical infrastructure model of
SROM. The model captures a sequence of influences among variables, start-
ing from the power supply at an electrical substation. The generated power
is fed into an industrial water plant, which produces water consumed by
oil field work. An oil field produces crude oil to be refined by a refinery.
Finally, refined fuel is used to generate power, which in turn is supplied to
various power substations, thus forming a closed loop.
SROM has been demonstrated in modeling and analysis of Iraqi recon-
struction and recruiting efforts (Robbins et al., 2005). Parameters were set
to reflect prevailing conditions in Iraq on May 1, 2003, including
• Regional makeup (governorates)
• Regional population
• Population subgroup distribution
• Population support for coalition
• Oil and gas infrastructure
• Power infrastructure
• Transportation infrastructure
• Economic—regional gross domestic product
Robbins (2005) claims that the SROM allows analysts to more precisely
investigate the multifaceted process that is nation building: “[Because] the
complexities of nation-building involve many different but interrelated
systems and institutions, understanding the significance of the dynamic
relationships between these systems and institutions is paramount to
operational success. The system dynamics model proposed in this study
allows decision-makers and analysts to investigate different sets of decision
approaches at a sub-national, regional level” (p. 135).
The Pre-Conflict Anticipation and Shaping (PCAS) program (Popp et
al., 2006) was an attempt to evaluate alternative DIME/PMESII model-
ing efforts to predict nation-state collapse and to anticipate instabilities
that might lead to conditions necessitating military intervention. One of
the approaches, led by Nazli Choucri, developed a “state stability model”
using a system dynamics approach; a high-level view of the model is given
in Figure 4-5.
Power Industrial Oil Oil Power
Substation Water Plant Field Refinery Generators
Industrial Refined
Power Crude
Water Fuel
FIguRE 4-4 SROM infrastructure model.
SOURCE: Robbins, Deckro, and Wiley (2005).
4-4.eps
OCR for page 132
BEHAVIORAL MODELING AND SIMULATION
+
Military
Capability +
Dissident
Institutional –
+
+ + Social
Capacity
Cohesion
Anti-Regime
Activity – +
– –
–
+ Population
+
+ –
External –
Regime Force
Resources
+
and Violence –
State
Institutional
+ + GNP
Capacity +
– +
Civic Capacity
+
and Social
Liberties
+ +
Regime +
+ Legitimacy
4-5.eps
FIguRE 4-5 High-level view of system dynamics implementation of state stability
model. redrawn
SOURCE: Popp (2005).
According to Popp (2005, p. 18), it “shows loads, demands and
stresses on state and the causal dependencies; shows feedback loops, tipping
points and unintended consequences; [and] shows the internal and lateral
pressures that can lead to conflict.” By looking at the loads (demands)
placed on the system (nation-state) and evaluating those demands in terms
of the system’s capabilities, an assessment of stability can be made based
on how much demands exceed capacity.
Finally, O’Brien’s Integrated Crisis Early Warning System (ICEWS) is
a new program at DARPA/IPTO aimed at following on from the PCAS
exploration just described. According to the announcement of the research
program, its goal “is to develop a comprehensive, integrated, automated,
generalizable, and validated system to monitor, assess, and forecast national,
sub-national, and international crises in a way that supports decisions on
how to allocate resources to mitigate them. ICEWS will provide Combat-
ant Commanders (COCOMs) with a powerful, systematic capability to
anticipate and respond to stability challenges in the Area of Responsibility
(AOR); allocate resources efficiently in accordance to the risks they are
designed to mitigate; and track and measure the effectiveness of resource
allocations toward end-state stability objectives, in near-real time” (see
OCR for page 138
BEHAVIORAL MODELING AND SIMULATION
State of the Art in Organizational Modeling
Here, we focus on simulation or computational organizational models.
A number of books contain overviews and examples of many models in
this area (Carley and Prietula, 1994; Carley and Gasser, 1999; Lomi and
Larsen, 2001). Some models consider organization theory questions; others
are more oriented to organizational design questions; and some can be used
for both purposes. We begin with the theory models and then consider the
design models, with comments when the models can be used both ways.
Organization Theory Models
There are numerous organization simulations or computational orga-
nizational models; here we review a few of them. Most, but not all, are
agent-based models in which the organization is represented as agents that
are linked together by communication or authority structures or both.
The earliest computational organizational model was a behavioral
theory of the firm in which the organization was modeled in terms of goals,
expectations, and choice (Cyert and March, 1963). Simple systems were
used to demonstrate how nonrational behavior could generate behavior
similar to that observed in real organizations. This was then extended in
the now canonical model, the garbage can model of organizational choice
(Cohen, March, and Olsen, 1972). This was a simple Fortran program in
which basic matching and accumulation functions were combined to show
how variations in the problem access, salience of problems, and energy of
the participants altered the level of work and the quality of outcomes.
The Lin and Carley models look at organizations as networks of com-
munication linkages among agents, such that agents learn only from the
information that they get from the outside world or that is provided to them
by another agent in the organization (Lin and Carley, 2003; Lin, Zhao,
Ismail, and Carley, 2006). Using these models, they investigated questions
of crisis response. They conducted a “matched-set” validation experiment,
in which they compared the behavior of 69 real-world organizations faced
with industrial crises with the behavior of the simulated versions of those
same 69 companies. Using what-if analysis, they were then able to show
that the type of decision making employed by the organization—for exam-
ple, following standard operating procedures or following the dictates of
historically based experience—often led organizations to false conclusions
about their performance.
This work was generalized and extended to produce the OrgAhead
model. OrgAhead is a multiagent model of organizational design and the
examination of the impact of learning and strategic adaptation on that
design (Carley and Svoboda, 1996). In this model, learning occurs at the
OCR for page 139
MACRO-LEVEL FORMAL MODELS
operational and structural levels, using experiential and expectation-based
learning models. From a technical standpoint, the model uses simulated
annealing15 to alter the communication and authority lines and number of
agents. The agents are information-processing units with a simple learning
component. OrgAhead can be thought of as an operationalized grounded
theory. The basis for OrgAhead is the body of research, both empirical
and theoretical, on organizational learning and organizational design. The
model has built into it several theories of different aspects of organiza-
tional behavior. From the information-processing tradition comes a view of
organizations as information processors composed of collections of intel-
ligent individuals, each of whom is boundedly rational and constrained in
actions, access to information by the current organizational design (rules,
procedures, authority structure, communication infrastructure, etc.), and
his or her own cognitive capabilities. Organizations are seen as capable
of changing their design (DiMaggio and Powell, 1983; Romanelli, 1991;
Stinchcombe, 1965) and as needing to change if they are to adapt to
changes in the environment or the available technology (Finne, 1991). Dif-
ferent organizational designs are seen as better suited to some environments
or tasks than others (Hannan and Freeman, 1977; Lawrence and Lorsch,
1967). Aspects of the model have been tuned to reflect the findings of
various empirical studies related to these theories. The set of theories that
are unified into a single computational theory of organizational behavior
interact in complex fashions to determine the overall level of organizational
performance.
Harrison and Carroll (1991) investigated the effect of turnover on
organizational culture for different prototypical organizations and poli-
cies. Their model is stated as a set of mathematical functions, which are
then simulated and yield data that are analyzed as if they were field data.
The model is essentially a cultural diffusion model operating at the group
level. On the basis of “virtual experiments” conducted with the model and
a follow-on analysis of the resulting simulation-based data, they found
that some employee turnover can help stabilize the culture of the organiza-
tion, suggesting that some previously held truths about turnover are not
general.
An alternative information diffusion model is Construct, developed
by Carley to examine the coevolution of structure and culture that results
from individual information exchange and the formation and dissolution
15 Simulated annealing is a technique to find a good solution to an optimization problem by try-
ing random variations of the current solution. A worse variation is accepted as the new solution
with a probability that decreases as the computation proceeds. The slower the cooling schedule,
or rate of decrease, the more likely the algorithm is to find an optimal or near-optimal solution
(see http://www.nist.gov/dads/HTML/simulatedAnnealing.html [accessed August 2007]).
OCR for page 140
40 BEHAVIORAL MODELING AND SIMULATION
of social networks (Carley, 1991). Construct has been used to examine the
impact of new technologies on the workplace (Carley and Schreiber, 2002),
performance under diverse leadership styles (Schreiber and Carley, 2004),
and the emergence of organizational vulnerabilities (Carley, 2004).
NK models, originally suggested by Kauffman, are simple optimiza-
tion models, often operationalized using genetic algorithms, in which N
is the number of actors and K is degree of connectedness among the
actors (Kauffman and Weinberger, 1989). NK models have been applied to
organization theory questions of adaptation (Levinthal, 1997), search and
stability (Rivkin and Siggelkow, 2003), modularity and innovation (Ethiraj
and Levinthal, 2004), imitation and benchmarking (Rivkin, 2000), and
other basic questions about organizations. The explicit modeling of rugged
landscapes permits one to understand the limitations of organization expla-
nations that implicitly assume smooth performance surfaces. It also yields
greater insights into the persistence of variety among organizations.
The SimVision model (earlier called VDT) is a project organization
model (Levitt, Thomsen, Kunz, Jin, and Nass, 1999) which explicitly
models the project tasks (similar to a critical path method network) and the
hierarchical organization structure. In essence, this model is the merger of
Gantt chart technology with a limited information-processing model for the
agents. The project tasks are linked by the project network, and each task
is assigned directly to an agent in the hierarchy. SimVision has been used as
a laboratory for organization experiments.16 For example, Carroll, Burton,
Levitt, and Kiviniemi (2006) found that “fast tracking” or concurrent engi-
neering of projects quickly leads to increased coordination demands that do
not reduce total project time; additional personnel can also increase project
time as they require time to manage; and decentralization increases coordi-
nation demands. Earlier, Kim and Burton (2002) found that decentraliza-
tion reduces project time but may also decrease quality. Long, Burton, and
Cardinal (2002) demonstrated that three simultaneous control approaches
are better than any single control approach. These studies began with orga-
nizational questions and observations of real organizations as base models.
The simulation experimental manipulations (“virtual experiments”) went
beyond real-world observations to investigate plausible conditions of what
could happen for a better understanding of potential outcomes. Field obser-
16 In the studies cited here it must be remembered that the conclusions drawn from analysis
of the simulation-based data (in turn generated by virtual experiments in the simulation
domain) are not to be confounded with conclusions drawn from an analysis of homologous
real-world data. This is in keeping with our earlier footnote regarding how simulation-based
data can be analyzed as if it were real-world data. It often can, but the fundamental issue
still remains regarding the validity of applying the simulation-based conclusions to real-world
organizational behavior. Naturally, the more validated the model, the more likely one is to be
correct in cross-applying one’s conclusions.
OCR for page 141
4
MACRO-LEVEL FORMAL MODELS
vations and generalizations are limited in their applicability and should be
used with caution in the design of future organizations. Simulation studies
provide deeper insight into what is possible and what is desirable for
organizational redesign and change. SimVision can also be applied as an
organizational design model.
Organizational Design Models
The term “organizational design” is used both to mean the design of
the organization and the process of design. The two meanings are different
but closely related. In a special issue of Organization Science, Dunbar and
Starbuck (2006) focus on the process of organizational design in its many
facets. The articles give insight into how design can be accomplished and
the challenges encountered.
SimVision was applied to investigate organization theory questions.
But it was originally created as an organizational design tool to help
project managers optimize projects and project management implemen-
tation (Levitt, 2004) This included avoiding unforeseen bottlenecks and
finding options to compress project time. One of the insights is that project
managers adapted quite well to minor variations from the normal base case
but less well when there were large changes in requirements. The simula-
tions were extremely useful in helping project managers reframe the project
and redesign the project.
Pattipati and colleagues (Pattipati et al., 2002; Levchuk, Levchuk,
Luo, Pattipati, and Kleinman, 2002a, 2002b; Levchuk, Levchuk, Meirina,
Pattipati, and Kleinman, 2004) have used multiobjective optimization
algorithms to develop organizational designs optimized to meet mission
requirements for military command and control organizations, focusing
specifically on Joint Task Force command teams. These designs specify
both structure and process by specifying roles in the organization defined
in terms of control of resources, responsibility for tasks, and requirements
for coordination. Designs are then tested in simulations of organizational
performance and finally tested in field experiments in which military
officers play the roles that were designed using the model. Studies have
shown that optimized organizational designs based on the model result
in performance that exceeds that observed under more traditional designs
suggested by military subject matter experts (Entin, 1999). A key find-
ing of this work is that sufficient training is essential for the officers to
function effectively in the innovative organizational structures developed
using the model.
Carroll, Gormley, Bilardo, Burton, and Woodman (2006) describe
an organizational design process at the National Aeronautics and Space
Administration (NASA), where SimVision and other organizational design
OCR for page 142
4 BEHAVIORAL MODELING AND SIMULATION
tools were used as decision aids in creating a new organization. The chal-
lenge was to create an organization that had multiple functional experts,
was geographically disperse, and had severe resource constraints in which
project time and quality were paramount. The design team began with the
construction of the design structure matrix; it gave a good beginning but
generated questions as well as answers. Next, they used OrgCon—an expert
system organizational diagnosis and design tool—to model the proposed
organization at a high level in terms of structural properties, such as formal-
ization and decentralization. One purpose of this modeling was to identify
“misfits” (Burton and Obel, 2004) that suggested a need for change; they
found few of them. But many questions remained. Then they created a
SimVision of the proposed design to obtain greater detail and better under-
standing of how the organization would actually work. Using variations
in the design, they confirmed that the design developed with the aid of the
tools was reasonable. Perhaps most importantly, the usual organizational
design approach would have resulted in an organization that would have
failed to meet the goals and would have incurred delays and unanticipated
costs. The results indicate that the tools can make a difference and lead
to better designs; furthermore, the theory-based notion of organizational
misfits aids in the process. It can be a bridge between theory and design
and theory and practice, as managers find the identification of misfits and
their correction both intuitive and practical. NASA had been accustomed
to using simulations in engineering design but not in organizational design.
Nonetheless, the culture was amenable to the application of such tools for
organizational design.
Similarly, OrgAhead was built to explore the relative effectiveness of
different organizational designs. For example, it was used to determine
the adaptability and performance characteristics of different designs under
consideration by the Naval Strategic Studies Group. Construct, referred to
earlier, has also been used to evaluate various organizational designs under
different turnover regimes. Moreover, when data are collected on the who,
what, where, and how of organizations, such data can first be assessed for
points of vulnerability in ORA and then Construct can be applied to the
same empirical description of the real organization to forecast its behavior
in terms of information diffusion and performance with or without turn-
over (Carley, Diesner, Reminga, and Tsvetovat, 2005).
Levis and Wagenhals (2000) and the subsequent work with Shin, Kim,
Bienvenu, and Shin led to the development of a Petri net model for design-
ing and assessing organizational architectures (Bienvenu, Shin, and Levis,
2000; Wagenhals, Shin, Kim, and Levis, 2000). Modeling agents, their
resources, and the decision process, this overall approach makes possible
the fine tuning of detailed designs of core groups in organizations. This
approach has been used consistently to evaluate command and control
OCR for page 143
4
MACRO-LEVEL FORMAL MODELS
structures. The key advantage of this approach is that designs can be opti-
mized to the specific communication and timing requirements.
Relevance, Limitations, and Future Directions
The relevance of organizational models to the requirements outlined in
Chapter 2 is obvious. Representative tasks, such as designing effective orga-
nizations and disrupting adversary organizations, are clear candidates for
the use of such models. If it were possible to accurately assess the probable
effectiveness of various organizational options before implementing them,
much effort could be saved and many potentially catastrophic mistakes
avoided.
Limitations of such models as they now exist include requirements for
data that may be totally unavailable or unavailable in appropriate formats
and structures, the need for culturally appropriate information on which
to base assumptions and algorithms, especially for non-Western organiza-
tions, and technical issues requiring further development and refinement of
the models themselves.
R&D requirements include better methods for obtaining and using
organizational performance data to provide leaders and managers with
better tools for restructuring their organizations as necessary. The vast
majority of current model-based organizational design methods are static.
That is, they use prior performance data about the organization to develop
future designs, but they do not use “streaming” performance data as it
comes in to understand or modify the organization’s structure and processes
in real time. Organizational models that could accept and use real-time data
could provide a tool for making organizations more flexible and able to
adapt to changing conditions and missions more quickly.
An additional area in need of research is the ability to combine models
at different levels of granularity and detail to represent large organiza-
tions, as well as the advantages and drawbacks of including more or less
detail. Including detail for all of the individuals in a large organization can
quickly lead to intractable size and computational infeasibility, but system-
level models may not be able to represent the detail that leads to emergent
behavior. For example, system dynamics models could be developed at the
level of the entire organization, with individual agents developed to repre-
sent key individuals or groups in the organization. Data could flow in both
directions between the detailed agent-based models and the organization-
level system model. Challenges and existing approaches for developing such
integrated multilevel models are discussed in Chapter 8.
Finally, innovative experimentation approaches are needed to advance
the state of the art in organizational modeling. Systematic controlled experi-
ments are not feasible for organizations of any size—team experiments
OCR for page 144
44 BEHAVIORAL MODELING AND SIMULATION
rarely include more than six to eight team members. However, the devel-
opment of agents that can represent the behaviors of members of the
organization in a realistic way opens the door for “hybrid” experiments
in which most roles in the organizations are played by agents, with only a
few played by live subjects. Research is needed on the best ways to use this
hybrid experimentation capability to advance organizational science: the
types of questions that can best be addressed in such experiments, the best
ways to “control” such experiments in the classical sense of experimental
control, the level of fidelity needed in the agents, and the statistical tech-
niques needed for analysis of the results.
REFERENCES
Abdel-Hemid, T., and Madnick, S.E. (1991). Software project dynamics: An integrated
approach. Englewood Cliffs, NJ: Prentice-Hall.
Axelrod, R. (1997). Advancing the art of simulation in the social sciences. In R. Conte, R.
Hegselmann, and P. Terna (Eds.), Simulating social phenomena (pp. 21–40). Berlin,
Germany: Springer.
Bienvenu, M.P., Shin, I., and Levis, A.H. (2000). C4ISR architectures III: An object-oriented
approach for architecture design. Systems Engineering, (4), 288–312.
Black, H.S. (1977). Inventing the negative feedback amplifier. IEEE Spectrum, 4, 54–60.
Buck, J.T., Ha, S., Lee, E.A., and Messerschmitt, D.G. (1994). Ptolemy: A framework for
simulating and prototyping heterogeneous systems. International Journal of Computer
Simulation, Special Issue on Simulation Software Development Component Development
Strategies, 4.
Burton, R.M. (2003). Computational laboratories for organization science: Questions, validity
and docking. Computational and Mathematical Organization Theory, (2), 91–108.
Burton, R.M., and Obel, B. (2004). Strategic organizational diagnosis and design: The
dynamics of fit, third edition. Boston: Kluwer Academic.
Carley, K.M. (1991). Designing organizational structures to cope with communication break-
downs: A simulation model. Industrial Crisis Quarterly, 5, 19–57.
Carley, K.M. (2004). Estimating vulnerabilities in large covert networks using multi-level data.
In Proceedings of the North American Association for Computational Social and Organi-
zational Science (NAACSOS) 004 Conference, June 27–29, 2004, Pittsburgh, PA.
Carley, K.M., and Gasser, L. (1999). Computational organization theory. In G. Weiss (Ed.),
Multiagent systems: A modern approach to distributed artificial intelligence (pp. 299–
330). Cambridge, MA: MIT Press.
Carley, K.M., and Prietula, M.J. (1994). Computational organization theory. Hillsdale, NJ:
Lawrence Erlbaum Associates.
Carley, K.M., and Schreiber, C. (2002). Information technology and knowledge distribution in
C3I teams. In Proceedings of the 00 Command and Control Research and Technology
Symposium Conference, Naval Postgraduate School, Monterey, CA.
Carley, K.M., and Svoboda, D. (1996). Modeling organizational adaptation as a simulated
annealing process. Sociological Methods and Research, 5(1), 138–168.
OCR for page 145
45
MACRO-LEVEL FORMAL MODELS
Carley, K.M., Diesner, J., Reminga, J., and Tsvetovat, M. (2005). Toward an interoperable
dynamic network analysis toolkit. Decision Support Systems, Special Issue on Cyber-
infrastructure for Homeland Security: Advances in Information Sharing, Data Mining,
and Collaboration Systems, 4(4), 1321–1323.
Carroll, T.N., Burton, R.M., Levitt, R.E., and Kiviniemi, A. (2006). Fallacies of fast track
heuristics: Implications for organization theory and project management. Working paper,
Duke University Fuqua School of Business.
Carroll, T.N., Gormley, T.J., Bilardo, V.J., Burton, R.M., and Woodman, K.L. (2006). Design-
ing a new organization at NASA: An organization design process using simulation.
Organization Science, , 202–214.
Cohen, M.D., March, J.G., and Olsen, J.P. (1972). A garbage can model of organizational
choice. Administrative Sciences Quarterly, (1), 1–25.
Cyert, R.M., and March, J.G. (1963). A behavioral theory of the firm. Englewood Cliffs, NJ:
Prentice-Hall.
Davis, P.K., and Anderson, R.H. (2004). Improving the composability of Department of
Defense models and simulations. Santa Monica, CA: RAND.
DiMaggio, P.J., and Powell, W.W. (1983). The iron cage revisited: Institutional isomorphism
and collective rationality in organizational fields. American Sociological Review, 4(2),
147–160.
Dunbar, R.L., and Starbuck, W.H. (Eds.). (2006). Learning to design organizations and learn-
ing from designing them. Organization Science, (2), 171–178.
Elmahdi, A., Malano, H., and Khan, S. (2006). Using a system dynamics approach to model
sustainability indicators for irrigation systems in Australia. Natural Resource Modeling,
(4), 465–481.
Entin, E.E. (1999). Optimized command and control architectures for improved process and
performance. In Proceedings of the Command and Control Research and Tech-
nology Symposium, Newport, RI.
Finne, H. (1991). Organizational adaptation to changing contingencies. Futures, (10),
1061–1074.
Forrester, J.W. (1961). Industrial dynamics. Cambridge, MA: MIT Press.
Forrester, J.W. (1968). Principles of systems, second edition. Waltham, MA: Pegasus
Communications.
Forrester, J.W. (1969). Urban dynamics. Cambridge, MA: MIT Press.
Forrester, J.W. (1971). World dynamics. Cambridge, MA: Wright-Allen Press.
Forrester, J.W. (1989). The beginning of system dynamics. Banquet speech at the International
Meeting of the System Dynamics Society, July 13, Stuttgart, Germany. Available: http://
sysdyn.clexchange.org/sdep/papers/D-4165-1.pdf [accessed May 2007].
Forrester, J.W., and Senge, P. (1980). Tests for building confidence in system dynamics models.
In A.A. Legasto, Jr., J.W. Forrester, and J.M. Lyneis (Eds.), System dynamics (pp. 209–
228, Studies in the Management Sciences, vol. 14). Amsterdam: North-Holland.
Hannan, M.T., and Freeman, J. (1977). The population ecology of organizations. American
Journal of Sociology, (5), 929–964.
Harrison, J.R., and Carroll, G.R. (1991). Keeping the faith: A model of cultural transmission
in formal organizations. Admininistrative Science Quarterly, , 552–582.
Karnopp, D.C., Margolis, D.L., and Rosenberg, R.C. (2006). System dynamics: Modeling and
simulation of mechatronic systems, 4th edition. Hoboken, NJ: John Wiley & Sons.
Kauffman, S.A., and Weinberger, E.D. (1989). The N-K model of rugged fitness landscapes
and its application to maturation of the immune response. Journal of Theoretical Biology,
4, 211–245.
OCR for page 146
4 BEHAVIORAL MODELING AND SIMULATION
Kim, J., and Burton, R.M. (2002). The effect of task uncertainty and decentralization on
project team performance. Computational and Mathematical Organization Theory, (4),
365–384.
Lawrence, P.R., and Lorsch, J.W. (1967). Organization and environment: Managing differen-
tiation and integration. Boston: Graduate School of Business Administration, Harvard
University.
Levchuk, G.M., Levchuk, Y.N., Luo, J., Pattipati, K.R., and Kleinman, D.L. (2002a). Norma-
tive design of organizations—Part I: Mission planning. IEEE Transactions on Systems,
Man, and Cybernetics. Part A: Systems and Humans, (3), 346–359.
Levchuk, G.M., Levchuk, Y.N., Luo, J., Pattipati, K.R., and Kleinman, D.L. (2002b). Norma-
tive design of organizations—Part II: Organizational structure. IEEE Transactions on
Systems, Man, and Cybernetics. Part A: Systems and Humans, (3), 360–375.
Levchuk, G.M., Levchuk, Y.N., Meirina, C., Pattipati, K.R., and Kleinman, D.L. (2004).
Normative design of project-based organizations—Part III: Modeling congruent, robust,
and adaptive organizations. IEEE Transactions on Systems, Man, and Cybernetics. Part
A: Systems and Humans, 4(3), 337–350.
Levinthal, D.A. (1997). Adaptation on rugged landscapes. Management Science, 4, 934–950.
Levis, A.H., and Wagenhals, L.W. (2000). C4ISR Architectures I: Developing a process for
C4ISR architecture design. Systems Engineering, (4), 225–247.
Levitt, R.E. (2004). Computational modeling of organizations comes of age. Computational
and Mathematical Organization Theory, 0(2), 127–145.
Levitt, R.E., Thomsen, J.C., Kunz, T.R., Jin, Y., and Nass, C. (1999). Simulating project
work processes and organizations: Toward a micro-contingency theory of organizational
design. Management Science, 45(11), 1479–1495.
Lin, Z., and Carley, K.M. (2003). Designing stress resistant organizations: Computational
theorizing and crisis applications. Boston: Kluwer.
Lin, Z., Zhao, X., Ismail, K., and Carley, K.M. (2006). Organizational design and restructur-
ing in response to crises: Lessons from computational modeling and real world cases.
Organizational Science, (5), 598–618.
Lomi, A., and Larsen, E.R. (2001). Dynamics of organizations: Computational modeling and
organization theories. Menlo Park, CA: AAAI Press/MIT Press.
Long, C.P., Burton, R.M., and Cardinal, L.B. (2002). Three controls are better than one: A
computational model of complex control systems. Computational and Mathematical
Organization Theory, (3), 197–220.
McRuer, D.T., and Krendel, E.S. (1974). Mathematical models of human pilot behavior.
Hawthorne, CA: Systems Technology.
Meadows, D.H., Meadows, D.I., Randers, J., and Behrens, W.W.I. (1972). The limits to
growth. New York: Universe Books.
Morecroft, J.D.W., and Sterman, J. (1994). Modeling for learning organizations. Portland,
OR: Productivity Press.
Ogata, K. (2003). System dynamics, fourth edition. Englewood Cliffs, NJ: Prentice-Hall.
Pattipati, K.R., Meirina, C., Pete, A., Levchuk, G., and Kleinman, D.L. (2002). Decision
networks and command organizations. In A.P. Sage (Ed.), Systems engineering and
management for sustainable development, Encyclopedia of life support systems. Oxford,
England: UNESCO–EOLSS.
Popp, R. (2005). Briefing to Committee on Organizational Modeling from Individuals to
Societies, National Research Council, Keck Center, Washington, DC.
OCR for page 147
4
MACRO-LEVEL FORMAL MODELS
Popp, R., Kaisler, S.H., Allen, D., Cioffi-Revilla, C., Carley, K.M., Azam, M., Russell, A.,
Choucri, N., and Kugler, J. (2006). Assessing nation-state instability and failure. Aerospace
Conference, 00 IEEE, , 4–11. Available: http://ieeexplore.ieee.org/iel5/11012/34697/
01656054.pdf?isNumber= [accessed Feb. 2008].
Quadrat-Ullah, H. (2005). Structural validation of system dynamics and agent-based sim-
ulation models. In Proceedings of the th European Conference on Modeling and
Simulation. Available: http://www.econ.iastate.edu/tesfatsi/EmpValidSDACE.Hassan.pdf
[accessed Feb. 2008].
Radzicki, M.J. (1997) Introduction to system dynamics: Origin of system dynamics. Available:
http://www.systemdynamics.org/DL-IntroSysDyn/start.htm [accessed Feb. 2008].
Richardson, G.P. (1991). Feedback thought in social science and systems theory. Philadelphia:
University of Pennsylvania Press.
Richardson, G.P., and Pugh, A.L. III. (1981). Introduction to system dynamics modeling with
DYNAMO. Cambridge, MA: MIT Press.
Richmond, B., and Peterson, S. (1992). STELLA II. An introduction to systems thinking.
Hanover, NH: High Performance Systems Inc.
Rivkin, J.W. (2000). Imitation of complex strategies. Management Science, 4(6), 824–844.
Rivkin, J.W., and Siggelkow, N. (2003). Balancing search and stability: Interdependencies
among elements of organizational design. Management Science, 4(3), 290–311.
Robbins, M. (2005). Investigating the complexities of nationbuilding: A sub-national regional
perspective. Master’s thesis, Department of the Air Force Air University. Available: http://
stinet.dtic.mil/cgi-bin/GetTRDoc?AD=ADA435214&Location=U2&doc=GetTRDoc.pdf
[accessed Feb. 2008].
Robbins, M., Deckro, R.F., and Wiley, V.D. (2005). Stabilization and reconstruction opera-
tions model (SROM). Presented at the Center for Multisource Information Fusion Fourth
Workshop on Critical Issues in Information Fusion: The Role of Higher Level Informa-
tion Fusion Systems Across the Services, University of Buffalo. Available: http://www.
infofusion.buffalo.edu/ [accessed Feb. 2008].
Roberts, N.D.F., Anderson, R.M., Deal, R.M., Garet, M.S., and Shaffer, W.A. (1983). Intro-
duction to computer simulation: A systems dynamics modeling approach. Reading, MA:
Addison-Wesley.
Rohleder, T.R., Bischak, D.P., and Baskin, L.B. (2007). Modeling patient service centers with
simulation and system dynamics. Health Care Management Science, 0(1), 1–12.
Romanelli, E. (1991). The evolution of new organizational forms. Annual Review of Sociology,
, 79–103.
Sage, A.P. (1977). Methodology for large-scale systems. New York: McGraw-Hill.
Sage, A.P., and Armstrong, J.E. (2000). Introduction to systems engineering. (Wiley Series in
Systems Engineering and Management). Hoboken, NJ: John Wiley & Sons.
Schreiber, C., and Carley, K.M. (2004). Going beyond the data: Empirical validation lead-
ing to grounded theory. Computational and Mathematical Organization Theory, 0(2),
155–164.
Scott, W.R. (1998). Organizations: Rational, natural, and open systems, fourth edition. Upper
Saddle River, NJ: Prentice-Hall.
Senge, P.M., Kleiner, A., Roberts, C., Ross, R., and Smith, B. (1994). The fifth disipline field-
book. New York: Doubleday.
Springael, J., Kunsch, P.L., and Brans, J.-P. (2002). A multicriteria-based system dynamics
modeling of traffic congestion caused by urban commuters. Central European Journal
of Operations Research, 0(1), 81–97.
OCR for page 148
4 BEHAVIORAL MODELING AND SIMULATION
Sterman, J.D. (2000). Business dynamics: Systems thinking and modeling for a complex world.
New York: McGraw-Hill.
Stinchcombe, A. (1965). Organization-creating organizations. Trans-Actions, , 34–35.
Wagenhals, L.W., Shin, I., Kim, D., and Levis, A.H. (2000). C4ISR Architectures II: Structured
analysis approach for architecture design. Systems Engineering, (4), 248–287.
Weiner, N. (1948). Cybernetics: Or the control and communication in the animal and the
machine. Cambridge, MA: MIT Press.
Williams, T. (2002). Modeling complex projects. Hoboken, NJ: John Wiley & Sons.
