Part II
Analytic Methods
In Part II, four papers present the contributions of four social science approaches to intelligence analysis: operations research, game theory, signal detection theory, and qualitative analysis. These four approaches were selected for their immediate applicability to the needs of intelligence analysts and because their benefits and limitations are well understood as the result of extensive scientific research and testing.
In Chapter 2, Edward H. Kaplan introduces readers to the field of operations research (OR). After reviewing the field’s origins in the applications of applied mathematics to military decision-making problems, Kaplan describes current methods in OR, showing how they could be adopted for intelligence analysis. He stresses OR’s value in organizing diverse pieces of information for understanding the operational capabilities and challenges of all actors in a situation. He focuses on optimization, probability modeling, and decision analysis as OR tools that are particularly well suited for intelligence analysis. He shows how OR from its inception has recognized the value of timely but imperfect analysis—as is often needed with intelligence analysis.
In Chapter 3, Bruce Bueno de Mesquita describes the fundamental assumptions of game theory and its applications to intelligence analysis. He notes its ability to help analysts pay attention to events that do not happen; explain discontinuities; understand the constraints of uncertainty, risks, costs and benefits; and coordinate multiple actors. As an example, he shows how game theory clarifies the potentially misleading role of selection effects in interpreting historical patterns. Bueno de Mesquita also shows the
value of game theory reasoning itself, which is enhanced when it can be combined with empirical and quantitative analysis.
In Chapter 4, Gary H. McClelland presents the contributions of signal detection theory to improving the performance and evaluation of analytic judgments and tradecraft. Signal detection theory provides an orderly way of treating how well analysts understand uncertain situations and what decision rules guide their judgments about them. McClelland illustrates the approach with applications to technology, medicine, and science policy, which parallel the challenges faced by intelligence analysts who must understand uncertain situations and convey their conclusions to policy makers. In particular, the paper shows how signal detection theory can be used to clarify the lessons of 9/11 and the “failures” of intelligence about Iraq weapons of mass destruction—distinguishing decision rules (e.g., systematic bias toward false alarms) from failures to understand distortion.
In Chapter 5, Kiron K. Skinner describes the essential roles of formal qualitative analysis in intelligence analysis. She shows how political science provides disciplined methods for increasing the usefulness and accuracy of qualitative analysis. Skinner’s paper illustrates these methods with lessons from two historical intelligence failures, drawing on the “strategic perspective,” a theory of decision making that integrates observations of state behavior, political leadership, and the connections between domestic politics and international relations.