Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Summary F resh from the contributions made by science to the World War II suc- cess, at mid-century the nation adopted a broad policy to invest heav- ily in science and technology as a foundation for economic growth, social welfare, and national security. The emphasis was on the physical and biological sciences, but the social sciences were mobilized with respect to selected foreign and domestic challenges--area studies for the former and large-scale empirical projects on social welfare for the latter. The 1966 study Equality of Educational Opportunity (known as the Coleman report) is a convenient marker for the arrival of "big" social science. It was designed to inform national and state policy relevant to reducing racial disparities in public education. Other large-scale research projects followed: on a negative income tax, housing allowances, and health insurance, among others. Evaluation research was announced as a new research specialty. Later in the century emphasis was placed on performance metrics, social indicators, ranking schemes, comparative assessment, and related tools and concepts based in social science. Private- sector organizations--university centers and institutes, think tanks, survey houses, and for-profit consulting firms--rapidly expanded in numbers and scope, as did graduate-level schools to prepare professionals for careers in public policy. The federal statistical system made available its significant information base for policy analysis in these nongovernmental settings. The federal government recruited social scientists in executive agencies and on congressional staffs. 1

OCR for page 1
2 USING SCIENCE AS EVIDENCE IN PUBLIC POLICY By the end of the 20th century, a multibillion dollar policy enterprise was in place, drawing on private philanthropic support as well as federal and state funding. The task of this loosely interconnected policy enterprise is to describe social conditions, advise on policy interventions, test alterna- tive program designs, and evaluate outcomes. This work is funded on the promise that good science will be used to decide what social conditions need attention, what should be a public responsibility or better left to the market or not-for-profit actors, and what interventions--to grow the economy, improve welfare, protect security--are efficient and effective. As the policy enterprise expanded and extended its reach, interest mounted in whether its knowledge products were being adequately used. Continued investment in producing the knowledge suggested it was used and valued, but how valuable and for whom was uncertain. This uncertainty was addressed in a 1978 National Research Council report, Knowledge and Policy: The Uncertain Connection. The report found that, despite numer- ous social science studies of policy interventions and steps to increase their relevance to and use for policy making, "we lack systematic evidence as to whether these steps are having the results their sponsors hope for. . . ." More than three decades later our report returns to the "uncertain con- nection," to again ask what is known about how scientific knowledge is used in public policy and how it can be more effectively used. The Committee on the Use of Social Science Knowledge in Public Policy was charged by the National Research Council "to review the knowledge utilization and other relevant literature to assess what is known about how social science knowledge is used in policy making . . . [and] to develop a framework for further research that can improve the use of social science knowledge in policy making." The first charge, to assess what is known, led us to an early and obvi- ous point. Knowledge from all the sciences is relevant to policy choices: the physical sciences inform energy policy on renewable efficiencies; the biolog- ical sciences inform public health policy on infectious diseases; the engineer- ing sciences inform national defense policy on weapon design; the social sciences inform economic policy on international trade trends. Under standing whether, why, and how this scientific knowledge is used, however, is uniquely suited to the methods and theories of the social sciences. Making "use" of scientific knowledge is what people and organizations do. And what people and organizations do is the focus of social science. To date, there has not been much success in explaining the use of science in public policy. We base this statement on three findings. First,

OCR for page 1
SUMMARY 3 although there are heuristically valuable typologies of ways science is used in policy, the typologies have not (and perhaps cannot) guide empirical research programs. Second, the research specialty labeled "knowledge utilization" has focused on challenges highlighted by the "two communi- ties" metaphor (researchers and policy makers, each with their distinctive cultures) and proposed various innovations to improve communication and interaction between science and policy--brokering, translation, interaction models. There is little systematic research on whether these innovations are improving the use of science in policy, although there are clear indications that they are being usefully applied in practice settings. In fact, it is not even clear that the two communities metaphor is the most fruitful way to frame a study of knowledge use in policy. Third, although the relatively recent ap- proach known as evidence-based policy and practice, focused on improving understanding of "what works," has influenced the production of scientific knowledge, it has made little contribution to understanding the use of that knowledge. In some of its more prominent formulations the issue of "use," because it involves political and value considerations, is said to be outside the scope of evidence-based policy. If more than three decades of worrying about science use in public policy has not produced satisfactory explanations, it may be that we have been looking in the wrong place--for a coherent typology of use or ways to bridge the gap between two communities. The committee turned its attention to a research framework that draws on recent developments in social science perhaps better suited to explaining the use of science in public policy. The first step in constructing the framework reprised a familiar point. Science, when it has something to offer, should be at the policy table. But it shares that table with an array of nonscientific reasons for making a policy choice: personal and political beliefs and values are present, as are lessons from experience, trial and error learning, and reasoning by analogy. Obvi- ously, political matters and pressures weigh heavily when policy choices are made. Nevertheless science is a unique voice. What science has to say about policy choices results from investigations governed by systematic and rule-governed efforts that guard against self-deception--against believing something is true because one wants it to be. Because science is designed to be disinterested, if a policy question involves what are the "real" condi- tions or what will "probably" happen if one policy is implemented instead of another, science is generally a more dependable and defensible guide than informed hunches, analogies, or personal experience. Also, at least in

OCR for page 1
4 USING SCIENCE AS EVIDENCE IN PUBLIC POLICY a democracy, political leaders are obliged to give reasons for their policy choices--the theory of democratic accountability underpins this obligation. These reasons often require science-based description of conditions need- ing attention and explanations of what is likely to happen (or did happen) because of a policy intervention. Science has five tasks related to policy: (1) identify problems, such as endangered species, obesity, unemployment, and vulnerability to natural disasters or terrorist acts; (2) measure their magnitude and seriousness; (3) review alternative policy interventions; (4) systematically assess the likely consequences of particular policy actions--intended and unintended, desired and unwanted; and (5) evaluate what, in fact, results from policy. Across all of these tasks, there are political and value considerations that are outside the scope of science. We acknowledge that and build it into the recommended research framework. A FRAMEWORK FOR RESEARCH ON USE Policy is made in many settings. It evolves from a many faceted social process involving multiple actors engaged in assembling, interpreting, and debating what evidence is relevant to the policy choice at hand, and then, perhaps, using that evidence to claim that a particular policy choice is bet- ter than its alternatives. This process is best understood as a form of policy argument or practical reasoning that is persuasive with respect to the benefit or harm of policy actions. Policy argument includes generalizations, extrap- olations, assumptions, analogies, metaphors, anecdotes, and other elements of reasoning that differ from and can contradict scientific reasons. From this perspective, scientific knowledge is "evidence" when that knowledge is used in support of statements relevant to policy claims. "Evidence" does not re- side only in the world where science is produced; it emerges in the political world of policy making, where it is interpreted, made sense of and is used, perhaps persuasively, in policy arguments. Evidence-influenced politics is suggested as a more informative metaphor, descriptively and prescriptively, than evidence-based policy. Our research framework argues for more careful study of policy ar- gumentation, as well as for increased roles for the psychology of decision making and for systems perspectives. The social sciences offer important knowledge about how mental models, belief systems, organizational rules, societal norms, and other factors influence the behavior of decision mak- ers. They also offer important knowledge about how people learn, when

OCR for page 1
SUMMARY 5 they optimize and when they satisfice; why they organize themselves, form institutions, communicate, establish norms, and develop routines; how they assess risks; and how they make decisions, individually and collectively. This array of scientific specialties has never fully addressed a key issue: when, why, how, even whether science is used in public policy making. Research can explain the cognitive operations and biases that policy makers and scientists bring to their work and the context-specific situations, practices, logics (ways of reasoning and understanding), and cultural assumptions of the settings in which they operate. Relevant research fields include social psychology, behavioral economics, decision theory, and organizational so- ciology. We urge scholars in these and related specialties to investigate the use of scientific knowledge in policy making. Policy interventions unfold in large, complex, dynamic social systems. A systems perspective helps decision makers and researchers think broadly about the many effects a policy may produce and the ways in which a planned social intervention interacts with other existing interventions and institutional practices. Rarely can the study of the individual components of a system lead to a full understanding of the system. There are systems effects on individual actors and the system as a whole, including emergent, indirect, and delayed effects, as well as unintended and unpredictable con- sequences from the interactivity of a system's elements. The social sciences bring a variety of approaches and methodologies to the study of complex systems. Examples of the use of systems thinking in the study of national security, obesity prevention, and the evaluation of complex social interven- tions illustrate its potential utility in policy making more broadly. THE NEXT GENERATION OF RESEARCHERS AND PRACTITIONERS The three actors central to advancing and applying the research frame- work are established scholars in the fields and specialties identified above, Ph.D. candidates in those fields and specialties, and administrators and faculty responsible for curricula in schools and programs characterized by the term "policy education." Established scholars have long-range research agendas and are not easily persuaded to drop them to pursue new questions. New research fields nevertheless emerge when even a few established scholars focus their theories and methods on a major question getting little attention. Among decision-making theorists, cognitive psychologists, and scholars of system

OCR for page 1
6 USING SCIENCE AS EVIDENCE IN PUBLIC POLICY properties are some, we expect, who will find that posing the issue of sci- ence use as needing their attention will be attractive. It is exciting to be in on the ground floor of a new field of research, especially when there is a large and influential audience waiting for guidance on how to strengthen science use in policy. A companion effort focuses on students at the Ph.D. stage. There are numerous examples of philanthropic and federal funding that shaped the choice of dissertation topics and the early research trajectory of young scholars--resulting in new scholarly fields of inquiry. With heightened political attention to the "broader impacts" of science, answers are being sought, for example, for better ways to link natural and social sciences in addressing policy challenges, to better understand how variability in the quality of scientific evidence affects its use, and to the value of investing in intermediaries promising to promote the use of science as evidence. The third audience is those responsible for the curriculum in public policy schools and programs in U.S. universities, which annually graduate thousands of students, many of whom find positions in the policy enter- prise. It would be useful to know in some detail, first, what these students are and are not being taught as it bears on the use of science in public policy. That would require an investigation far more extensive than the committee could undertake. We did conduct a limited review sufficient to reach con- clusions relevant to what our findings imply for policy education. Our point is simple: policy education should equip its graduates to promote the use of science in policy-making settings. Graduates need, obviously, a working familiarity with the substance of policy issues and competency to locate, assess, and introduce validated research on those is- sues. But more is needed. Success at promoting science depends on grasping the complexity of the policy world, and on understanding the assumptions underlying divergent policy framings, expert judgments, and consensus- building techniques, as well as standard analytic methods and approaches. Policy students can be taught to appreciate policy making through policy argument or practical reasoning and to understand that the relevance of and weight given to science depends on the policy context. They can recognize the limits of the persuasive power of scientific reasoning, the substantial institutional barriers and cultural resistance to new scientific knowledge, and the role of moral and ethical beliefs. For a century or more the social sciences have contributed to policy making in many ways, especially in informing policy design and evaluation. We see a fresh way they can further contribute: by specifically focusing on whether, why, and how science is, or is not, used as evidence in public policy.