Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 1
1
INTRODUCTION AND OVERVIEW
The enactment of the America COMPETES Act in 2006 (and its
reauthorization in 2010), the increase in research expenditures under the
2009 American Recovery and Reinvestment Act (ARRA), and President
Obama’s general emphasis on the contribution of science and technology
to economic growth have all heightened interest in the role of scientific
and engineering research in creating jobs, generating innovative
technologies, spawning new industries, improving health, and producing
other economic and societal benefits. Along with this interest has come a
renewed emphasis on a question that has been asked for decades: Can the
impacts and practical benefits of research to society be measured either
quantitatively or qualitatively?
On April 18-19, 2011, the Board on Science, Technology, and
Economic Policy (STEP) of the National Research Council and the
Committee on Science, Engineering and Public Policy (COSEPUP), a
joint unit of the National Academy of Sciences, the National Academy of
Engineering, and the Institute of Medicine, held a workshop to examine
this question The workshop brought together academic researchers,
research and development (R and D) managers from private industry,
representatives from government agencies, leaders of philanthropic
organizations, and others to look at the very broad range of issues
associated with evaluating the returns on federal investments (Appendix
A). Speakers included researchers who have worked on the topic for
decades and early-career researchers who are pioneering non-traditional
approaches to the topic. In recent years, new research has appeared and
new data sets have been created or are in development. Moreover,
international interest in the topic has broadened substantially— in Latin
America and Asia as well as in Europe. The workshop included
presentations by speakers from abroad to gain their perspectives on
methods of analysis. The workshop sought to assemble the range of work
1
OCR for page 2
2 MEASURING THE IMPACTS OF FEDERAL INVESTMENTS IN RESEARCH
that has been done in measuring research outcomes and to provide a
forum to discuss its methods. The workshop’s goal was not to identify a
single best method or few best methods of measuring research impacts.
The workshop considered methodological differences across fields of
research to identify which can be applied to the broad range of federal
research funding. It did not address the role of federal funding in the
development of technology.
The workshop was motivated by a 2009 letter from Congressman
Rush Holt (D-New Jersey). He asked the National Academies to look
into a variety of complex and interconnected issues, such as the short-
term and long-term economic and non-economic impact of federal
research funding, factors that determine whether federally funded
research discoveries result in economic benefits, and quantification of the
impacts of research on national security, the environment, health,
education, public welfare, and decision making. “Discussing the
economic benefits of research is critical when discussing research
programs during the annual federal appropriations process,” he wrote.
Obviously, no single workshop could examine all of those questions, but
it laid the groundwork for such an inquiry.
The workshop was sponsored by seven federal agencies: the
National Science Foundation (NSF), the National Institutes of Health
(NIH), the U.S. Department of Agriculture (USDA), the Environmental
Protection Agency (EPA), the U.S. Geological Survey (USGS), the
National Aeronautics and Space Administration (NASA), and the
Department of Energy (DOE). It was organized by a planning committee
co-chaired by Neal Lane, Malcolm Gillis University Professor at Rice
University and former director of NSF and the Office of Science and
Technology Policy (OSTP), and Bronwyn Hall, Professor at the
University of California, Berkeley, and the University of Maastricht.
Consistent with Congressman Holt’s concerns, the planning
committee focused the workshop on broad social effects of public
research investments – economic growth, productivity, and employment,
social values such as environmental protection and food security, public
goods such as national security, and the behavior of decision-makers and
the public. The near-term outputs of research— scientific publications
and other communications, citations to previous work, research
collaborations and networks, and even patents resulting from R and D—
were a not a principal focus of the meeting. Arguably, scientific and
technical training is a near-term output of research but was featured in
the workshop discussion because of its relationship to job creation and
OCR for page 3
3
INTRODUCTION AND OVERVIEW
wage growth. Moreover, a large proportion of the technical professionals
trained in research is subsequently employed in other than research
occupations. The planning committee did not stipulate a timeline for the
research impacts of interest, although policymakers’ interest is
concentrated on the short-to medium-term and the measurement
challenge becomes greater the longer the time horizon.
This summary of the workshop provides the key observations and
suggestions made by the speakers at the workshop and during the
discussions that followed the formal presentations. The views contained
in this summary are those of individual workshop participants and do not
represent the views of workshop participants as a whole, the organizing
committee, STEP, COSEPUP, or the National Academies. The
summaries of the workshop discussions have been divided into eight
chapters. After this introductory chapter, chapter 2 looks at several broad
issues involved in the use of performance measures for research. Chapter
3 examines the direct impacts of research on the economy and the quality
of life. Chapter 4 considers a closely related topic: the effects of
biomedical research on health. Chapter 5 reviews other impacts of
research that are not necessarily reflected in economic markets, including
international development, agricultural advances, and national security.
Chapter 6 moves on to what many speakers cited as one of the most
important benefits of research: the training of early career scientific
investigators who go on to apply their expertise and knowledge in
industry, government, and academia. Chapter 7 summarizes the views of
analysts from the United Kingdom, the European Union, and Brazil,
highlighting the somewhat different approaches to similar problems
being taken in other countries. Chapter 8 examines the emergence of new
metrics that may be more powerful in assessing the effects of research on
a wide variety of economic and societal indicators. And chapter 9
presents observations made during a final panel presentation on the
pitfalls, progress, and opportunities offered by continuing work on
measuring the impacts of federal investments in research.
OCR for page 4
4 MEASURING THE IMPACTS OF FEDERAL INVESTMENTS IN RESEARCH
Remarks of Congressman Rush Holt (D-NJ)
At the beginning of the workshop, Congressman Rush Holt,
whose 2009 letter initiated the process leading to the workshop,
addressed the group by video. His remarks have been slightly
shortened.
I can’t emphasize strongly enough the importance of your
gathering. Measuring the impact of federal investments in research
is a critical need for both government and society. We are living in
what may become a pivotal time in our history. For well over half a
century we have mined the investments that we made in the
immediate aftermath of the Second World War and the fear that
gripped us after the launch of Sputnik, from the airplane to the
aerospace industry, and from the semiconductor to the Internet.
American scientists have built the foundation of the strongest
economy in the world.
But the Sputnik era is over. American leadership and our
shared prosperity are in peril. As President Obama has said, we’re
in need of another Sputnik moment. According to the World
Economic Forum’s latest Global Competitiveness Report, the
United States ranks fourth in global competitiveness behind
Switzerland, Sweden, and Singapore. Further, the World Economic
Forum ranks the United States forty-eighth in the quality of math
and science education in our schools. Of course, any such rankings
of competitiveness or economic or educational achievement are
subject to challenge under methodology and, further, those
rankings may not be measuring what really can make or keep the
United States great or prosperous. However, today 77 percent of
global firms planning to build new R and D facilities say they will
build them in China or India, not in the United States. In 2009, 51
percent of U.S. patents were awarded to non-U.S. companies.
China has gone from fifteenth place to fifth in international patents.
Other countries are investing and implementing many of the
changes suggested five years ago here in the United States while
we continue to hedge and debate. We’re losing our leadership
position and our edge in the global economy.
History suggests that our long-term economic prosperity
depends on maintaining a robust, modern innovation infrastructure
and educational system. That’s why some of us worked hard to
OCR for page 5
5
INTRODUCTION AND OVERVIEW
include $22 billion in new R and D funding in the American
Recovery and Reinvestment Act. Those funds were an important
short—and long-term boost for our economy— short-term in hiring
lab technicians and electricians to wire the labs and administrators
and clerks to handle the programs, long-term in bringing
innovations yet to be determined. Sustainable economic growth
will require a sustained investment.
Although our economy has made progress, it continues to
struggle. We’re facing a time of serious budget pressure and,
perhaps more serious, political pressure that could imperil the
support and funding for federal research and development. Some
people are suggesting significant cuts for agencies like NSF, NIST,
DOE, NIH, NASA, and EPA.
We must be careful stewards of public funds. We need to
ensure that our money is being used wisely and efficiently on
programs that meet our objectives: creating jobs, building the
economy, and creating a sustainable energy future, for example.
Yet it is clear to me that cutting federal research funds is not a wise
way to balance our budget.
Decision making, whether individual or Congressional, often
happens through anecdotes. Nevertheless, we have to be
intellectually honest. We have to make sure that the anecdotes are
based on something substantial. We need data that will show us
what is working and who is being put to work. Evidence can
triumph over ideology—sometimes.
You are taking seriously the responsibility to provide hard
facts and evidence about our investments. Together, you are
building the infrastructure that we need to answer these important
questions. I believe that our technological leadership and the
foundation of our whole economy depend on it.
OCR for page 6
