National Academies Press: OpenBook
« Previous: Front Matter
Suggested Citation:"1 INTRODUCTION AND OVERVIEW." National Research Council. 2011. Measuring the Impacts of Federal Investments in Research: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13208.
×

1

INTRODUCTION AND OVERVIEW

The enactment of the America COMPETES Act in 2006 (and its reauthorization in 2010), the increase in research expenditures under the 2009 American Recovery and Reinvestment Act (ARRA), and President Obama’s general emphasis on the contribution of science and technology to economic growth have all heightened interest in the role of scientific and engineering research in creating jobs, generating innovative technologies, spawning new industries, improving health, and producing other economic and societal benefits. Along with this interest has come a renewed emphasis on a question that has been asked for decades: Can the impacts and practical benefits of research to society be measured either quantitatively or qualitatively?

On April 18-19, 2011, the Board on Science, Technology, and Economic Policy (STEP) of the National Research Council and the Committee on Science, Engineering and Public Policy (COSEPUP), a joint unit of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine, held a workshop to examine this question The workshop brought together academic researchers, research and development (R and D) managers from private industry, representatives from government agencies, leaders of philanthropic organizations, and others to look at the very broad range of issues associated with evaluating the returns on federal investments (Appendix A). Speakers included researchers who have worked on the topic for decades and early-career researchers who are pioneering non-traditional approaches to the topic. In recent years, new research has appeared and new data sets have been created or are in development. Moreover, international interest in the topic has broadened substantially—in Latin America and Asia as well as in Europe. The workshop included presentations by speakers from abroad to gain their perspectives on methods of analysis. The workshop sought to assemble the range of work

Suggested Citation:"1 INTRODUCTION AND OVERVIEW." National Research Council. 2011. Measuring the Impacts of Federal Investments in Research: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13208.
×

that has been done in measuring research outcomes and to provide a forum to discuss its methods. The workshop’s goal was not to identify a single best method or few best methods of measuring research impacts. The workshop considered methodological differences across fields of research to identify which can be applied to the broad range of federal research funding. It did not address the role of federal funding in the development of technology.

The workshop was motivated by a 2009 letter from Congressman Rush Holt (D-New Jersey). He asked the National Academies to look into a variety of complex and interconnected issues, such as the short-term and long-term economic and non-economic impact of federal research funding, factors that determine whether federally funded research discoveries result in economic benefits, and quantification of the impacts of research on national security, the environment, health, education, public welfare, and decision making. “Discussing the economic benefits of research is critical when discussing research programs during the annual federal appropriations process,” he wrote. Obviously, no single workshop could examine all of those questions, but it laid the groundwork for such an inquiry.

The workshop was sponsored by seven federal agencies: the National Science Foundation (NSF), the National Institutes of Health (NIH), the U.S. Department of Agriculture (USDA), the Environmental Protection Agency (EPA), the U.S. Geological Survey (USGS), the National Aeronautics and Space Administration (NASA), and the Department of Energy (DOE). It was organized by a planning committee co-chaired by Neal Lane, Malcolm Gillis University Professor at Rice University and former director of NSF and the Office of Science and Technology Policy (OSTP), and Bronwyn Hall, Professor at the University of California, Berkeley, and the University of Maastricht.

Consistent with Congressman Holt’s concerns, the planning committee focused the workshop on broad social effects of public research investments – economic growth, productivity, and employment, social values such as environmental protection and food security, public goods such as national security, and the behavior of decision-makers and the public. The near-term outputs of research—scientific publications and other communications, citations to previous work, research collaborations and networks, and even patents resulting from R and D—were a not a principal focus of the meeting. Arguably, scientific and technical training is a near-term output of research but was featured in the workshop discussion because of its relationship to job creation and

Suggested Citation:"1 INTRODUCTION AND OVERVIEW." National Research Council. 2011. Measuring the Impacts of Federal Investments in Research: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13208.
×

wage growth. Moreover, a large proportion of the technical professionals trained in research is subsequently employed in other than research occupations. The planning committee did not stipulate a timeline for the research impacts of interest, although policymakers’ interest is concentrated on the short-to medium-term and the measurement challenge becomes greater the longer the time horizon.

This summary of the workshop provides the key observations and suggestions made by the speakers at the workshop and during the discussions that followed the formal presentations. The views contained in this summary are those of individual workshop participants and do not represent the views of workshop participants as a whole, the organizing committee, STEP, COSEPUP, or the National Academies. The summaries of the workshop discussions have been divided into eight chapters. After this introductory chapter, chapter 2 looks at several broad issues involved in the use of performance measures for research. Chapter 3 examines the direct impacts of research on the economy and the quality of life. Chapter 4 considers a closely related topic: the effects of biomedical research on health. Chapter 5 reviews other impacts of research that are not necessarily reflected in economic markets, including international development, agricultural advances, and national security. Chapter 6 moves on to what many speakers cited as one of the most important benefits of research: the training of early career scientific investigators who go on to apply their expertise and knowledge in industry, government, and academia. Chapter 7 summarizes the views of analysts from the United Kingdom, the European Union, and Brazil, highlighting the somewhat different approaches to similar problems being taken in other countries. Chapter 8 examines the emergence of new metrics that may be more powerful in assessing the effects of research on a wide variety of economic and societal indicators. And chapter 9 presents observations made during a final panel presentation on the pitfalls, progress, and opportunities offered by continuing work on measuring the impacts of federal investments in research.

Suggested Citation:"1 INTRODUCTION AND OVERVIEW." National Research Council. 2011. Measuring the Impacts of Federal Investments in Research: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13208.
×

Remarks of Congressman Rush Holt (D-NJ)

At the beginning of the workshop, Congressman Rush Holt, whose 2009 letter initiated the process leading to the workshop, addressed the group by video. His remarks have been slightly shortened.

I can’t emphasize strongly enough the importance of your gathering. Measuring the impact of federal investments in research is a critical need for both government and society. We are living in what may become a pivotal time in our history. For well over half a century we have mined the investments that we made in the immediate aftermath of the Second World War and the fear that gripped us after the launch of Sputnik, from the airplane to the aerospace industry, and from the semiconductor to the Internet. American scientists have built the foundation of the strongest economy in the world.

But the Sputnik era is over. American leadership and our shared prosperity are in peril. As President Obama has said, we’re in need of another Sputnik moment. According to the World Economic Forum’s latest Global Competitiveness Report, the United States ranks fourth in global competitiveness behind Switzerland, Sweden, and Singapore. Further, the World Economic Forum ranks the United States forty-eighth in the quality of math and science education in our schools. Of course, any such rankings of competitiveness or economic or educational achievement are subject to challenge under methodology and, further, those rankings may not be measuring what really can make or keep the United States great or prosperous. However, today 77 percent of global firms planning to build new R and D facilities say they will build them in China or India, not in the United States. In 2009, 51 percent of U.S. patents were awarded to non-U.S. companies. China has gone from fifteenth place to fifth in international patents. Other countries are investing and implementing many of the changes suggested five years ago here in the United States while we continue to hedge and debate. We’re losing our leadership position and our edge in the global economy.

History suggests that our long-term economic prosperity depends on maintaining a robust, modern innovation infrastructure and educational system. That’s why some of us worked hard to

Suggested Citation:"1 INTRODUCTION AND OVERVIEW." National Research Council. 2011. Measuring the Impacts of Federal Investments in Research: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13208.
×

include $22 billion in new R and D funding in the American Recovery and Reinvestment Act. Those funds were an important short—and long-term boost for our economy—short-term in hiring lab technicians and electricians to wire the labs and administrators and clerks to handle the programs, long-term in bringing innovations yet to be determined. Sustainable economic growth will require a sustained investment.

Although our economy has made progress, it continues to struggle. We’re facing a time of serious budget pressure and, perhaps more serious, political pressure that could imperil the support and funding for federal research and development. Some people are suggesting significant cuts for agencies like NSF, NIST, DOE, NIH, NASA, and EPA.

We must be careful stewards of public funds. We need to ensure that our money is being used wisely and efficiently on programs that meet our objectives: creating jobs, building the economy, and creating a sustainable energy future, for example. Yet it is clear to me that cutting federal research funds is not a wise way to balance our budget.

Decision making, whether individual or Congressional, often happens through anecdotes. Nevertheless, we have to be intellectually honest. We have to make sure that the anecdotes are based on something substantial. We need data that will show us what is working and who is being put to work. Evidence can triumph over ideology—sometimes.

You are taking seriously the responsibility to provide hard facts and evidence about our investments. Together, you are building the infrastructure that we need to answer these important questions. I believe that our technological leadership and the foundation of our whole economy depend on it.

Suggested Citation:"1 INTRODUCTION AND OVERVIEW." National Research Council. 2011. Measuring the Impacts of Federal Investments in Research: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13208.
×

This page intentionally left blank.

Suggested Citation:"1 INTRODUCTION AND OVERVIEW." National Research Council. 2011. Measuring the Impacts of Federal Investments in Research: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13208.
×
Page 1
Suggested Citation:"1 INTRODUCTION AND OVERVIEW." National Research Council. 2011. Measuring the Impacts of Federal Investments in Research: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13208.
×
Page 2
Suggested Citation:"1 INTRODUCTION AND OVERVIEW." National Research Council. 2011. Measuring the Impacts of Federal Investments in Research: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13208.
×
Page 3
Suggested Citation:"1 INTRODUCTION AND OVERVIEW." National Research Council. 2011. Measuring the Impacts of Federal Investments in Research: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13208.
×
Page 4
Suggested Citation:"1 INTRODUCTION AND OVERVIEW." National Research Council. 2011. Measuring the Impacts of Federal Investments in Research: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13208.
×
Page 5
Suggested Citation:"1 INTRODUCTION AND OVERVIEW." National Research Council. 2011. Measuring the Impacts of Federal Investments in Research: A Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/13208.
×
Page 6
Next: 2 THE USES AND MISUSES OF PERFORMANCE MEASURES »
Measuring the Impacts of Federal Investments in Research: A Workshop Summary Get This Book
×
Buy Paperback | $54.00 Buy Ebook | $43.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The enactment of the America COMPETES Act in 2006 (and its reauthorization in 2010), the increase in research expenditures under the 2009 American Recovery and Reinvestment Act (ARRA), and President Obama's general emphasis on the contribution of science and technology to economic growth have all heightened interest in the role of scientific and engineering research in creating jobs, generating innovative technologies, spawning new industries, improving health, and producing other economic and societal benefits. Along with this interest has come a renewed emphasis on a question that has been asked for decades: Can the impacts and practical benefits of research to society be measured either quantitatively or qualitatively?

On April 18-19, 2011, the Board on Science, Technology, and Economic Policy (STEP) and the Committee on Science, Engineering and Public Policy (COSEPUP) of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine, held a workshop to examine this question. The workshop sought to assemble the range of work that has been done in measuring research outcomes and to provide a forum to discuss its method. The workshop was motivated by a 2009 letter from Congressman Rush Holt (D-New Jersey). He asked the National Academies to look into a variety of complex and interconnected issues, such as the short-term and long-term economic and non-economic impact of federal research funding, factors that determine whether federally funded research discoveries result in economic benefits, and quantification of the impacts of research on national security, the environment, health, education, public welfare, and decision making.

Measuring the Impacts of Federal Investments in Research provides the key observations and suggestions made by the speakers at the workshop and during the discussions that followed the formal presentations.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!