AN ASSESSMENT OF THE SBIR PROGRAM AT THE NATIONAL SCIENCE FOUNDATION

Committee for Capitalizing on Science, Technology, and Innovation: An Assessment of the Small Business Innovation Research Program

Policy and Global Affairs

Charles W. Wessner, Editor

NATIONAL RESEARCH COUNCIL OF THE NATIONAL ACADEMIES

THE NATIONAL ACADEMIES PRESS

Washington, D.C.
www.nap.edu



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
Committee for Capitalizing on Science, Technology, and Innovation: An Assessment of the Small Business Innovation Research Program Policy and Global Affairs Charles W. Wessner, Editor THE NATIONAL ACADEMIES PRESS Washington, D.C. www.nap.edu

OCR for page R1
THE NATIONAL ACADEMIES PRESS 500 Fifth Street NW Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the Councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This study was supported by Contract/Grant No. DASW01-02-C-0039 between the National Academy of Sciences and U.S. Department of Defense, NASW-03003 between the National Academy of Sciences and the National Aeronautics and Space Administra- tion, DE-AC02-02ER12259 between the National Academy of Sciences and the U.S. Department of Energy, NSFDMI-0221736 between the National Academy of Sciences and the National Science Foundation, and N01-OD-4-2139 (Task Order #99) between the National Academy of Sciences and the U.S. Department of Health and Human Services. The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project. International Standard Book Number-13: 978-0-309-10487-6 International Standard Book Number-10: 0-309-10487-4 Limited copies are available from the Policy and Global Affairs Division, National Research Council, 500 Fifth Street, NW, Washington, D.C. 20001; (202) 334-1529. Additional copies of this report are available from the National Academies Press, 500 Fifth Street, NW, Lockbox 285, Washington, D.C. 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); http://www.nap.edu. Copyright 2008 by the National Academy of Sciences. All rights reserved. Printed in the United States of America

OCR for page R1
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examina- tion of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council. www.national-academies.org

OCR for page R1

OCR for page R1
Committee for Capitalizing on Science, Technology, and Innovation: An Assessment of the Small Business Innovation Research Program Chair Jacques S. Gansler Roger C. Lipitz Chair in Public Policy and Private Enterprise and Director of the Center for Public Policy and Private Enterprise School of Public Policy University of Maryland David B. Audretsch M. Christina Gabriel Distinguished Professor and Director, Innovation Economy Ameritech Chair of Economic The Heinz Endowments Development Trevor O. Jones Director, Institute for Development Strategies Chairman and CEO Indiana University BIOMEC, Inc. Gene Banucci Charles E. Kolb Executive Chairman President ATMI, Inc. Aerodyne Research, Inc. Jon Baron Henry Linsert, Jr. Executive Director Chairman and CEO Coalition for Evidence-Based Policy Martek Biosciences Corporation Michael Borrus W. Clark McFadden Founding General Partner Partner X/Seed Capital Dewey & LeBoeuf, LLP Gail Cassell Duncan T. Moore Vice President, Scientific Affairs and Kingslake Professor of Optical Distinguished Lilly Research Scholar Engineering for Infectious Diseases University of Rochester Eli Lilly and Company Kent Murphy Elizabeth Downing President and CEO CEO Luna Innovations 3D Technology Laboratories 

OCR for page R1
Linda F. Powers Charles Trimble Managing Director CEO, retired Toucan Capital Corporation Trimble Navigation Tyrone Taylor Patrick Windham President President Capitol Advisors Windham Consulting on Technology, LLC PROJECT STAFF Charles W. Wessner Sujai J. Shivakumar Study Director Senior Program Officer McAlister T. Clabaugh Jeffrey McCullough Program Associate Program Associate David E. Dierksheide Program Officer i

OCR for page R1
RESEARCH TEAM Zoltan Acs David H. Finifter University of Baltimore The College of William and Mary Alan Anderson Michael Fogarty Consultant University of Portland Philip A. Auerswald Robin Gaster George Mason University North Atlantic Research Robert-Allen Baker Albert N. Link Vital Strategies, LLC University of North Carolina Robert Berger Benjamin Roberts Robert Berger Consulting, LLC Harvard University Grant Black Rosalie Ruegg University of Indiana South Bend TIA Consulting Peter Cahill Donald Siegel BRTRC, Inc. University of California at Riverside Dirk Czarnitzki Paula E. Stephan University of Leuven Georgia State University Julie Ann Elston Andrew Toole Oregon State University Rutgers University Irwin Feller Nicholas Vonortas American Association for the George Washington University Advancement of Science ii

OCR for page R1
POLICY AND GLOBAL AFFAIRS Ad hoc Oversight Board for Capitalizing on Science, Technology, and Innovation: An Assessment of the Small Business Innovation Research Program Robert M. White, Chair University Professor Emeritus Electrical and Computer Engineering Carnegie Mellon University Anita K. Jones Mark B. Myers Lawrence R. Quarles Professor of Visiting Professor of Management Engineering and Applied Science The Wharton School School of Engineering and Applied University of Pennsylvania Science University of Virginia iii

OCR for page R1
Contents PREFACE xv SuMMARY 1 1 INTRODuCTION 12 1.1 Small Business Innovation Research Program Creation and Assessment, 12 1.2 SBIR Program Structure, 13 1.3 SBIR Reauthorizations, 14 1.4 Structure of the NRC Study, 15 1.5 SBIR Assessment Challenges, 16 2 FINDINGS AND RECOMMENDATIONS 21 3 SCOPE AND METHODOLOGY 41 3.1 Study Scope, 41 3.2 Methodology Design, 41 3.3 Methods Used, 42 3.3.1 Surveys, 42 3.3.2 Interviews with NSF SBIR Managers, 43 3.3.3 Review of Program Documents and Data, 43 3.3.4 Case Studies of Firms, 43 ix

OCR for page R1
x CONTENTS 4 OVERVIEW OF THE NSF SBIR PROGRAM 47 4.1 A Brief History of NSF’s SBIR Program, 47 4.1.1 The Nation’s First SBIR Program, 47 4.1.2 Factors That Led to Establishment of the NSF’s SBIR Program, 47 4.1.3 The NSF’s Early Emphasis on Commercialization, 48 4.2 NSF SBIR Demographics, 49 4.2.1 Description of the NSF’s SBIR Grants, 49 4.2.2 Annual Dollar Outlays and Number of Grants, 50 4.2.3 Applications and Success Rates, 55 4.2.4 Geographical Location of Grants and Applicants, 56 4.2.5 Women and Minorities as Applicants, Grantees, and Principal Investigators, 59 4.2.6 Multiple Grant Winners and New Grant Winners, 69 4.3 Program Organization and Structure, 73 4.3.1 Organization, 73 4.3.2 Staffing, 73 4.4 Descriptive Overview of the NSF’s SBIR Program, 75 4.4.1 Primary Program Objectives, 75 4.4.2 Program Phases, 76 4.4.3 Use of Topics, 76 4.4.4 Proposal Selection Criteria and Process, 78 4.4.5 Solicitations and Proposal Submissions, 79 4.4.6 Support for Commercialization, 79 4.5 NSF SBIR “Success Stories”, 80 5 COMMERCIALIZATION 83 5.1 Commercialization Strategies, 83 5.2 Commercial Results, 85 5.2.1 Characteristics of SBIR-Funded Firms as Indicated by NRC Firm Survey Data, 85 5.2.2 Commercialization Progress Indicated by NRC Phase II Survey Data, 89 5.2.3 Projects Not Continuing into Phase II as Revealed by NRC Phase I Survey Data, 99 5.2.4 Commercialization as Illustrated by Selected Case Study Data, 102 5.2.5 Commercial Progress as Indicated by Agency-Initiated Data and Analysis, 113 5.2.6 Commercialization Insights Provided by a Committee of Visitors, 117 5.3 Conclusions on Commercialization, 117

OCR for page R1
xi CONTENTS 6 SuPPORT TO AGENCY MISSION AND TO SMALL BuSINESS 120 6.1 Agency Differences: Contracting versus Grant Agencies, 120 6.2 NSF—a Nonprocuring Agency, 121 6.3 NSF Support of Small Business, 122 6.3.1 Basic Demographics of NSF Support for Small Business, 122 6.3.2 NSF Small Business Research Funding as a Share of NSF R&D Spending, 122 7 CONTRIBuTIONS TO KNOWLEDGE 125 7.1 Research Program Perspectives, 125 7.1.1 Attention to Research Quality and Knowledge Creation, 125 7.1.2 Attention to Knowledge Dissemination and Spillover Effects, 126 7.2 NRC Study Findings on Knowledge Creation and Dissemination by the NSF’s SBIR Program, 127 7.2.1 Patents, Copyrights, Trademarks, and Scientific Publications, 127 7.2.2 Licensing, 127 7.2.3 Tracking Knowledge Dissemination by Citation Analysis, 129 7.2.4 Equity Sales, 130 7.2.5 Partnerships of Small Firms with Other Companies and Investors, 130 7.2.6 Small Firms and Universities, 131 7.2.7 Risk Profile, 132 7.3 Indicators, Not Measures of Benefit, 135 8 PROGRAM MANAGEMENT 136 8.1 Topic Development and Selection, 136 8.1.1 Topics, 136 8.1.2 Sources for Topic Ideas, 137 8.1.3 Agency-Driven versus Investigator-Driven Approach to Topics, 141 8.1.4 Topic Decision Making, 142 8.2 Outreach, 144 8.2.1 Agency Outreach Objectives, 145 8.2.2 Outreach Programs, 146 8.2.3 Agency Outreach Benchmarks and Metrics, 147 8.3 Grant Selection, 148 8.3.1 Description of Selection Processes for Phase I, Phase II, and Phase IIB Grants, 148 8.3.2 Peer Review Panels—Membership, Selection, and Qualifications, 152 8.3.3 Transparency of Selection Process, 154

OCR for page R1
xii CONTENTS 8.3.4 Scoring Procedures, 155 8.3.5 Role of Program Manager, 156 8.3.6 Resubmission Procedures and Outcomes, 156 8.4 Training (After Successful Application), 157 8.4.1 Training Programs for Agency Phase I and Phase II Grantees, 158 8.4.2 Rating the Effectiveness of Various Training Efforts, 160 8.4.3 Take-up Rates and Projections, 161 8.4.4 Constraints on Commercialization Assistance Training, 161 8.5 The NSF Phase IIB Program, 162 8.5.1 Description, 162 8.5.2 Use of Matching Funds, 163 8.5.3 Application and Selection Procedures, 164 8.5.4 Role of Program Manager in Phase IIB, 165 8.6 The Grant Cycle and Funding Gaps, 165 8.6.1 Phase I to Phase II Gap, 165 8.6.2 Other Funding Sources, 168 8.6.3 Bridge Funding Programs (After Phase II), 170 8.7 Reporting Requirements, 171 8.7.1 Reports Submitted to the Agency by SBIR Winners, 171 8.7.2 Report Utilization and Utility to the Agency, 172 8.8 Evaluation and Assessment, 172 8.8.1 Annual and Intermittent Agency Evaluation of its SBIR Program, 172 8.8.2 Operational Benchmarks for NSF’s SBIR Program, 174 8.8.3 Evaluators (Internal and External), 174 8.8.4 Annual Evaluation and Assessment Budget, 175 8.9 Flexibility, 175 8.9.1 Program Manager Discretion, 175 8.9.2 Program Manager Funding Discretion, 175 8.9.3 Program Manager Perceptions of Constraints, 176 8.10 Size—Funding Amounts and Sources, 176 8.10.1 Formal and Effective Limits on Size and Duration of Grants, 176 8.10.2 Distribution of Funding to Phase I and Phase II Grants Within the Specified Limits, 178 8.11 Online Capabilities and Plans, 181 8.11.1 FastLane System, 181 8.11.2 Barriers to Online Capabilities and Plans, 182 8.12 Administrative Resources, 182 8.12.1 Funding of Program Administration, 182 8.12.2 Administration Budget as a Percentage of Agency SBIR Funding, 183

OCR for page R1
xiii CONTENTS 8.12.3 Evaluation and Assessment Funding, 183 8.13 Best Practices and Program Evolution, 183 8.13.1 Adoption of Best Practices from Other Agencies, 183 8.13.2 Evolution of the NSF’s Program During the NRC Study, 184 8.14 Conclusion, 187 APPENDIxES A NSF SBIR PROGRAM DATA 191 B NRC PHASE II SuRVEY AND NRC FIRM SuRVEY 205 C NRC PHASE I SuRVEY 231 D SELECTED CASE STuDIES 240 Faraday Technology, Inc., 249 Immersion Corporation, 259 ISCA Technology, Inc., 266 Language Weaver, 273 MER Corporation, 283 MicroStrain, Inc., 288 National Recovery Technologies, Inc., 296 NVE Corporation, 304 T/J Technologies, Inc., 311 WaveBand Corporation, 320 E STRuCTuRED INTERVIEW GuIDE uSED IN CORYELL STuDY OF PHASE IIB GRANTS 326 F REquIRED NSF POST-GRANT ANNuAL COMMERCIALIZATION REPORT (DEEMED INEFFECTIVE BY THE NSF) 335 G BIBLIOGRAPHY 337

OCR for page R1

OCR for page R1
Preface Today’s knowledge economy is driven in large part by the nation’s capac- ity to innovate. One of the defining features of the U.S. economy is a high level of entrepreneurial activity. Entrepreneurs in the United States see opportuni- ties and are willing and able to take on risk to bring new welfare-enhancing, wealth-generating technologies to the market. Yet, while innovation in areas such as genomics, bioinformatics, and nanotechnology present new opportunities, converting these ideas into innovations for the market involves substantial chal- lenges.1 The American capacity for innovation can be strengthened by addressing the challenges faced by entrepreneurs. Public-private partnerships are one means to help entrepreneurs bring new ideas to market.2 The Small Business Innovation Research (SBIR) program is one of the largest examples of U.S. public-private partnerships. Founded in 1982, the SBIR program was designed to encourage small business to develop new processes and products and to provide quality research in support of the many missions of the U.S. government. By including qualified small businesses in the nation’s R&D (research and development) effort, SBIR grants are intended to stimulate innova- tive new technologies to help agencies meet the specific research and develop- ment needs of the nation in many areas, including health, the environment, and national defense. 1See Lewis M. Branscomb, Kenneth P. Morse, Michael J. Roberts, Darin Boville, Managing Techni- cal Risk: Understanding Priate Sector Decision Making on Early Stage Technology Based Projects, Gaithersburg, MD: National Institute of Standards and Technology, 2000. 2For a summary analysis of best practice among U.S. public-private partnerships, see National Research Council, Goernment-Industry Partnerships for the Deelopment of New Technologies: Summary Report, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2002. x

OCR for page R1
xi PREFACE As the SBIR program approached its twentieth year of operation, the U.S. Congress asked the National Research Council to conduct a “comprehensive study of how the SBIR program has stimulated technological innovation and used small businesses to meet federal research and development needs” and to make recommendations on still further improvements to the program. 3 To guide this study, the National Research Council drew together an expert committee that included eminent economists, small businessmen and women, and venture capitalists, led by Dr. Jacques Gansler of the University of Maryland (formerly Undersecretary of Defense for Acquisition and Technology.) The membership of this committee is listed in the front matter of this volume. Given the extent of ‘green-field research’ required for this study, the Committee in turn drew on a distinguished team of researchers to, among other tasks, administer surveys and case studies, and develop statistical information about the program. The member- ship of this research team is also listed in the front matter of this volume. This report is one of a series published by the National Academies in response to the congressional request. The series includes reports on the Small Business Innovation Research Program at the Department of Defense, the Depart- ment of Energy, the National Aeronautics and Space Administration, the National Institutes of Health, and the National Science Foundation—the five agencies responsible for 96 percent of the program’s operations. It includes, as well, an Overview Report that provides assessment of the program’s operations across the federal government. Other reports in the series include a summary of the 2002 conference that launched the study, and a summary of the 2005 conference on SBIR and the Phase III Challenge of Commercialization that focused on the Department of Defense and NASA. PROJECT ANTECEDENTS The current assessment of the SBIR program follows directly from an earlier analysis of public-private partnerships by the National Research Council’s Board on Science, Technology, and Economic Policy (STEP). Under the direction of Gordon Moore, Chairman Emeritus of Intel, the NRC Committee on Govern- ment-Industry Partnerships prepared eleven volumes reviewing the drivers of cooperation among industry, universities, and government; operational assess- ments of current programs; emerging needs at the intersection of biotechnology and information technology; the current experience of foreign government part- nerships and opportunities for international cooperation; and the changing roles of government laboratories, universities, and other research organizations in the national innovation system.4 3See the SBIR Reauthorization Act of 2000 (H.R. 5667-Section 108). 4For a summary of the topics covered and main lessons learned from this extensive study, see National Research Council, Goernment-Industry Partnerships for the Deelopment of New Technolo- gies: Summary Report, op. cit.

OCR for page R1
xii PREFACE This analysis of public-private partnerships included two published studies of the SBIR program. Drawing from expert knowledge at a 1998 workshop held at the National Academy of Sciences, the first report, The Small Business Innoa- tion Research Program: Challenges and Opportunities, examined the origins of the program and identified some operational challenges critical to the program’s future effectiveness.5 The report also highlighted the relative paucity of research on this program. Following this initial report, the Department of Defense (DoD) asked the NRC to assess the Department’s Fast Track Initiative in comparison with the operation of its regular SBIR program. The resulting report, The Small Business Innoation Research Program: An Assessment of the Department of Defense Fast Track Initiatie, was the first comprehensive, external assessment of the Depart- ment of Defense’s program. The study, which involved substantial case study and survey research, found that the SBIR program was achieving its legislated goals. It also found that DoD’s Fast Track Initiative was achieving its objective of greater commercialization and recommended that the program be continued and expanded where appropriate.6 The report also recommended that the SBIR program overall would benefit from further research and analysis, a perspective adopted by the U.S. Congress. SBIR REAuTHORIZATION AND CONGRESSIONAL REquEST FOR REVIEW As a part of the 2000 reauthorization of the SBIR program, Congress called for a review of the SBIR programs of the agencies that account collectively for 96 percent of program funding. As noted, the five agencies meeting this criterion, by size of program, are the Department of Defense, the National Institutes of Health, the National Aeronautics and Space Administration, the Department of Energy, and the National Science Foundation. Congress directed the NRC, via H.R. 5667, to evaluate the quality of SBIR research and evaluate the SBIR program’s value to the agency mission. It called for an assessment of the extent to which SBIR projects achieve some measure of commercialization, as well as an evaluation of the program’s overall economic and noneconomic benefits. It also called for additional analysis as required to support specific recommendations on areas such as measuring outcomes for 5See National Research Council, The Small Business Innoation Research Program: Challenges and Opportunities, Charles W. Wessner, ed., Washington, DC: National Academy Press, 1999. 6See National Research Council, The Small Business Innoation Research Program: An Assess- ment of the Department of Defense Fast Track Initiatie, Charles W. Wessner, ed., Washington, DC: National Academy Press, 2000. Given that virtually no published analytical literature existed on SBIR, this Fast Track study pioneered research in this area, developing extensive case studies and newly developed surveys.

OCR for page R1
xiii PREFACE agency strategy and performance, increasing federal procurement of technologies produced by small business, and overall improvements to the SBIR program. 7 ACKNOWLEDGMENTS On behalf of the National Academies, we express our appreciation and recognition for the insights, experiences, and perspectives made available by the participants of the conferences and meetings, as well as by survey respondents and case study interviewees who participated over the course of this study. We are also very much in debt to officials from the leading departments and agencies. Among the many who provided assistance to this complex study, for this volume, we are especially in debt to Kesh Narayanan, Joseph Hennessey, and Ritchie Coryell of the National Science Foundation. Valuable, independent contributions and observations were provided by Roland Tibbetts, formerly of the National Science Foundation. The Committee’s research team deserves recognition for their instrumental role in the preparation and many revisions of this report. In that regard, special thanks are due to Rosalie Ruegg of TIA Consulting who served as the lead researcher for the NSF study. Her timely and insightful contributions played a key role in the committee’s analysis. Without their collective efforts and close coop- eration, amidst many other competing priorities, it would not have been possible to prepare this report. Among the many contributing Committee members, special thanks are due to Christina Gabriel, Kent Murphy, and Patrick Windham. NATIONAL RESEARCH COuNCIL REVIEW This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Academies’ Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the process. We wish to thank the following individuals for their review of this report: Heidi Jacobus, Cybernet Systems Corporation; Brad Knox, Aflac Insurance; Jeanne Powell, National Institute of Standards and Technology; and Richard Wright, National Institute of Standards and Technology. 7Chapter 3 of the Committee’s Methodology Report describes how this legislative guidance was drawn out in operational terms. National Research Council, An Assessment of the Small Business Inno- ation Research Program: Project Methodology, Washington, DC: The National Academies Press, 2004. Access this report at .

OCR for page R1
xix PREFACE Although the reviewers listed above have provided many constructive com- ments and suggestions, they were not asked to endorse the conclusions or recom- mendations, nor did they see the final draft of the report before its release. The review of this report was overseen by Robert Frosch, Harvard University, and Robert White, Carnegie Mellon University. Appointed by the National Acad- emies, they were responsible for making certain that an independent examina- tion of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution. Jacques S. Gansler Charles W. Wessner

OCR for page R1