Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 1
An Assessment of the SBIR Program Summary I. INTRODUCTION The Small Business Innovation Research (SBIR) program was created in 1982 through the Small Business Innovation Development Act. In 2005, the 11 federal agencies administering the SBIR program disbursed over $1.85 billion dollars in competitive awards to innovative small firms. As the SBIR program approached its twentieth year of operation, the U.S. Congress requested the National Research Council (NRC) of the National Academies to “conduct a comprehensive study of how the SBIR program has stimulated technological innovation and used small businesses to meet Federal research and development needs” and to make recommendations with respect to the SBIR program. Mandated as a part of SBIR’s reauthorization in late 2000, the NRC study has assessed the SBIR program as administered at the five federal agencies that together make up some 96 percent of SBIR program expenditures. The agencies, in order of program size, are the Department of Defense (DoD), the National Institutes of Health (NIH), the National Aeronautics and Space Administration (NASA), the Department of Energy (DoE), and the National Science Foundation (NSF). Based on that legislation, and after extensive consultations with both Congress and agency officials, the NRC focused its study on two overarching questions.1 First, how well do the agency SBIR programs meet four societal objectives 1 Three primary documents condition and define the objectives for this study: These are the Legislation—H.R. 5667, the NAS-Agencies Memorandum of Understanding, and the NAS contracts accepted by the five agencies. These are reflected in the Statement of Task addressed to the Committee by the Academies’ leadership. Based on these three documents, the NRC Committee developed a comprehensive and agreed-upon set of practical objectives to be reviewed. These are outlined in the Committee’s formal Methodology Report, particularly Chapter 3: Clarifying Study Objec
OCR for page 2
An Assessment of the SBIR Program of interest to Congress? That is: (1) to stimulate technological innovation; (2) to increase private sector commercialization of innovations; (3) to use small business to meet federal research and development needs; and (4) to foster and encourage participation by minority and disadvantaged persons in technological innovation.2 Second, can the management of agency SBIR programs be made more effective? Are there best practices in agency SBIR programs that may be extended to other agencies’ SBIR programs? To satisfy the congressional request for an external assessment of the program, the NRC analysis of the operations of the SBIR program involved multiple sources and methodologies. A large team of expert researchers carried out extensive NRC-commissioned surveys and case studies. In addition, agency-compiled program data, program documents, and the existing literature were reviewed. These were complemented by extensive interviews and discussions with program managers, program participants, agency “users” of the program, as well as program stakeholders. The study as a whole sought to understand operational challenges and to measure program effectiveness, including the quality of the research projects being conducted under the SBIR program, the challenges and achievements in commercialization of the research, and the program’s contribution to accomplishing agency missions. To the extent possible, the evaluation included estimates of the benefits (both economic and noneconomic) achieved by the SBIR program, as well as broader policy issues associated with public-private collaborations for technology development and government support for high technology innovation. Taken together, this study is the most comprehensive assessment of SBIR to date. Its empirical, multifaceted approach to evaluation sheds new light on the operation of the SBIR program in the challenging area of early-stage finance. As with any assessment, particularly one across five quite different agencies and departments, there are methodological challenges. These are identified and discussed at several points in the text.3 This important caveat notwithstanding, the scope and diversity of the report’s research should contribute significantly to the understanding of the SBIR program’s multiple objectives, measurement issues, operational challenges, and achievements. tives. See National Research Council, An Assessment of the Small Business Innovation Research Program—Project Methodology, Washington, DC: The National Academies Press, 2004, accessed at <http://books.nap.edu/catalog.php?record_id=11097#toc>. 2 These congressional objectives are found in the Small Business Innovation Development Act (PL 97-219). In reauthorizing the program in 1992 (PL 102-564), Congress expanded the purposes to “emphasize the program’s goal of increasing private sector commercialization developed through Federal research and development and to improve the Federal government’s dissemination of information concerning small business innovation, particularly with regard to woman-owned business concerns and by socially and economically disadvantaged small business concerns.” 3 See, for example, Box 4-1 in Chapter 4, which discusses the multiple sources of bias in innovation surveys.
OCR for page 3
An Assessment of the SBIR Program II. SUMMARY OF KEY FINDINGS The core finding of the study is that the SBIR program is sound in concept and effective in practice. It can also be improved. Currently, the program is delivering results that meet most of the congressional objectives. Specifically, the program is: Stimulating Technological Innovation4 Generating Multiple Knowledge Outputs. SBIR projects yield a variety of knowledge outputs. These contributions to knowledge are embodied in data, scientific and engineering publications, patents and licenses of patents, presentations, analytical models, algorithms, new research equipment, reference samples, prototypes products and processes, spin-off companies, and new “human capital” (enhanced know-how, expertise, and sharing of knowledge).5 Linking Universities to the Public and Private Markets. The SBIR program supports the transfer of research into the marketplace, as well as the general expansion of scientific and technical knowledge, through a wide variety of mechanisms. With regard to SBIR’s role in linking universities to the market, three metrics from the NRC Phase II Survey and NRC Firm Survey reveal the relatively high level of university connections. Over a third of respondents to the NRC surveys reported university involvement in their SBIR project. Among those reporting university involvement, More than two-thirds of companies reported that at least one founder was previously an academic; About one-third of founders were most recently employed as academics before founding the company; and Some 27 percent of projects had university faculty as contractors on the project, 17 percent used universities themselves as subcontractors, and 15 percent employed graduate students. These data underscore the significant level of involvement by universities in the program and highlight the program’s contribution to the transition of university research to the marketplace.6 4 See related Finding F in Chapter 2. 5 Surveys commissioned by the NRC for this study indicate that 29.3 percent of projects received at least one patent related to their SBIR research (see Table 4-11). The NRC survey also determined that 45.4 percent of respondents reported publishing at least one related peer-reviewed scientific paper (see Table 4-12). These data fit well with case studies and interviews of firms, which suggest that SBIR companies are proud of the quality of their research. Further, as highlighted in some firm interviews, metrics like patents and publications do not tell the full story; there may be benefits from the development and diffusion of knowledge that are not reflected in any qualitative metric. See, for example, the case study of Language Weaver in Appendix C of this report. 6 See Section 18.104.22.168 on university faculty and company formation. Also see Table 4-13 on university involvement in SBIR projects.
OCR for page 4
An Assessment of the SBIR Program Increasing Private Sector Commercialization of Innovations7 A Commercial Enabler for Small Firms. Small technology companies use SBIR awards to advance projects, develop firm-specific capabilities, and ultimately create and market new commercial products and services. Company Creation. Just over 20 percent of companies responding to the NRC Firm Survey indicated that they were founded entirely or partly because of a prospective SBIR award.8 The Decision to Initiate Research. Companies responding to the NRC Phase II Survey reported that over two-thirds of SBIR projects would not have taken place without SBIR funding.9 Providing Alternative Development Paths. Companies often use SBIR to fund alternate development strategies, exploring technological options in parallel with other activities. Reaching the Market. Although the data vary by agency, respondents to the NRC Phase II Survey indicate that just under half of the projects do reach the marketplace.10 Given the very early stage of SBIR investments, and the high degree of technical risk involved (reflected in risk assessment scores developed during some agency selection procedures), the fact that a high proportion of projects reach the market place in some form is significant, even impressive. A Small Percentage of Projects Account for Most Successes. As with investments made in early stage companies by angel investors or venture capitalists, SBIR awards result in sales numbers that are highly skewed.11 A small percentage of projects will likely achieve large growth and significant sales revenues—i.e., become commercial “home runs.” Meanwhile many small successes together will continue to meet agency research needs and 7 See related Finding B in Chapter 2. 8 See Chapter 4, Table 4-7. 9 See Figure 4-19. 10 There are many different sources of commercialization data, all of which suggest commercialization activity. Survey data from the NRC (see Appendix A) indicates that 47 percent of respondents for awards made 1992-2002 report some sales (see Section 4.2.3); resurveying respondents to their own survey from approximately the same time period, NIH found that 62.8 percent reported sales. (See National Institutes of Health, Report on the Second of the 2005 Measures Updates: NIH SBIR Performance Outcomes Data System (PODS), September 26, 2005 p. 10.) DoD also maintains an outcomes database where data are submitted by firms each time they apply for another award. This database also shows that the percentage of projects grows as time elapses, and again generates comparable figures. For example, approximately 53 percent of Navy projects from 1999 have reached the market. (DoD Company Commercialization Report Database, provided to NRC by DoD in August 2005.) It therefore appears that about half of all funded projects reach the market; follow-on research similar to that conducted by NIH is likely to generate numbers that are both higher and better reflective of the long run commercialization from the program. See National Institutes of Health, Report on the Second of the 2005 Measures Updates: NIH SBIR Performance Outcomes Data System (PODS), op. cit., p. 10. 11 See Figure 4-2.
OCR for page 5
An Assessment of the SBIR Program comprise a potentially important contribution to the nation’s innovative capability. SBIR Is an Input, Not a Panacea. SBIR can be a key input to encourage small business commercialization, but most major commercialization successes require substantial post-SBIR research and funding from a variety of sources.12 SBIR awards will have been in many cases a major, even decisive input—but only one of the many contributions needed for success. Using Small Businesses to Meet Federal Research and Development Needs13 Flexible Adaptation to Agency Mission. The effective alignment of the program with widely varying mission objectives, needs, and modes of operation is a central challenge for an award program that involves a large number of departments and agencies. The SBIR program has been adapted effectively by the management of the individual departments, services, and agencies, albeit with significant differences in mode of operation reflecting their distinct missions and operational cultures. This flexibility in program management and modes of operation is one of the great strengths of the program. Meeting Agency Procurement Needs. The SBIR program helps to meet the procurement needs of diverse federal agencies. At the Department of Defense, the Navy has achieved significant success in improving the insertion of SBIR-funded technologies into the acquisition process. The commitment of upper management to the effective operation of the program appears to be a key element of this success. Teaming among the SBIR program managers, agency procurement managers, the SBIR awardees, and, increasingly, the prime contractors is important in the transition of technologies from projects to products to integration in systems. At DoD, the growing importance of the SBIR program within the defense acquisition system is reflected in the growing interest of prime contractors, who are seeking opportunities to be in support of SBIR projects—a key step toward acquisition.14 Providing Widely Distributed Support for Innovation Activity15 Large Number of Firms. During the fourteen years between 1992 and 12 See Table 4-2 for a list of sources of additional investments. 13 See related Finding C in Chapter 2. 14 The growing interest of Defense prime contractors is recorded in National Research Council, SBIR and the Phase III Challenge of Commercialization, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2007. 15 See related Finding D in Chapter 2.
OCR for page 6
An Assessment of the SBIR Program 2005, inclusive, more than 14,800 firms received at least one Phase II award, according to the SBA Tech-Net database.16 Many New Participants. Each year, over one third of the firms awarded SBIR funds participate in the program for the first time.17 This steady infusion of new firms is a major strength of the program and suggests that SBIR is encouraging innovation across a broad spectrum of firms, creating additional competition among suppliers for the procurement agencies, and providing agencies new mission-oriented research and solutions. Fostering Participation by Minority and Disadvantaged Persons in Technological Innovation18 A Mixed Record. Woman- and minority-owned firms face substantial challenges in obtaining early-stage finance.19 Recognizing these challenges, the legislation calls for fostering and encouraging the participation of women and minorities in SBIR. Given this objective, some current trends are troubling. Agencies do not have a uniformly positive record in collecting data and monitoring funding flows for research by woman- and minority-owned firms. While support for woman-owned businesses is increasing, support for minority-owned firms has not increased. For example, at DoD, which accounts for over half the SBIR program funding, the share of Phase II awards going to woman-owned businesses increased from 8 percent at the time of the 1992 reauthorization (1992-1994) to 9.5 percent (in a program increasing in overall size) for the most recent years covered by the NRC Phase II Survey (1999-2001).20 The share of Phase I awards to minority-owned firms at DoD has declined quite substantially since the mid 1990s and fell below 10 percent for the first time in 2004 and 2005.21 Data on Phase II awards suggest that the decline in Phase I award shares for minority-owned firms is reflected in Phase II. 16 See U.S. Small Business Administration, Tech-Net Database, accessed at <http://tech-net.sba.gov/index.cfm>. 17 See Section 4.5.3 for a discussion of the incidence of new entrants. See also Figures 4-21, 4-22, 4-23, and 4-24 for data on new entrants at NSF, NIH and DoD. 18 See related Finding E in Chapter 2. 19 Academics represent an important future pool of applicants, firm founders, principal investigators, and consultants. Recent research shows that owing to the low number of women in senior research positions in many leading academic science departments, few women have the chance to lead a spinout. “Under-representation of female academic staff in science research is the dominant (but not the only) factor to explain low entrepreneurial rates amongst female scientists.” See Peter Rosa and Alison Dawson, “Gender and the commercialization of university science: academic founders of spinout companies,” Entrepreneurship & Regional Development, 18(4):341-366, July 2006. 20 See Figure 2-4 in National Research Council, An Assessment of the SBIR Program at the Department of Defense, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2009. 21 This statistic is drawn from the DoD Awards database.
OCR for page 7
An Assessment of the SBIR Program Documenting and monitoring the participation by women and minorities is complex, given, inter alia, the variations in the demographics of the applicant pool. In some cases, agency efforts in this area have been inadequate. Agencies are encouraged to collect, analyze, and regularly report on this important element of the program.22 Support for Woman and Minority-Principal Investigators. Beyond support for woman- and minority-owned firms, support for woman and minority principal investigators can be an important step, supporting the potential entrepreneurs of the future. III. SUMMARY OF KEY RECOMMENDATIONS The recommendations below are intended to improve the operation of an already effective program. They seek to maintain, and reinforce, positive features of program management, such as the flexibility in approach by different agencies. They also identify pressing needs, e.g., for better data collection and analysis and opportunities for improvements in program operations in areas such as award size, cycle time, and outreach to minorities.23 Retain Program Flexibility24 SBA and SBIR. The SBA has oversight responsibility for the eleven SBIR programs underway across the federal government. The agency is to be commended for its flexibility in exercising its oversight responsibilities, which allows the agencies to adapt the program to fit their needs and methods of operation. This flexibility has proven fundamental to the program’s success, and should be preserved. Encourage Program Innovation.25 As noted above, it is essential to retain and encourage the flexibility that enables SBIR program management to innovate towards an even more effective multiphase program. Preserve the Basic Program Structure.26 The three phase approach of the SBIR program should be maintained. Proposals to “bypass” Phase I are neither necessary nor appropriate. Permitting companies to apply directly to Phase II would have the potential to change the program, significantly reducing funds for Phase I. Such a shift does not seem necessary given the current flexibility in award size. 22 This is a correction of the text in the prepublication version released on July 27, 2007. 23 These recommendations are provided in substantially greater detail in Chapter 2. 24 See related Recommendation A in Chapter 2. 25 See related Recommendation H in Chapter 2. 26 See related Recommendation G in Chapter 2.
OCR for page 8
An Assessment of the SBIR Program Conduct Regular Evaluations.27 Regular, rigorous program evaluation is essential for quality program management and accountability, and improved program output. Accordingly, the SBIR program managers should give greater attention and resources to the systematic evaluation of the program supported by reliable data and should seek to make the program as responsive as possible to the needs of small company applicants. Annual Reports. Top agency management should make a direct annual report to Congress on the state of the SBIR program at their agency. This report should include a statistical appendix, which would provide data on awards, processes, outcomes, and survey information. Internal Evaluation. Agencies should be encouraged—and funded—to develop improved data collection technologies and evaluation procedures. Where possible, agencies should be encouraged to develop interoperable standards for data collection and dissemination.28 External Evaluation. Agencies should be directed to commission an external evaluation of their SBIR programs on a regular basis. Improve Program Processes Topic Definition.29 SBIR program managers should ensure that solicitation topics are broadly defined and that topics are defined from the “bottom-up” based on agency mission needs. Project Selection.30 Agencies should also ensure that project selection procedures are transparent and flexible and are attuned to the needs of small businesses. Cycle Time.31 The processing periods for awards vary substantially by agency, and appear to have significant effects on recipient companies.32 Agencies should closely monitor and report on cycle times for each element of the SBIR program: topic development and publication, solicitation, application review, contracting, Phase II application and selection, and Phase III contracting. Agencies should also specifically report on initiatives to shorten decision cycles. Pilot Programs.33 The agencies should be strongly encouraged to develop pilot programs to address possible improvements to the SBIR program. Agencies should equally ensure that such program modifications are de- 27 See related Recommendation B in Chapter 2. 28 The agencies should consider providing data from a range of sources—including agency data-bases, agency surveys, the patent office, and bibliographic databases, along with data from award recipients themselves. 29 See related Recommendation D in Chapter 2. 30 Ibid. 31 Ibid. 32 Drawn from case study interviews and gap data from the NRC Phase II Survey. 33 See related Recommendation H in Chapter 2.
OCR for page 9
An Assessment of the SBIR Program signed, monitored and evaluated, so that positive and negative results can be effectively determined. Readjust Award Sizes34 One-time Adjustment. The real value of SBIR awards, last increased in 1995, has eroded due to inflation. Given that Congress did not indicate that the real value of awards should be allowed to decline, this erosion in the value of awards needs to be addressed. In order to restore the program to the approximate initial levels, adjusted for inflation, the Congress should consider making a one-time adjustment that would give the agencies latitude to increase the standard size of Phase I awards to $150,000, and to increase the standard size of Phase II awards to approximately $1,000,000.35 Maintain Flexibility. It should be stressed that recommendations are intended as guidance for standard award size. The SBA should continue to provide the maximum flexibility possible with regard to award size and the agencies should continue to exercise their judgment in applying the program standard. The diversity of agency and project needs does not permit a one-size-fits-all approach. Continue to Focus on Increased Private-sector Commercialization Encourage Continued Experimentation.36 The agencies should be strongly encouraged to develop programs that seek to improve the commercialization outcomes of the SBIR program. Some agencies have sought, with the approval of SBA, to experiment with SBIR funding beyond Phase II in order to improve the commercialization potential of SBIR funded technologies. NIH has substantially increased its use of supplementary awards—additional funding provided largely at the discretion of the program manager to help meet unexpected research costs. The NSF Phase IIB initiative and the NIH Competing Continuation Awards are positive examples that might well be adapted elsewhere. Mission Agencies Create a Phase III Pull.37 By working with prime contractors, create mechanisms (such as the Navy’s Phase IIB SBIR or Phase III funding with program dollars) to help bridge the “Valley of Death” between Phase II and application funding. 34 See related Recommendation I in Chapter 2. 35 Recognizing that these values are not identical, the erosion by inflation of the amounts available for Phase I appear more constraining than for Phase II. One may argue that, for parity, the Phase II should be raised to approximately $1,125,000. The trade-offs involve attracting adequate numbers of quality proposals, and providing sufficient resources to enable firms to actually carry out the necessary work, while also minimizing the impact on the number of awards. While recognizing that there is necessarily an arbitrary element in fixing these amounts, the Committee is confident that a $1,000,000 Phase II award will maintain the program’s attraction to innovative small businesses. 36 See related Recommendation H in Chapter 2. 37 Ibid.
OCR for page 10
An Assessment of the SBIR Program Multiple Winners Should Be Judged on Output, Not Numbers of Awards.38 In the case of multiple award winners who qualify in terms of the selection criteria, the acceptance/rejection decision should be based on their performance on past grants in terms of commercialization success and addressing agency needs, rather than on the number of grants received. Firms able to provide quality solutions to solicitations should not be excluded, a priori, from the program except on clear and transparent criteria (e.g., quality of research and/or commercialization performance). Improve Participation and Success by Women and Minorities39 Improve Data Collection and Analysis. Agencies should arrange for an independent analysis of a sample of past proposals from woman- and minority-owned firms and from other firms (to serve as a control group). This will help identify specific factors accounting for the lower success rates of woman- and minority-owned firms, as compared with other firms, in having their Phase I proposals granted. Extend Outreach to Younger Women and Minority Students. Agencies should be encouraged to solicit women and underrepresented minorities working at small firms to apply as principal investigators and senior co-investigators for SBIR awards, and should track their success rates. Encourage Participation. Agencies should develop targeted outreach to improve the participation rates of woman- and minority-owned firms, and strategies to improve their success rates based on causal factors determined by analysis of past proposals and feedback from the affected groups.40 Increase Management Funding for SBIR41 Enhance Program Utilization. To enhance program utilization, management, and evaluation, consideration should be given to the provision of additional program funds for management and evaluation. Additional funds might be allocated internally within the existing agency budgets, drawn from the existing set-aside for the program, or by modestly increasing the set-aside for the program, currently at 2.5 percent of external research budgets. Optimize the Return on Investment. The key point is that a modest addition to funds for program management and evaluation are necessary to 38 See related Recommendation J in Chapter 2. 39 See related Recommendation E in Chapter 2. 40 This recommendation should not be interpreted as lowering the bar for the acceptance of proposals from woman- and minority-owned companies, but rather as assisting them to become able to meet published criteria for grants at rates similar to other companies on the basis of merit, and to ensure that there are no negative evaluation factors in the review process that are biased against these groups. 41 See related Recommendation C in Chapter 2.
OCR for page 11
An Assessment of the SBIR Program optimize the nation’s return on the substantial annual investment in the SBIR program. Additional Resources Could be Used Effectively. In summary, the program is proving effective in meeting congressional objectives. It is increasing innovation, encouraging participation by small companies in federal R&D, providing support for small firms owned by minorities and women, and resolving research questions for mission agencies in a cost-effective manner. Should the Congress wish to provide additional funds for the program in support of these objectives, those funds could be employed effectively by the nation’s SBIR program.