4
NIH SBIR Program—Outcomes

4.1
INTRODUCTION

The Congress has tasked the National Academies to assess whether and to what extent the SBIR program at NIH has met the congressionally mandated objectives for the program, and to suggest possible areas for improvement in program operations. Although Congress has over the years identified a number of objectives for the program, these mandated objectives are usually summarized as follows:

  • Supporting the commercialization of federally funded research.

  • Supporting the agency’s mission.1

  • Supporting small business and in particular woman- and minority-owned businesses.

  • Expanding the knowledge base.

These four areas define the structure and content of this chapter. A subsequent chapter reviews program management in more detail, and provides a basis for possible improvements to the program.

Such an assessment raises difficult methodological challenges, which are discussed and to the maximum extent possible resolved in the NRC’s Methodology

1

The mission of NIH is “… science in pursuit of fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to extend healthy life and reduce the burdens of illness and disability.” Accessed at: <http://www.nih.gov/about/>.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 79
4 NIH SBIR Program—Outcomes 4.1 INTRODUCTION The Congress has tasked the National Academies to assess whether and to what extent the SBIR program at NIH has met the congressionally mandated objectives for the program, and to suggest possible areas for improvement in program operations. Although Congress has over the years identified a number of objectives for the program, these mandated objectives are usually summarized as follows: • Supporting the commercialization of federally funded research. • Supporting the agency’s mission.1 • Supporting small business and in particular woman- and minority-owned businesses. • Expanding the knowledge base. These four areas define the structure and content of this chapter. A subsequent chapter reviews program management in more detail, and provides a basis for possible improvements to the program. Such an assessment raises difficult methodological challenges, which are dis- cussed and to the maximum extent possible resolved in the NRC’s Methodology 1The mission of NIH is “. . . science in pursuit of fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to extend healthy life and reduce the burdens of illness and disability.” Accessed at: . 7

OCR for page 79
0 SBIR AT THE NATIONAL INSTITUTES OF HEALTH Report.2 One issue however should be briefly discussed here too—the question of comparators. Assessment is usually done by comparison—comparing programs and ac- tivities, in this case. Three kinds of comparison seem possible: with other NIH programs, with SBIR programs at other agencies, and with early stage technology development funding in the private sector, such as venture capital activities. Yet none of these comparisons is valid. Other award programs at NIH have fundamentally different objectives, such as promoting basic research (e.g., RO1 awards), developing medical capacity (awards for medical centers), or training. No other NIH award programs have as a primary goal the commercial exploitation of research. This fundamental differ- ence in objectives must be taken into account in evaluating the SBIR program at NIH. SBIR programs at other agencies are organized very differently and—at DoD and NASA at least—have quite different objectives. NIH SBIR might be compared with venture capital activities, but these are typically focused closer to market, and include much larger investment (an average investment round of $7 million in 2005 as against less than $1 million for SBIR). VC investments are also focused on companies, not projects, further invalidating comparisons.3 Finally, while the question of commercialization is the most readily subject to measurement—through accessible data on sales and licensing revenues and other metrics—Congress has not prioritized among the four mandated objectives and each is equally important to NIH. 4.2 COMMERCIALIZATION How well has the NIH SBIR program fostered commercialization of funded research? The following sections examine a variety of relevant indicators. 4.2.1 Proposed Commercialization Indicators and Benchmarks Three sets of indicators are used to evaluate the extent to which SBIR grant- ees have commercialized their funded research: 1. Sales and licensing revenues (“sales” hereafter unless otherwise noted). Revenues flowing into the company from the commercial marketplace 2 National Research Council, An Assessment of the Small Business Innoation Research Program: Project Methodology, Washington, DC: The National Academies Press, 2004. 3 See National Venture Capital Association, Money Tree Report, November, 2006. The mean venture capital deal size for the first three quarters of 2006 was $8.03 million. This trend has been acceler- ated by the growth of larger venture firms. See P. Gompers and J. Lerner, The Venture Capital Cycle, Cambridge: The MIT Press, 1999, Ch. 1.

OCR for page 79
 NIH SBIR PROGRAM—OUTCOMES constitute an important measure of commercial success, as sales are an indicator of realized market demand for the output from a project. There is however no single agreed benchmark against which to measure whether agencies have met the legislative objectives for commercializa- tion. It seems, therefore, reasonable to assess commercialization against a range of benchmarks: (a) Reaching the market—any sales. (b) Reaching $1 million in cumulative sales—which has been approxi- mately the median size of a Phase I plus a Phase II award at NIH. (c) Reaching $5 million in cumulative sales—which could be viewed as a modest commercial success. (d) Reaching $50 million in cumulative sales—which could be viewed as full commercial success. 2. R&D investments and research contracts. Beyond sales, further R&D investments and contracts are also good evidence that results from the project are moving toward commercialization. These investments and contracts may include partnerships, further grants and awards, or govern- ment contracts. The benchmarks for success at each of these levels should be the same as those above, namely: (a) Any R&D additional funding. (b) Funding of $1 million or more. (c) Funding of $5 million or more. (d) Funding of $50 million or more. 3. Sale of equity constitutes a less clear-cut indicator of commercial activ- ity. A company which is sold because its acquirer is seeking a successful product has generated returns. Key metrics include: (a) Equity investment in the company by independent third party. (b) Sale or merger of the entire company. Using these metrics, to what extent have NIH SBIR companies commercialized? 4.2.2 Sales and Licensing Revenues from NIH SBIR Awards Data from three sources indicate that 30-40 percent of NIH projects funded between 1992 and 2002 have reached the marketplace. (These three data sources all refer here only to NIH projects. Note however that subsequent NIH resurveys suggest that this may substantially understate the eventual commercialization rate.) The projects underlying the percentages in Figure 4-1 have generated posi-

OCR for page 79
 SBIR AT THE NATIONAL INSTITUTES OF HEALTH BOX 4-1 A Note on Data Sources Research on the NIH SBIR program has benefited from the existence for three independent data sources on outcomes from the program.a These are: • he NRC Phase II Survey (2005), which sent at least one questionnaire to ev- T ery Phase II recipient at NIH, 1992-2005. Firms with multiple awards received more questionnaires, but normally not for each award. • he NIH’s “National Survey to Evaluate the NIH SBIR Program: Final T Report” (hereafter the NIH Survey) (2003), which sent one questionnaire to each firm with a Phase II award 1992-2002. This survey has subsequently been updated. • he DoD Commercialization Reports (CCRs), through which firms apply- T ing for future awards at DoD must report on commercialization outcomes for awards at all agencies, including NIH. Data on about 12 percent of NIH awards can be found in the DoD database. aFor details on the NRC Phase II Survey, see Appendix B. For details on the NIH Survey, see National Institutes of Health, National Survey to Evaluate the NIH SBIR Program: Final Report, July 2003. Available online at: . The DoD Index is not publicly available. tive revenue from sales or licensing. Follow-on surveys at NIH indicate that this figure could eventually grow to about 60 percent of projects. However, determin- ing that projects have generated some revenues is insufficient, in three respects: First, the distribution of sales by size of revenue is important: Projects generating $50 million in sales have substantially greater commercial returns than those gen- erating $100,000. Second, data on sales to date are insufficient: Accurate analysis requires the adjustment of this raw data set to take account projections of future sales. Third, it is useful to distinguish between sales and licensing revenues. 4.2.2.1 Sales Ranges Figure 4-2 shows the number of grantees achieving each of the specified sales benchmarks. There are general similarities between the three data sources. The majority of sales (at least 68 percent for all three sources) are concentrated in the $0-$1 million range. None of the sources indicate that more than 10 percent of projects generated $5 million in cumulative revenues. Each data source recorded one (different) project with more than $50 million in revenues. The DoD database indicates lower commercialization results than the two surveys. Entries in the DoD database constitute a formal part of the SBIR ap- plication process, capturing updated data at that time about commercialization from all previous SBIR Phase II awards, and companies may therefore be more

OCR for page 79
 NIH SBIR PROGRAM—OUTCOMES 50 45 40.7 40 Percent of Sales > Zero 35 31.6 30.3 30 25 20 15 10 5 0 NRC NIH DoD Data Source FIGURE 4-1 Percentage of NIH SBIR projects reaching the market from 1992-2002. SOURCE: NRC Phase II Survey, DoD Commercialization database, and National Insti- tutes of Health, National Surey to Ealuate the NIH SBIR Program: Final Report, July 2003. 4-1 100 90 Percent of Reporting Projects 80 70 60 50 40 30 20 10 0 >0 and <1M 1M to <5M 5M to <50M 50M + Sales (Dollars) NIH DoD NRC FIGURE 4-2 Sales by sales range, total for 1992-2002. SOURCE: National Institutes of Health, NRC Phase II Survey, and DoD Commercializa- tion database. 4-2

OCR for page 79
 SBIR AT THE NATIONAL INSTITUTES OF HEALTH BOX 4-2 Multiple Sources of Bias in Survey Response Large innovation surveys involve multiple sources of bias that can skew the results in both directions. Some common survey biases are noted below. These biases were tested for and responded to in the NRC surveys.a • uccessful and more recently funded firms are more likely to respond. S Research by Link and Scott demonstrates that the probability of obtaining research project information by survey decreases for less recently funded proj- ects and increases with the award amount.b Nearly 40 percent of respondents in the NRC Phase II Survey began Phase I efforts after 1998, partly because the number of Phase I awards increased, starting in the mid 1990s, and partly because winners from more distant years are harder to reach. They are harder to reach as time goes on because small businesses regularly cease opera- tions, are acquired, merge, or lose staff with knowledge of SBIR awards. • uccess is self-reported. Self-reporting can be a source of bias, although the S dimensions and direction of that bias are not necessarily clear. In any case, policy analysis has a long history of relying on self-reported performance measures to represent market-based performance measures. Participants in such retrospective analyses are believed to be able to consider a broader set of allocation options, thus making the evaluation more realistic than data based on third party observation.c In short, company founders and/or principal inves- tigators are in many cases simply the best source of information available. • urvey sampled projects at firms with multiple awards. Projects from firms S with multiple awards were underrepresented in the sample, because they could not be expected to complete a questionnaire for each of dozens or even hundreds of awards. • ailed firms are difficult to contact. Survey experts point to an “asymmetry” F in their ability to include failed firms for follow-up surveys in cases where the firms no longer exist.d It is worth noting that one cannot necessarily infer that the SBIR project failed; what is known is only that the firm no longer exists. • ot all successful projects are captured. For similar reasons, the NRC N Phase II Survey could not include ongoing results from successful projects in firms that merged or were acquired before and/or after commercialization of the project’s technology. The survey also did not capture projects of firms that did not respond to the NRC invitation to participate in the assessment. • ome firms may not want to fully acknowledge SBIR contribution to S project success. Some firms may be unwilling to acknowledge that they re- ceived important benefits from participating in public programs for a variety of reasons. For example, some may understandably attribute success exclusively to their own efforts. • ommercialization lag. While the NRC Phase II Survey broke new ground in C data collection, the amount of sales made—and indeed the number of projects that generate sales—are inevitably undercounted in a snapshot survey taken at a single point in time. Based on successive data sets collected from NIH SBIR award recipients, it is estimated that total sales from all responding proj- ects will likely be on the order of 50 percent greater than can be captured in a

OCR for page 79
 NIH SBIR PROGRAM—OUTCOMES single survey.e This underscores the importance of follow-on research based on the now-established survey methodology. 14 12 Sales (Millions of Dollars) 10 8 6 4 2 0 1 2 3 4 5 6 7 8 9 10 11 12 Survey taken Years after Phase II Award FIGURE B-4-1 Survey bias due to commercialization lag. These sources of bias provide a context for understanding the response rates to the NRC Phase I and Phase II Surveys conducted for this study. For the NRC Phase II Survey for NIH, of the 1,127 firms that could be contacted out of a sample size of 1,680, 496 responded, representing a 44 percent response rate. The NRC Phase I Survey captured 10 percent of the 7,049 awards made by NIH Figure B-4-1 between 1992 and 2001. See Appendixes B and C for additional information on the surveys. aFor a technical explanation of the sample approaches and issues related to the NRC surveys, see Appendix B. bAlbert N. Link and John T. Scott, Evaluating Public Research Institutions: The U.S. Ad- vanced Technology Program’s Intramural Research Initiative, London: Routledge, 2005. cWhile economic theory is formulated on what is called ‘revealed preferences,’ meaning individuals and firms reveal how they value scarce resources by how they allocate those resources within a market framework, quite often expressed preferences are a better source of information especially from an evaluation perspective. Strict adherence to a revealed pref- erence paradigm could lead to misguided policy conclusions because the paradigm assumes that all policy choices are known and understood at the time that an individual or firm reveals its preferences and that all relevant markets for such preferences are operational. See {1} Gregory G. Dess and Donald W. Beard, “Dimensions of Organizational Task Environments.” Administrative Science Quarterly, 1984, 29: 52-73. {2} Albert N. Link and John T. Scott, Public Accountability: Evaluating Technology-Based Institutions, Norwell, Mass.: Kluwer Academic Publishers, 1998. dAlbert N. Link and John T. Scott, Evaluating Public Research Institutions: The U.S. Ad- vanced Technology Program’s Intramural Research Initiative, op. cit. eData from NIH indicates that a subsequent survey taken two years later would reveal very substantial increases in both the percentage of firms reaching the market, and in the amount of sales per project.

OCR for page 79
 SBIR AT THE NATIONAL INSTITUTES OF HEALTH 90 77.9 80 63.6 70 60 Percent 50 40 30 20 10 0 DoD NRC Data Source FIGURE 4-3 Degree of all sales concentrated in companies reporting $5 million+ in sales. SOURCES: NRC Phase II Survey, DoD Commercialization database. 4-3 likely to ensure that their responses are conservative. The DoD responses are also from fewer companies, as they include a number of companies with numerous responses: Only 108 companies accounted for all the DoD responses, compared with 495 companies for the NIH Survey.4 Sales are highly concentrated. Figure 4-3 shows that the few projects gen- erating at least $5 million per year in revenues account for most of the revenues reported for all projects, ranging from slightly over 60 percent for DoD respon- dents to more than 75 percent for NRC respondents. This degree of sales concentration confirms the view that from the perspec- tive of sales, the SBIR program at NIH generates a considerable number of projects that reach the market, no more than 10 percent of which generate sales greater than $5 million in total from the surveyed projects. Two of these larger winners, Optiva5 and Martek, are discussed in Box 4-3 and Box 4-4. 4 National Institutes of Health, National Surey to Ealuate the NIH SBIR Program: Final Report, July 2003 [NIH Survey]. The NIH Survey addressed one questionnaire to every firm winning a Phase II award during the selected period; the DoD data derives from firms applying at DoD who had also won previous NIH Phase II awards, and were thus required to answer commercialization questions about those awards. 5 Interview with David Guiliani, Optiva founder, July 2006. See also Puget Sound Business Journal, “Philips to Acquire Optiva Corp.” August 22, 2000.

OCR for page 79
7 NIH SBIR PROGRAM—OUTCOMES BOX 4-3 Optiva Corporation Medicine and dentifrice dispensing sonic brush, sonic toothbrush Optiva, formed as Tech in 1987 by an entrepreneur and two University of Washington professors, controls more than 26 percent of the U.S. power-tooth- brush market, generating a $300 million business and 500 jobs mostly in Sno- qualmie, Washington. By 1997, Optiva was named the fastest-growing company in the U.S. by Inc. magazine, and its CEO was selected as the Small Business Person of the Year. In August 2000, Philips agreed to acquire Optiva for an undisclosed price (reputed to be approximately $1 billion). At the time, Optiva had more than 600 employees and more than $175 million in annual sales from the Sonicare line. By 2001, Optiva had sold its 10 millionth power toothbrush, and had become the #1 producer of power toothbrushes in the U.S. market. BOX 4-4 Martek Biosciences Corporation Products from microalgae Martek Biosciences Corporation develops and commercializes products from microalgae. Martek’s products include fatty acids (omega-3 docosahexaenoic acid and omega-6 arachidonic acid) which are used as ingredients in infant formula and animal feeds. Martek’s DHA-rich oil can also be used in nutritional supple- ments and functional foods for older children and adults. Martek also produces fluorescent algal pigments used for diagnostic and pharmaceutical research purposes. Martek has become an important player in three markets: • nfant formula. Martek has developed and patented two fermentable strains I of microalgae which produce oils rich in docosahexaenoic acid, DHA. In like manner, another patented process was developed for a fungus that produces an oil rich in arachidonic acid, ARA. Both DHA and ARA are found in breast milk and are important nutrients in infant development. Thus the two oils are used in infant formulas. • utritional supplements. The DHA-rich oil can also be used in supplements N and functional foods for older children and adults. • ife sciences and research. Martek also makes and sells a series of propri- L etary and nonproprietary fluorescent markers. These products have applica- tions in drug discovery (high-throughput screening), DNA microarray detection and flow cytometry. Martek developed the technology underlying these products directly as a result of SBIR funding, according to Henry Linsert, founder and CEO. The result has been explosive growth for the company, rising from about $5 million in 2000 to more than $185 million in 2004.

OCR for page 79
 SBIR AT THE NATIONAL INSTITUTES OF HEALTH 4.2.2.2 Sales Expectations Nineteen percent of NRC Phase II Survey respondents did not yet report sales but expected sales in the future (see Figure 4-4). Table 4-1 shows that these expectations are strongly concentrated in the immediate out years. These expectations may, however, be overly optimistic. Table 4-2 shows the elapsed time between the end of the Phase II award and the date of first sales. In some cases, possibly where the award is for improvements to existing technolo- gies, first sales may occur before the Phase II award is even completed. The data set in Table 4-2 shows that the median elapsed time to sale is less than two years—more than half of all projects reporting sales claim a date of first sale within two years of the start of the Phase II award. This number can be negative in cases where companies were using SBIR to improve products already in the market. Further, NRC Phase II Survey responses indicate that more than 85 percent of first sales occurred before the end of the 4th year after the date of the award. About 19 percent of all NRC Phase II Survey respondents claimed that they anticipated sales in the future. However, if the survey data accurately predicts the distribution of first sales across elapsed time since award, these respondents appear to be overly optimistic. The likelihood of a project generating initial sales diminishes with time elapsed since the award. Table 4-3 focuses on the projects from the NRC Phase II Survey that still anticipate sales. It identifies the award year, and assigns a per- centage likelihood of first sales, based on the distribution in Table 4-2. The NRC Phase II Survey data indicate that a vast majority (86.2 percent) of first sales are made within 4 years after the date of award. Consequently, projects that have not Sales expected (19%) No response (29%) Sales not expected (6%) Sales (46%) FIGURE 4-4 Sales expectations. SOURCE: NRC Phase II Survey.

OCR for page 79
 NIH SBIR PROGRAM—OUTCOMES TABLE 4-1 Year of Expected Sales Year of Expected Sales Number of Projects 2005 22 2006 20 2007 20 2008 11 2009 1 2010 6 2011 2 2012 1 SOURCE: NRC Phase II Survey. generated a first sale within four years have a 13.8 percent likelihood that they will do so—historically, 86.2 percent of projects will have reported sales by then if they are going to have sales at all. These percentages can be used to adjust the claims of respondents, in Ta- ble 4-3. They indicate that while 95 projects report that they still expect sales, we estimate that in the end only five will actually reach the market. It is important to note that this analysis refers only to first sales. The bulk of sales in almost all cases occur at different and unknown periods after the first sale. This is an important point: The sales data from the survey are effectively a snapshot of sales taken at a specific point in the lifetime of a product. Most prod- uct revenue returns are bell-shaped—ramping up from initial sales to a maximum TABLE 4-2 Years Elapsed Between Start of Phase II Award and Year of First Sale Elapsed Years Number of Projects Percentage of Responding Projects –11 1 0.4 –7 1 0.4 –4 1 0.4 –3 4 1.8 –2 6 2.7 –1 9 4.0 0 18 8.0 1 29 12.9 2 50 22.3 3 48 21.4 4 26 11.6 5 17 7.6 6 8 3.6 7 3 1.3 8 1 0.4 9 2 0.9 SOURCE: NRC Phase II Survey.

OCR for page 79
 NIH SBIR PROGRAM—OUTCOMES BOX 4-12 Three Rivers Holdings, Inc. The SmartWheel: Development of Wheelchair Pushrim Force and Measurement Device SmartWheel is a product designed to measure accurately all the key param- eters involved in the propulsion of wheelchairs by their occupants. These include including stroke frequency, propulsion angle, acceleration, forces applied to the handrim, velocity, and distance traveled. According to the company, “The Smart- Wheel is the only commercial product in the world that measures propulsion biomechanics in the natural environment of the wheelchair user.” SmartWheel has by now been in use as a research tool for more than ten years as a means of measuring and analyzing pain and injury among wheelchair users and also as a means of assessing interventions to address problems. Currently, SmartWheels are in use at leading research institutions including the Rehabilitation Institute of Chicago, the University of Michigan, the Rehabilitation Institute of Montreal, the University of Washington, the Kessler Medical Rehabili- tation Research and Education Corporation, the University of Pittsburgh, and the University of Alberta. SmartWheels is now being adapted for use as a clinical product. It has four main uses: • ustification of equipment decisions for insurance reimbursement, using pre- J cise data to identify users who cannot provide the force need to propel a manual chair effectively • election of the appropriate manual wheelchair, once again by the application S of precise data to the selection process • raining that allows wheelchair users to improve propulsion efficiency by re- T ducing the stress on their arms through use of a longer stroke, reducing stroke frequency, and minimizing wasted forces (e.g., pushing directly down on the handrim). • reation of an individualized patient database, showing the effect of adjust- C ments and creating a longitudinal record for selected metrics The company notes that SBIR awards were used to facilitate its transformation from a hard-wired noncommercial research tool to a wireless, user-friendly com- mercial clinical and research tool. Leading experts were hired as consultants, and speed to market was accelerated. All the evidence suggests that use of SmartWheel will continue to expand clinically, and that increasing numbers of wheelchair users will benefit from the technology. Yet commercially, this will never be a major success: The company expects that if sales double in 2005 and continue to grow thereafter, revenues will still only be $1 million in 2006. Still, the social benefits for the specific niche of SmartWheel users greatly exceed any commercial return.

OCR for page 79
0 SBIR AT THE NATIONAL INSTITUTES OF HEALTH 600,000,000 Total SBIR Funding at NIH (Dollars) 500,000,000 400,000,000 300,000,000 200,000,000 100,000,000 0 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 Year FIGURE 4-15 Total SBIR funding for small business at NIH, 1983-2004. SOURCE: NIH awards database. has been driven by the doubling of the overall NIH extramural research budget over that period, with a proportion of that funding allocated for small business. However, this data set does not answer a related question: To what extent has the SBIR program replaced other funding for small businesses at NIH. This ques- 4-15 tion can be addressed by comparing the level of SBIR funding with that available through all other small business funding mechanisms at NIH (see Figure 4-16). The awards data show quite clearly that the share of small business funding being disbursed through the SBIR program has fallen steadily since soon after 95 Research Funding at NIH (Percent) 90 SBIR Share of Small Business 85 80 75 70 65 60 55 50 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 Year FIGURE 4-16 SBIR share of small business research funding at NIH, 1983-2004. SOURCE: National Institutes of Health. 4-16

OCR for page 79
 NIH SBIR PROGRAM—OUTCOMES the inception of the program at NIH in 1983. After peaking at 90 percent of all small business research funding in the mid-1980’s, the SBIR program’s share fell steadily to about 72 percent in 2003, before falling further in 2004 (which may be an outlier). These data clearly invalidate the hypothesis that SBIR has replaced other forms of small business funding at NIH. 4.4.2 The Decision to Begin the Project Figure 4-17 shows that almost half of NRC Phase II Survey respondents were sure that their projects would not have occurred at all without SBIR fund- ing. Altogether, almost 75 percent thought that would have been the case. NIH Survey data are comparable, with 64 percent of respondents anticipating that the projects would have been a “no go” in the absence of SBIR funding. These Definitely yes (5%) Probably yes (8%) Definitely not NRC Phase II Survey Uncertain (45%) (14%) Probably not (28%) Project go without award Not Applicable (15%) (21%) NIH Survey Project no go without award (64%) FIGURE 4-17 Greenlighting the project. SOURCE: NRC Phase II Survey and National Institutes of Health, National Surey to Ealuate the NIH SBIR Program: Final Report, July 2003. 4-17

OCR for page 79
 SBIR AT THE NATIONAL INSTITUTES OF HEALTH figures suggest that SBIR often makes the difference between a research project being pursued or not. Even for projects that would have continued in the absence of SBIR fund- ing, delays and other changes would have been caused by the resulting paucity of funds. 51 percent of these respondents noted that the scope of their projects would have been narrower; 19 of the 43 firms that would have continued any- way expected their project would have been delayed. Fifteen firms expected this delay would have been at least 12 months, and 13 expected a delay of at least 24 months, generating an average expected delay in project start of 8 months. Sixty- three percent expected that project completion would also have been delayed. 4.4.3 Company Foundation Responses to the NRC Firm Survey indicate that almost 25 percent of NIH firms that received SBIR Phase II awards were founded entirely or in part as a result of SBIR awards (see Table 4-20). 4.4.4 Company Foundation and Academia Case study interviews suggest that SBIR has facilitated of the movement of technologies and researchers from university labs to the commercial environ- ment. Data from the NRC Firm Survey strongly support this hypothesis. More than 80 percent of NIH respondent companies had at least one founder from academia (see Table 4-21). The same survey found that about a third of founders were most recently employed in an academic environment before founding the new company. This data set, thus, strongly suggests that SBIR has indeed encour- aged academic scientists to work in a more commercial environment. 4.4.5 Growth Effects While there are no data about the effect of SBIR awards on company growth, except for the employment data discussed above (which do not seek to explain TABLE 4-20 SBIR Awards and Firm Foundation: Was the Firm Founded Wholly or Partly Because of the Referenced SBIR Award? Number of Responses Percent of Responses No 342 74.8 Yes 49 10.7 Yes, in part 66 14.4 457 100.0 SOURCE: NRC Firm Survey.

OCR for page 79
 NIH SBIR PROGRAM—OUTCOMES TABLE 4-21 Academics as Founders Number of Responses Percent None 86 18.9 At least one 369 81.1 All 455 100.0 SOURCE: NRC Firm Survey. the cause of growth,) the NRC Firm Survey did ask respondents to provide their own estimates of SBIR impacts on growth (see Table 4-22). Almost half of respondents indicated that more than half of the growth ex- perienced by their firm was directly attributable to SBIR. This too is evidence of the powerful impact winning an NIH SBIR award can have on the development of a small business. 4.4.6 Support for Woman- and Minority-owned Businesses One of the congressional mandates for the SBIR program is to support the work of women and minorities in science. The primary metric for this sup- port is the extent to which SBIR programs fund woman- and minority-owned businesses. There is an extensive analysis of awards to woman- and minority-owned firms in Chapter 3 of this report. A review of the available data in Chapter 3 draws the following conclusions: • Together, woman- and minority-owned firms account for an average of about 15 percent of Phase I awards at NIH (2003-2006). • The trend for minority-owned firms is downward since 1993, with some annual variation, and minority-owned firms have accounted for less than 4 percent of Phase I awards since 2003. TABLE 4-22 SBIR Impacts on Company Growth (percentage impact of SBIR on overall company growth) Number of Responses Percent Less than 25 132 29.5 25 to 50 100 22.4 51 to 75 78 17.4 More than 75 137 30.6 Total 447 100.0 SOURCE: NRC Firm Survey.

OCR for page 79
 SBIR AT THE NATIONAL INSTITUTES OF HEALTH 14 12 Percent of all Awards 10 Woman-owned firms 8 Minority-owned firms Linear (Woman-owned 6 firms) 4 Linear (Minority-owned firms) 2 0 92 93 94 95 96 97 98 99 00 01 02 03 04 05 06 19 19 19 19 19 19 19 19 20 20 20 20 20 20 20 Year FIGURE 4-18 Phase I Award share of woman- and minority-owned firms, 1992-2006. SOURCE: National Institutes of Health. • The share of Phase I applications from woman- and minority-owned firms has declined since early 1992, although absolute numbers have risen. • This is true in particularNew 4-18 of minority-owned firms, whose share of applica- tions has declined from about 10 percent in 1996 to just over 5 percent in 2005. • Lower levels of awards are partly explained by lower success rates—the rate at which applications are selected to become awards. The data show that woman- and minority-owned firms are consistently less successful in the Phase I selection process—that lower percentages of their applications generate awards. Minority applicants saw a particularly steep decline in success rates from 1999 to 2004, with some recovery in 2005-2006. 30 Awards as Percent of Applications 25 20 Woman-owned firms 15 Minority-owned firms Other small businesses 10 5 0 92 93 94 95 96 97 98 99 00 01 02 03 04 05 06 19 19 19 19 19 19 19 19 20 20 20 20 20 20 20 Year FIGURE 4-19 Success rates for Phase I awards by demographic, 1992-2006. SOURCE: National Institutes of Health. 4-19

OCR for page 79
 NIH SBIR PROGRAM—OUTCOMES However, success rates for minority-owned firms remain about five per- centage points lower than the rates for firms that are neither woman- or minority-owned. The data themselves provide no answer to the question of why woman- and minority-owned firms have lower success rates. One promising hypothesis is that these firms tend to be formed more recently, and have both a shorter track record and less experience principal investigators, both of which may militate against success in the NIH selection process. Finally, it is important to note that while woman-owned firms have main- tained and even slightly increased their share of SBIR Phase I awards at NIH, they remain at an average of about 10 percent of all awards (2003-2006). At the same time, the percentage of women among recent life sciences doctorates has increased dramatically. According to NSF, in 1999 and 2000 women accounted for more than 61 percent of all life sciences doctorates awarded.16 In that con- text, maintaining a ten percent share of awards is much less impressive, and NIH might well wish to undertake further analysis to determine why so few of these new doctorates appear to be applying for NIH SBIR funding (note that there is no requirement that a company exist in order to apply for an award, although a company must be formed in order to accept one.) 4.5 SBIR AND THE EXPANSION OF KNOWLEDGE Metrics for assessing knowledge outputs from research programs are well- known, but far from comprehensive. Patents, peer-reviewed publications, and, to a lesser extent, copyrights and trademarks, are all widely used metrics. They are each discussed in detail below. However, it is also important to understand that these metrics do not capture the entire transfer of knowledge involved in programs such as SBIR. 4.5.1 Patents The NRC Phase II Survey data indicate that about 34 percent of respondents received patents related to their SBIR-funded project (see Table 4-23). About 41 percent of projects generated at least one patent application, and about 82 percent of those applications were successful. The NIH Survey generated similar data indicating that 37 percent of re- spondents received a patent related to their SBIR award (although wording of the question makes it impossible to know whether the patent was awarded for work completed before or after the award). It is possible that a positive response 16 Derived from National Science Foundation, Division of Science Resources Statistics, Women, Minorities, and Persons with Disabilities in Science and Engineering: 00, NSF 04-317, Arlington, VA: National Science Foundation, 2004.

OCR for page 79
 SBIR AT THE NATIONAL INSTITUTES OF HEALTH TABLE 4-23 Projects Reporting Patent Applications and Patent Awards Applications Awarded Number Percent Number Percent No 249 58.7 280 66.0 Yes 175 41.3 144 34.0 424 100.0 424 100.0 Total 679 305 SOURCE: NRC Phase II Survey. reflects a patent application rather than patent approval. The very small number of “pending items” reported suggests that this may sometimes have been the case. A negative correlation found between projects with patents and those with marketing activities could indicate differences between projects targeted at prod- ucts and those focused on knowledge. However, marketing activities are posi- tively strongly correlated with knowledge outputs, indicating that this kind of substitution effect is not detectable. Once again, relationships between survey results and other variables might provide extremely useful insights. For example, Figure 4-22 shows patenting outputs by size of firm. Analysis of the scientific importance of the patents listed was not possible because the patents themselves were not disclosed in the course of the survey. 11-20 >21 6-10 3-5 1 2 FIGURE 4-20 Number of patents per company reporting patenting activity. SOURCE: National Institutes of Health, National Surey to Ealuate the NIH SBIR Pro- gram: Final Report, July 2003. 4-20

OCR for page 79
7 NIH SBIR PROGRAM—OUTCOMES 350 No Marketing 300 Number of Awardees 250 Ongoing or Completed Marketing 200 150 100 50 0 None One or More Patents, Copyrights, Trademarks, or Pending Similar Items FIGURE 4-21 Awardees with one or more patents, copyrights, or trademarks—by mar- keting status. SOURCE: NRC Phase II Survey. 4.5.2 Scientific Publications 4-21 The NIH Survey did not distinguish between scientific publications and ar- ticles in the trade and popular find email with replacement text. did not press. However, the NRC Phase II Survey did so, and it determined that slightlyretyped than half (53.5 percent) of the respondents Text is more here. w. masom 500 + 301-500 201-300 101-200 Number of Employees 51-75 41-50 31-40 26-30 21-25 16-20 11-15 6-10 0-5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Number of Patents FIGURE 4-22 Number of patents, projects with at least one reported patent—by size of company. 4-22 SOURCE: NRC Phase II Survey.

OCR for page 79
 SBIR AT THE NATIONAL INSTITUTES OF HEALTH had published at least one scientific paper related to their SBIR grant. About 33 percent of those with publications had published only a single paper, but one company had published 165 papers on the basis of its SBIR project, and several others had published at least 50 (as shown in Table 4-24). This data set fits well with case studies and interviews, which suggest that SBIR companies are proud of the quality of their research. Publications are featured prominently on many grantee Web sites, and companies like Advanced Brain Monitoring, SAM Technologies, and Polymer Research all made a point of stating during interviews that their work was of the highest technical quality, as measured in the peer-reviewed publications. Publications therefore fill two important roles in the study of SBIR programs. First, they provide an indication of the quality of the research being con- ducted with program funds. More than half of the funded projects appear to be of sufficient value to generate at least one publication. Second, publications are themselves the primary mechanism through which knowledge is transmitted within the scientific community. The existence of ar- ticles based on SBIR projects is therefore direct evidence that the results of these projects are being disseminated widely. This, in turn, implies the NIH SBIR is meeting its congressional mandate to support scientific outcomes. It is useful to note that the non-SBIR portion of the NIH research program does not have any mechanism in place for determining whether similar knowledge effects are being generated at the same rate as in the SBIR program. Note also that comparisons with SBIR programs at other agencies may be less than completely valid, as the publishing culture may be different outside the biomedical scientific world. 4.5.3 SBIR and Universities SBIR can have further effects on the spread of knowledge through the involvement of university staff and students in SBIR projects. For example, TABLE 4-24 Publications Number of Publications Number of Responses Total Publications 1 72 72 2 52 104 3 32 96 4 19 76 5 15 75 6-10 15 133 11-30 9 146 30+ 7 420 Totals 236 1,122 SOURCE: NRC Phase II Survey.

OCR for page 79
 NIH SBIR PROGRAM—OUTCOMES TABLE 4-25 University Involvement in SBIR Projects 4% The Principal Investigator (PI) for this Phase II project was a faculty member. 7% The Principal Investigator (PI) for this Phase II project was an adjunct faculty member. 34% Faculty or adjunct faculty member (s) work on this Phase II project in a role other than PI, e.g., consultant. 15% Graduate students worked on this Phase II project. 16% University/College facilities and/or equipment were used on this Phase II project. 5% The technology for this project was licensed from a University or College. 6% The technology for this project was originally developed at a University or College by one of the participants in this Phase II project. 24% A University or College was a subcontractor on this Phase II project. SOURCE: NRC Phase II Survey. Advanced Targeting Systems, in San Diego, has forged an extended and very successful research partnership with a senior scientist at the University of Utah. Other companies have made similar arrangements. Just over half (54 percent) of all respondents indicated that there had been involvement by university faculty, graduate students, and/or a university itself in developed technologies. This involvement took a number of forms, as shown by Table 4-25. The wide range of roles played by university staff and students indicates once more the multiple ways in which SBIR projects feed the knowledge base of the nation. Involvement in these projects provides opportunities for university staff different than those available within the academy.