2
Study Methodology

Reflecting the two core questions (How many firms have actually been excluded by the ruling from participation in the NIH SBIR program? And what is the likely effect of this exclusion on these firms and on the NIH SBIR program) the methodology for this study is divided into two areas.

2.1
IDENTIFYING VENTURE-FUNDED FIRMS AND ESTIMATING THE “EXCLUSION EFFECT”

Identifying Venture-funded Firms

Identifying firms that have received venture funding is a challenge. SBIR-funded firms, which are in most cases privately held, are not required to reveal whether they have received third-party investment. As a result this information is not collected and stored by SBIR-funding agencies or SBA.

In order to establish the distribution of venture funding prior to the SBA ruling,1 this research initially focused on firms winning Phase II awards from 1992-2002 inclusive.

Phase II awards were selected for study because they account for the largest amount of NIH SBIR funding. As Phase II awards continue to grow in size, and are extended in length, the proportion of funding allocated for post-Phase I continues to grow. What is more, commercial success almost always comes after

1

The years prior to the ruling are years when a “natural” level of participation for venture-funded companies might be established.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 22
2 Study Methodology Reflecting the two core questions (How many firms have actually been ex - cluded by the ruling from participation in the NIH SBIR program? And what is the likely effect of this exclusion on these firms and on the NIH SBIR program) the methodology for this study is divided into two areas. 2.1 IDENTIFYING VENTURE-FUNDED FIRMS AND ESTIMATING THE “EXCLUSION EFFECT” Identifying Venture-funded Firms Identifying firms that have received venture funding is a challenge. SBIR- funded firms, which are in most cases privately held, are not required to reveal whether they have received third-party investment. As a result this information is not collected and stored by SBIR-funding agencies or SBA. In order to establish the distribution of venture funding prior to the SBA ruling,1 this research initially focused on firms winning Phase II awards from 1992-2002 inclusive. Phase II awards were selected for study because they account for the largest amount of NIH SBIR funding. As Phase II awards continue to grow in size, and are extended in length, the proportion of funding allocated for post-Phase I continues to grow. What is more, commercial success almost always comes after 1The years prior to the ruling are years when a “natural” level of participation for venture-funded companies might be established. 22

OCR for page 22
23 STUDY METHODOLOGY Phase II rather than just Phase I—so a focus on the latter includes almost all commercial successes. The focus on 1992-2002 coincides with the rapid development and matu- ration of the biotechnology industry. It also reflects the reauthorization of the program in 1992, which led to prioritization of the legislative goals of SBIR with an emphasis on commercialization. Finally, and not least, the focus on data from 1992 to 2002 is driven by the availability of data. The firms winning Phase II awards also constitute the universe of firms ad - dressed by the NRC’s Phase II Survey,2 and the NIH Phase II Survey,3 which provide the best available data on outcomes from the SBIR program. These data will be critical to the second phase of the analysis, identifying impacts. The standard database on venture funding—also used by GAO in its 2006 study4—is the Thomson VentureSource database, and this is the primary source of venture funding data utilized in this National Research Council study. 5 The NRC compiled a list of firms that received at least one Phase II award at NIH between 1992 and 2002 inclusive and Thomson VentureSource ran that list against its own database of firms that had received venture funding as of the end of 2006. Because VentureSource requires that firms be identified based on company name, a process was developed to broaden the net, identifying all firms that could conceivably be matched to firms in the NIH awards database (using wildcards in the database search6). These possible matches were then tested manually against known company addresses to eliminate false positives from the results dataset. 7 The final list from VentureSource provides considerable detail on venture capital investments: It indicates the name of the company, the date of the round of funding, the type of funding, and the amount.8 This appears to be the most 2 For details on the NRC Phase II Survey, which included at least one questionnaire to every Phase II winner 1992-2002 inclusive, see National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2009. 3 For details on the methodology and scope of the NIH Phase II Survey, see National Institutes of Health, National Surey to Ealuate the NIH SBIR Program: Final Report, July 2003, available at . 4 U.S. Government Accountability Office, Small Business Innoation Research: Information on Awards Made by NIH and DoD in Fiscal Years 2002 through 200, GAO-06-565, Washington, DC: U.S. Government Accountability Office, April 2006. 5A secondary database owned by was also used by GAO but was not made available to the NRC. 6We supplied a list of firm names; VentureSource replaced part of the name with wildcards and then searched the database. 7The VentureSource search results included information on the city and state of the firm, and these data were cross-checked against city and state data from the SBIR awards database. 8 “Type of funding” refers to terminology within the venture capital community. Thomson and others distinguish between seed funding, A round funding, B round funding and C round funding.

OCR for page 22
2 VENTURE FUNDING AND THE NIH SBIR PROGRAM definitive list of venture-funded firms available and it is used in the first phase of this study.9 Estimating the “Exclusion Effect” It is important to keep in mind that simply receiving some venture funding is in itself not disqualifying. To be disqualified from participating in the SBIR program, firms must be owned or controlled by firms that themselves fail one of the two tests—breaching the size requirement and/or the individual ownership requirement—outlined above in Box 1-1.10 Thus, the list of venture-funded firms was then analyzed to determine whether it was likely that these firms would in fact be excluded by the SBA ruling. Unfortunately, privately owned firms are often very reluctant to provide in- formation about their ownership structure. It is therefore not possible to determine directly which venture-funded firms are owned or controlled by their venture investors (and hence excluded) and which are not. Nor it is practical to examine the ownership structure of every venture capital firm that provides funding in order to determine whether their ownership structure or the collective character of their other investments breach the eligibility requirements. It is also worth noting that firm ownership and control are by no means syn - onymous.11 Ownership of a majority of outstanding voting shares is sufficient to provide formal control. However, key personnel may still exert significant— sometimes predominant—control over key decisions. Conversely, 51 percent ownership is not necessarily required in order to exert effective control. However, both because the SBA ruling focuses on 51 percent ownership and because any statistical analysis must find ways to draw bright lines through murky questions, this study assumes that the critical delineator for the purpose of access to SBIR is 51 percent ownership. After considerable discussion, and drawing on their extensive experiences, the Committee agreed on two proxies for venture control of a firm 12: 9 It is important to note that there is considerable heterogeneity among venture capital firms themselves. For a review of some of the differences, see William A. Sahlman, “The Structure and Governance of Venture-capital Organizations,” Journal of Financial Economics, 27(2):473-521, October 1990. 10There are also small biotechnology companies that are 51 percent owned by individuals but are excluded based on affiliation of a venture capital company’s other portfolio companies. 11 Eugene Fama and Michael Jensen distinguished between ownership and control in their classic 1983 paper. See Eugene Fama and Michael Jensen, “Separation of Ownership and Control,” Journal of Law and Economics, XXVI, 1983. The role of venture capital control of small innovative busi - nesses is discussed in Paul Gompers and Josh Lerner, “The Venture Capital Revolution,” The Journal of Economic Perspecties, 15(2):145-168, Spring 2001. 12 Committee members with extensive knowledge of venture capital include Linda Powers (Toucan Capital Corporation), Michael Borrus (X/Seed Capital), Clark McFadden (Dewey & LeBoeuf, LLP), and Pete Linsert (Columbia Biosciences Corporation).

OCR for page 22
2 STUDY METHODOLOGY • Venture capital investments of $5 million or more; or • At least two separate rounds of venture funding. In both cases, it is reasonable to assume that 51 percent of company shares will have passed into the hands of the new investors.13 Using these proxies, the Committee screened the original list of venture- funded firms to develop a list of firms that it identified as having been excluded from the SBIR program on the basis of the SBA ruling. This screened list is referred to as “venture-funded firms” throughout this report. 14 Next, this list was adjusted to take into account other grounds for exclusion. Most notably, this list includes firms that would be subject to exclusion from the SBIR program on other grounds, regardless of their venture capital ownership structure. This includes circumstances where the firm— 1. Had more than 500 employees. 2. Was purchased by a foreign firm, or by a U.S. firm not 51 percent individual-owned. After eliminating firms that were venture-funded and met criteria 1 or 2 above for venture control, but also met one or more of the criteria above, we were left with a final list of venture-funded companies that were likely to be excluded based on the SBA ruling. Grouping the Firms In order to provide the most relevant analysis, the Committee focused on two sets of variables: • Whether a firm was or was not venture-funded, as defined above. • Whether a firm was among the top 200 most prolific winners of NIH Phase II funding during the 1992-2002 period.15 This approach generated the matrix in Table 2-1. 13This assumption was later tested against companies known to be excluded on venture capital grounds. We found that our criteria in these cases matched known outcomes with a high degree of accuracy. 14 It is important to note that these categories of majority venture funded firms as opposed to venture-funded firms are frequently not static. As often happens in the biotechnology industry, a company may have some venture funding and be eligible for SBIR Phase I and Phase II awards, but then receive venture capital additional funding and become ineligible for further SBIR funding. 15The numerical basis for whether a firm was among the top 200 most prolific winners was driven by the need for more detailed data and the constraints imposed by limited study resources. “Prolific” is defined as being among the winners of the most NIH Phase II SBIR awards in 1992-2002.

OCR for page 22
2 VENTURE FUNDING AND THE NIH SBIR PROGRAM TABLE 2-1 Firms Funded by the NIH SBIR Program (1992-2002) Top 200 Winners Other Winners Totals Percent VC-Funded 35 150 185 12 Not VC-Funded 165 1,186 1,351 88 Total 200 1,336 1,536 100 SOURCE: U.S. Small Business Administration Tech-Net Database, VentureSource. NOTE: Table 2-1 refers only to majority venture-funded firms that meet the two core venture capital investment criteria for control—namely more than $5 million in investment or more than one round of venture control funding. The Top 200 winners accounted for 42.3 percent of Phase II awards during 1992-2002. Majority venture-funded firms were clustered among the top 200 winners. They accounted for a higher percentage of the top 200 winners (17.5 percent) than they did of all NIH winners. Table 2-1 shows that overall, about 12 percent of firms that won SBIR Phase II awards from 1992-2002 inclusive were also venture capital-funded. In addition, firms that were more prolific in winning SBIR awards were also more successful in attracting significant venture capital funding: 35 (17.5 percent) of the top 200 award winners were majority venture-funded, as against 11.2 per- cent of the remaining 1,336 SBIR award winners.16 2.2 METHODOLOGY FOR MEASURING THE IMPACT OF THE SBA RULING The primary objective of this part of the assessment is to compare outcomes for SBIR projects implemented by venture-funded and non-venture-funded firms respectively. Two kinds of outcomes are utilized: • Project-leel outcomes identified by the NRC and NIH surveys. These include sales and the attraction of additional funding as the two key metrics. 17 • Firm-leel outcomes, generated by cross-referencing firms with Phase II awards against revenue and employment data extracted from the Hoover’s database of small firms. 16 It is important to note, as well, that while SBIR enhances the success of venture-backed firms, the obverse is also true. It is also possible that the top 200 SBIR award winners had topics that were more advanced in their development. 17Additional funding as used in the NRC and NIH surveys means funding from all sources, includ - ing VCs, angel funding, and additional non-SBIR federal funding (a major component).

OCR for page 22
27 STUDY METHODOLOGY Project-level Outcomes Until recently, there were no available data on outcomes from SBIR pro - grams through which to compare the performances of venture-funded and non- venture-funded firms. Recent surveys by the NRC and the NIH, however, provide at least initial indications of relative outcomes.18 The NIH survey generated 768 responses (1 per firm) and the NRC 496 responses from 368 firms.19 Together, the surveys generated responses cover- ing at least one project from 861 firms that received Phase II awards during this period.20 Firm-level Outcomes While the NRC/NIH data provide important insights into outcomes from spe- cific surveyed projects, it is also useful to generate a different perspective based on the development of the firm, rather than an individual project. Utilizing the Hoover’s database of small firms, we developed a dataset of current revenue and employee data, which provides a useful proxy for the over- all commercial success of the firm.21 Unfortunately, Hoover’s does not maintain time-series data on individual firms, so our metrics were based on the size of firm revenues and number of employees as of the most recent data available from Hoover’s (in most cases, for 2006). As each data point in this dataset had to be collected manually, it was not cost effective to collect individual data on more than 1,200 firms. Accordingly, while we collected data for all 183 venture-funded firms,22 we limited data col- lection for non-venture-funded firms to an equivalent random sample: 35 firms among the top 200 most prolific Phase II winners, and 148 firms from among the pool of firms that were not among the most prolific winners and were not venture-funded. 18 See National Institutes of Health, National Surey to Ealuate the NIH SBIR Program: Final Report, op. cit. The NRC survey results for NIH are presented in National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, C. Wessner, op. cit. 19There is some overlap between the projects surveyed by the NIH and NRC. However, as project ID data is not directly comparable, it is not possible to determine exactly the dimensions of that overlap. 20 Not every firm responded to the survey questionnaires. The 861 firms are a subset of the 1,536 firms that won Phase II awards 1992-2002. 21The Hoover’s database of small firms is an extension of the well-known Dunn and Bradstreet database. It is the most comprehensive source of information on small businesses in the United States. Even so, it offers only a current snapshot (no historical data), and is not entirely comprehensive as it, in turn, is based in part on survey data. Hoover’s Small Business Database can be accessed on line at . 22Table 3-4 derives the total number of firms that were excluded or possibly excluded as a direct impact of the SBA directive.

OCR for page 22
2 VENTURE FUNDING AND THE NIH SBIR PROGRAM BOX 2-1 Downward Bias in the Estimation of the Impact of the SBA Directive Assessing the impact of the SBA exclusion of majority venture-funded firms is difficult because obtaining data on the ownership structure of companies, espe- cially private-held small firms, is hard to do. This assessment therefore relies on a sample of firms that received Phase II SBIR awards during the period 1992-2002 (before the directive was issued). The use of such data is likely to yield a down- ward bias in the estimate of the effects of the SBA exclusion for two reasons: • Recent shift of venture capital to the biotechnology sector. The life sci- ences and biotechnology were relatively embryonic industries during the sample period (before 2002). More importantly, venture capital started shifting towards biotechnology and life sciences after the Internet bubble burst in 2001.a Thus, the number and proportion of small life science-based firms receiving venture funding may be much higher than can be captured from data focused on the 1992-2002 period. • Venture funding barred as biotechnology firms matured. Leading papers in the economics of innovation literature point out that technological and product development opportunities (and thus, commercialization of research) become more prevalent as an industry matures.b Venture funding is focused mainly on product development. This implies that it is likely that the SBA ruling may be excluding a higher percentage of firms in this sector as the industry matures. aDouglas P. Lee and Mark D. Dibner, “The Rise of Venture Capital and Biotechnology in the US and Europe,” Nature Biotechnology, 23:672-676, 2005. bOn product innovation and commercialization in a wide variety of industries, see Edwin Mansfield, “Academic Research and Industrial Innovation: An Update of Empirical Findings,” Research Policy, 26(7-8):773-776, April 1998. 2.3 CASE STUDY AND OTHER DATA One final area of analysis concerns the impact of the ruling on the biotech industry itself. As noted earlier, BIO and other groups have claimed that the exclusion of venture funded companies from SBIR will result in a critical “gap” in the funding flow, one that may prevent important discoveries from being commercialized. To substantiate these claims, BIO conducted telephone and Internet sur- veys of its “emerging company” membership—defined as firms with fewer than 350 employees and no marketable products.23 The surveys suggest that a large 23 BIO appears to have contacted all 650 firms that make up its emerging company membership, with a response rate of about 41 percent. BIO provided no additional background information about the surveys. For a description of the surveys, see Appendix E of this report.

OCR for page 22
29 STUDY METHODOLOGY majority of responding biotech companies would apply for NIH funding absent the ruling, and that most would not use the funds for their lead product. Two thirds of the proposed research was reported to be focused on preclinical or discovery stages. BIO also conducted six case studies designed to show that promising lines of early-stage research have been abandoned or delayed as a result of the ruling. These cases, however, do not provide counterfactual evidence based on products that were funded prior to the SBA ruling, which would have been excluded by the ruling. These cases will be discussed in the context of the NRC’s own cases, which included some venture-funded firms. An analysis of the evidence submitted by BIO is included in Appendix E of this report.