National Academies Press: OpenBook
« Previous: 1 Assessing the SBIR Fast Track and Phase II Enhancement Programs at the Department of Defense
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 28
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 29
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 30
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 31
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 32
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 33
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 34
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 35
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 36
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 37
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 38
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 39
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 40
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 41
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 42
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 43
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 44
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 45
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 46
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 47
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 48
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 49
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 50
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 51
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 52
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 53
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 54
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 55
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 56
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 57
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 58
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 59
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 60
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 61
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 62
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 63
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 64
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 65
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 66
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 67
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 68
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 69
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 70
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 71
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 72
Suggested Citation:"2 Survey Analysis." National Research Council. 2009. Revisiting the Department of Defense SBIR Fast Track Initiative. Washington, DC: The National Academies Press. doi: 10.17226/12600.
×
Page 73

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

2 Survey Analysis1 As we have seen in the previous chapter, the Fast Track and Phase II Enhancement programs at the Department of Defense, while sharing the common objective of supporting SBIR commercialization, are different programs with largely different objectives. The analysis in this chapter thus focuses on two separate comparisons based on surveys of DoD SBIR awards: that between Fast Track award winners and a control group, and that between Phase II Enhancement award winners and a control group.2 2.1 SUMMARY OF SURVEY FINDINGS The NRC survey generated a range of useful data related to characteristics and outcomes for FT and PIIE projects in relation to the control group: 1. Demographics of each group. 2. Outcomes from awards. 3. Gap reduction. 1 The survey methodology for this update of the DoD Phase II Enhancement program is described in Appendix A. 2 This chapter presents an exposition and analysis of the survey responses for the three sample groups: Fast Track, Phase II Enhancement, and the population matched control group. The control group refers to subjects that did not participate in the Fast Track or Phase II Enhancement programs, but in all other respects were treated in the same way as the experimental group. Appendix B contains the questions on the Firm Survey and a summary of the individual responses. The 232 respondents answered 18 questions, many containing multiple parts; thus the complete data display is 232 by 72 fields and includes some answers (fields) that are at least one sentence long. Hence, summaries are the only reasonable way to publish the data. Appendix C contains the questions on the Project Survey and a summary of the individual responses. The 240 respondents answered 36 questions, many containing multiple parts. The NRC surveyed all companies receiving Phase II awards during the study period. Firms with multiple awards received more than one project survey. As a result, there are more project surveys than firm surveys. 28

SURVEY ANALYSIS 29 2.1.1 Survey Demographics The survey aimed to reach projects that fell into three groups: • Fast Track (FT). The survey sought responses from the 217 FT awards made from 1997-2002. • Phase II Enhancement (PIIE) awards. The survey selected a sample of 210 PIIE awards. (The sample was weighted to adjust for the year of award). Thirty-four also received FT awards.3 • PIIE and FT. The survey also addressed 34 projects that received both FT and PIIE awards.4 • Control group (neither FT nor PIIE). The sample of 376 was weighted to ensure that the number of awards per year and the source of awards by DoD service was similar to those for the FT and PIIE projects samples combined.5 Thus, in total, 837 (217+210+34+376=837) projects were surveyed. Responses are provided in the Table 2-1. TABLE 2-1 Survey Response Rates by Category Award Surveys Email Could Not Completed Overall Contacted Award Contact Contact Survey Response Response Category (Number) (Number) (Number) Rate (%) Rate (%) Fast Track 156 61 50 23 32 Phase II 198 12 69 33 35 Enhancement Both Fast 32 2 14 41 44 Track and Phase II Enhancment Control 331 45 107 28 32 Group SOURCE: NRC Project Survey 2006. NOTE: Fourteen responses were received from projects that received both FT and PIIE awards. In aggregating the data, responses from these projects were added to both the FT and PIIE categories. 3 See Appendix A, footnote 5, for details. 4 For a description of the sample selection, see Appendix A. 5 The control group sample was larger to account for the possibility that these projects might generate a lower response rate.

30 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE BOX 2-1 Multiple Sources of Bias in Survey Response Large innovation surveys involve multiple sources of bias that can skew the results in both directions. Some common survey biases are noted below.a o Successful and more recently funded firms are more likely to respond. Research by Link and Scott demonstrates that the probability of obtaining research project information by survey decreases for less recently funded projects and it increased the greater the award amount.b Nearly 40 percent of respondents in the NRC Project Survey began Phase I efforts after 1998, partly because the number of Phase I awards increased, starting in the mid 1990s, and partly because winners from more distant years are harder to reach. They are harder to reach as time goes on because small businesses regularly cease operations, are acquired, merge, or lose staff with knowledge of SBIR awards. o Success is self-reported. Self-reporting can be a source of bias, although the dimensions and direction of that bias are not necessarily clear. In any case, policy analysis has a long history of relying on self-reported performance measures to represent market-based performance measures. Participants in such retrospectively analyses are believed to be able to consider a broader set of allocation options, thus making the evaluation more realistic than data based on third party observation.c In short, company founders and/or principal investigators are in many cases simply the best source of information available. o Survey sampled projects at firms with multiple awards. Projects from firms with multiple awards were under-represented in the sample, because they could not be expected to complete a questionnaire for each of dozens or even hundreds of awards. o Failed firms are difficult to contact. Survey experts point to an “asymmetry” in their ability to include failed firms for follow-up surveys in cases where the firms no longer exist.d It is worth noting that one cannot necessarily infer that the SBIR project failed; what is known is only that the firm no longer exists. o Not all successful projects are captured. For similar reasons, the NRC Project Survey could not include ongoing results from successful projects in firms that merged or were acquired before and/or after commercialization of the project’s technology. The survey also did not capture projects of firms that did not respond to the NRC invitation to participate in the assessment. o Some firms may not want to fully acknowledge SBIR contribution to project success. Some firms may be unwilling to acknowledge that they received important benefits from participating in public programs for a

SURVEY ANALYSIS 31 variety of reasons. For example, some may understandably attribute success exclusively to their own efforts. o Commercialization lag. While the NRC Project Survey broke new ground in data collection, the amount of sales made—and indeed the number of projects that generate sales—are inevitably undercounted in a snapshot survey taken at a single point in time. Based on successive data sets collected from NIH SBIR award recipients, it is estimated that total sales from all responding projects will likely be on the order of 50 percent greater than can be captured in a single survey.e This underscores the importance of follow-on research based on the now-established survey methodology. 14 12 Sales (Millions of Dollars) 10 8 6 4 2 0 1 2 3 4 5 6 7 8 9 10 11 12 Survey Taken Years after Phase II Award FIGURE B-2-1 Survey bias due to commercialization lag. Figure B-2-1 These sources of bias provide a context for understanding the response rates to the NRC Survey conducted for this study. continued

32 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE o Limited statistical analysis. Owing to the constraints imposed in some cases by low sample size, more sophisticated statistical analysis could not be performed. Accordingly, while the analysis below compares a control group of projects with Fast Track and Phase IIE projects respectively, different outcomes may not always be statistically significant. a For a technical explanation of the sample approaches and issues related to the NRC surveys, see Appendix A. b Albert N. Link, and John T. Scott, Evaluating Public Research Institutions: The U.S. Advanced Technology Program’s Intramural Research Initiative, London: Routledge, 2005. c While economic theory is formulated on what is called “revealed preferences,” meaning individuals and firms reveal how they value scarce resources by how they allocate those resources within a market framework, quite often expressed preferences are a better source of information especially from an evaluation perspective. Strict adherence to a revealed preference paradigm could lead to misguided policy conclusions because the paradigm assumes that all policy choices are known and understood at the time that an individual or firm reveals its preferences and that all relevant markets for such preferences are operational. See {1} Gregory G. Dess and Donald W. Beard, "Dimensions of Organizational Task Environments," Administrative Science Quarterly, 29:52-73, 1984; {2} Albert N. Link and John T. Scott, Public Accountability: Evaluating Technology-Based Institutions, Norwell, MA: Kluwer Academic Publishers, 1998. d Albert N. Link, and John T. Scott, Evaluating Public Research Institutions: The U.S. Advanced Technology Program’s Intramural Research Initiative, op. cit. e Data from NIH indicates that a subsequent survey taken two years later would reveal very substantial increases in both the percentage of firms reaching the market, and in the amount of sales per project. See National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2008. 2.1.2 Firm and Project Characteristics6 Phase II Enhancement (PIIE) and Fast Track (FT) are aimed at different objectives and have different requirements, so it is not surprising that the characteristics of firms in these programs were also different. Fast Track Projects Firms that tended to be smaller and founded more recently than the control group and also to have less SBIR experience undertook these projects. • Employees. Forty-three percent of Fast Track respondents reported having five employees or fewer at the time of the award, about a quarter more than the 36 percent in this category for the control group.7 6 All data provided in this report refer to NRC Fast Track and Phase II Recipient survey respondents in a particular category, not all awardees, unless specifically noted otherwise. 7 See Figure 2-2, NRC Project Survey 2006, Question 16, Appendix C.

SURVEY ANALYSIS 33 • Foundation year. The median year of foundation was 1995 for Fast Track recipients and 1988 for the control group. More than 2/3rds of the Fast Track firms were founded after 1992, compared with less than a third of the control group.8 • Prior Phase II awards. Almost half (48 percent) of FT winners reported zero prior Phase II awards, compared to 23 percent of control group firms.9 • Size of awards. FT projects received about $100,000 more in SBIR funding than did the control groups projects.10 Phase II Enhancement Projects Overall, firms with PIIE were more closely aligned with the experience profile of the control group: • Employees. PIIE firms reported that only 24 percent of firms had five or fewer employees. This is one third less than the control group.11 • Foundation year. PIIE firms were slightly younger than the control group, with a median foundation year of 1990 (compared with 1998).12 • Prior SBIR experience. PIIE and control group firms had almost identical prior SBIR experience, with about 24 percent having received zero prior SBIR Phase II awards, and 17-20 percent having received more than five prior Phase II awards.13 • Size of awards. As one important feature of PIIE is the provision of up to $500,000 in matching funds from SBIR, it is not surprising that on average PIIE projects received about $400,000 more than control group projects.14 Thus, the data indicate that younger firms with fewer employees have tended to utilize Fast Track, while PIIE firms are quite similar in profile to control group. Both received more funding than the control group, PIIE firms considerably more. 8 See Table App-A-3 in Appendix A 9 See Figure 2-4, NRC Project Survey 2006, Question 19, Appendix C. 10 See DoD Awards Database. 11 See Figure 2-5, NRC Project Survey 2006, Question 16b, Appendix C. 12 NRC Firm Survey 2006. Part A, Question 7, of the Firm survey asked firms the year their company was founded. 13 See NRC Project Survey 2006, Question 19. See Appendix C. 14 See DoD Awards Database.

34 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE 2.1.3 Project Outcomes The NRC methodology identified a number of metrics that can be used to develop an assessment of project outcomes for SBIR. These include: • Employment effects. • Sales revenues. • Sales by sector. • Additional investments. • The project initiation decision. • Project delays absent SBIR. • Knowledge effects. 2.1.3.1 Employment Responding firms from all three groups showed substantial employment gain between the time of the award and the time of the survey. • FT firms started from a much lower employment profile. They grew larger firms at about the same rate as the control group, but also rapidly reduced the share of firms with 1-5 employees, from 43 percent to 16 percent.15 • PIIE. Starting from about the same base, PIIE firms grew about twice as fast as the control group in the largest employment category (firms with 50 or more employees). Forty percent of responding PIIE firms were in this category at the time of the survey.16 2.1.3.2 Revenues The survey asked firms to attribute sales revenues (as well as expected sales and sales by licensees) to the related SBIR project. Given the high degree of skew in reported sales, where a few projects report very large positive results, it is important to ascertain both the overall level of projects that reach the market, and also their distribution. Reaching the market. All three groups reached the market at approximately the same rate17, which suggests that ceteris paribus, neither FT nor PIIE had a significant effect in enhancing the sales rate experienced by the control group’s projects (sales rate is defined as the percentage of projects that sold at least $1 in the marketplace).18 15 See Figure 2-5, NRC Project Survey 2006, Question 16b. See Appendix C. 16 Ibid. 17 See DoD Awards Database. 18 Use of the sales rate is valid only as one benchmark among many to capture the numerous dimensions of commercialization. However, it is still an important metric.

SURVEY ANALYSIS 35 Expected sales. FT and control group projects surveyed each reported that about 55 percent of those without sales to date still anticipated reaching the market at some point. About 66 percent of PIIE projects reported similar expectations.19 However, a comparison of expected sales from the previous (2000) survey and actual sales later reported indicates that firm descriptions of expected sales were highly optimistic, which suggests that expected sales reported in the current survey should be treated with caution. Distribution of sales by size. Given the high degree on skew in SBIR outcomes, in which a few projects account for a substantial share of total revenues generated by SBIR projects, it is important to review the distribution of sales by size. • FT. The distribution of sales for the Fast Track and Control Groups are quite similar. FT did not report a project with sales greater than $10 million, but it did report more projects with sales between $5 million and $10 million. 20 • PIIE. The PIIE group generated two awards with more than $10 million in cumulative revenues, and six with revenues of $5 million to $10 million, compared to one and two respectively for the control group.21 Given the very small numbers of awards involved, caution should be employed when drawing conclusions from these data. They are however suggestive. Sales by sector. One of the primary missions of the DoD SBIR program is to provide technology for use within DoD. Consequently, sales to DoD itself and to prime contractors for DoD are an important metric that the program is meeting its objectives. The survey reports substantial differences between the groups. In both cases, FT and PIIE projects are more likely to generate sales to DoD and its prime contractors than are the control group firms. • FT. About 70 percent of surveyed FT project sales by value were made to DoD and its primers. In contrast, this sector generated about 38 percent of sales for the control group. About 1/3rd of total revenues for FT projects came from direct sales to DoD, compared with 25 percent for the control group.22 • PIIE. Similarly, 65 percent of PIIE project revenues were from DoD and the primes. Direct sales to DoD in this case accounted for about 45 percent of the total.23 19 See NRC Project Survey 2006, Question 9. See Appendix C. 20 Ibid. 21 Ibid. 22 See NRC Project Survey 2006, Question 4. See Appendix C. 23 Ibid.

36 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE 2.1.3.3 Additional Investments Many projects require additional investment after Phase II before they can reach the market. These investments are therefore an additional metric that progress is being made with the project. Both the FT and PIIE programs require additional third party investment as a condition of the award, although the precise requirements are different. Consequently, both FT and PIIE projects generated much higher rates of additional investment than the control group. In this case then, the scale of additional investment may be significant. • FT. On average, FT projects attracted about $300,000 more in development funding than the control group. Interestingly, FT projects attracted smaller amounts of federal development funding, and considerably more private sector funding.24 This likely reflects the fact that FT projects must attract third party funding very early in the project cycle, when federal technology development funding is less available. Almost 35 percent of FT projects reported some kind of negotiations for the sale of equity, compared with 4 percent of the control group. And 14 percent had completed agreements at the time of the survey, compared with 1 percent of the control group.25 • PIIE. On average, PIIE projects generated just under $1 million in additional non-SBIR investment, in comparison with about $500,000 for the control group.26 That difference is entirely accounted for by the relative success of PIIE firms in attracting non-SBIR federal R&D funding (averaging just over $500,000 per project). 2.1.3.4 The Project Initiation Decision Firms were asked whether they would have undertaken the project absent the SBIR award in question, and also whether the SBIR. Data from the survey indicates that neither FT nor PIIE had a significant effect on this decision. • FT. Firms report that about the 66 percent of FT projects (66 percent) would certainly or probably not have been started without SBIR funding. This is approximately the same percentage as for the control group (70 percent).27 24 See NRC Project Survey 2006, Question 23. See Appendix C 25 See NRC Project Survey 2006, Question 12. See Appendix C. 26 See NRC Project Survey 2006, Question 23. See Appendix C. 27 See NRC Project Survey 2006, Question 13. See Appendix C.

SURVEY ANALYSIS 37 • PIIE. Here 75 percent of firms reported that the certainly or probably would not have started the project without SBIR—again, similar to results for the control group (70 percent).28 2.1.3.5 Project Delays Firms that believed their project would have proceeded without SBIR funding were further surveyed to determine whether those projects would have been significantly delayed without SBIR. • FT respondents indicated that on average their projects would have been delayed by 13 months, in comparison to 6 months for the control group.29 This is a substantial difference, which suggests that FT participants saw a particular value in the improved PI-PII transition offered via the FT program. • PIIE projects would have been delayed by 5 months, which is less than the control group.30 This relatively low anticipated delay may be based on the availability of the funding that was eventually used to make the PIIE match, which might have allowed rapid forward movement on the project even without SBIR. 2.1.4 Phase I-Phase II Funding Gap and Fast Track FT was designed explicitly to help reduce the funding gap that can occur between the end of Phase I funding and the beginning of Phase II. Survey data indicates that for responding projects, FT had a substantial impact in reducing funding gaps relative to those experienced by the control group. • Likelihood of funding gap. Survey data indicate that only about 35 percent of FT projects experienced a funding gap between PI and PII, in comparison with 70 percent of control group projects.31 • Length of funding gap. The average PI-PII funding gap for all FT projects was just over 1 month, compared with just under 6 months for the control group. For projects with some gap, FT projects averaged a gap of 4 months compared with more than 8 months for the control group.32 28 Ibid. 29 See NRC Project Survey 2006, Question 15a. See Appendix C. 30 Ibid. 31 See NRC Project Survey 2006, Question 31. See Appendix C. 32 Ibid.

38 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE • Impact of the gap. Just under 20 percent of FT respondents reported that the funding gap had caused them to stop work on the project, compared to 40 percent of the control group projects.33 • Trends. Data from DoD indicates that the size of the gap has increased and the percentage of projects reporting a gap has increased in recent years.34 2.2 DETAILED SURVEY RESULTS In order to ensure that appropriate comparisons are made, this section is divided into two: the first part addresses comparisons between the Fast Track projects and the control group; the second compares responses from the Phase II Enhancement group and the control group. 2.2.1 Phase II Completion Respondents were asked to categorize the status of the efforts resulting from the surveyed Phase II awards. Some had already discontinued the effort, but most had realized some level of commercialization or were in post-Phase II (development of the technology) phase. Although all Phase II awards in the three sample groups35 had been made by the end of 2002, by the summer of 2006, 15 of the 240 responses had not yet completed Phase II. Two of these 15 had garnered both a Fast Track award and a Phase II Enhancement. The status chart (Figure 2-1) shows that firms benefiting from the Phase II Enhancement awards appeared more likely than the control group to have products, services, or processes in use by the target population.36 Fast Track projects were not more likely to be in the comem4rcialization stage than the control group. 33 See NRC Project Survey 2006, Question 33. See Appendix C. 34 Information from the DoD Fast Track Database. 35 Fast Track awards that subsequently received a Phase II Enhancement were analyzed in both groups; thus, Fast Track analyzed 64 responses and Phase II Enhancement, 83 responses. 36 The chart shows the percentage of each response from each sample. The low number of responses and the relatively small differences in responses for each sample does not support strong conclusions for this question

SURVEY ANALYSIS 39 Phase II not 100 complete 90 Discontinued. No sales/additional 80 funding Percent of Respondents 70 Discontinued after sales/additional 60 funding 50 Post Phase II development 40 Commercialization 30 underway 20 Products/services/ processes in use 10 0 Fast Track Phase II Control Enhancement Group FIGURE 2-1 Status of Phase II awards. SOURCE: NRC Project Survey 2006, Question 1. See Appendix C. Figure 2-1.eps 2.2.2 Comparison of Firm Metrics The firm survey examined several factors that might influence firm success in SBIR. These include the size and age of the firm and its prior experience in SBIR. Two of the factors used in sample selection were number of Prior Phase II awards and size of the firm. Firm size was examined both in terms of number of employees and annual revenue. 2.2.2.1 Demographics: FT firms were Smaller and Younger Among firms responding to the survey, Fast Track award winners were smaller, younger, and with less SBIR experience at the time of the surveyed Phase II award than the control group. • Fast Track As shown in Figure 2-2, 43 percent of the Fast Track firms reported five or fewer employees at the time of award, compared with 36 percent of the control group.37 37 See NRC Firm Survey 2006, Question 6, in Appendix B.

40 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE >50 100 21-50 90 6-20 1-5 80 Percent of Respondents 0 70 60 50 40 30 20 10 0 Fast Track Phase II Control Group Enhancement FIGURE 2-2 Firm size when Phase II proposed: Number of employees. Figure 2-2.eps SOURCE: NRC Project Survey 2006, Question 16a. See Appendix C. • Twenty percent of Fast Track had more than 20 employees at the time of award, compared with 41 percent of the control group.38 • The median founding date reported by the Fast Track awardees was 1995. Two-thirds of these firms reported that they were founded after 1992, compared with a median founding date of 1988 for the control group. Less than a third of these firms were founded after 1992.39 • PIIE. Twenty-four percent of the Phase II Enhancement reported five or fewer and 36 percent of the firms responding for control group awards who reported five or fewer employees.40 • Forty-three percent of the Phase II Enhancement awards reported a firm size of more than 20 employees at the time of award, compared with 41 percent for the control group.41 • For Phase II Enhancement the median founding date was 1990. Only 37 percent of the firms responding for Phase II Enhancement reported a founding date after 1992. The control group reported a median founding date of 1988. Less than a third of these firms were founded after 1992.42 38 Ibid. 39 NRC Firm Survey 2006. Part A, Question 7, of the Firm Survey asked firms the year their company was founded. This paragraph summarizes 232 responses. 40 See NRC Firm Survey 2006, Question 6, in Appendix B. 41 Ibid. 42 NRC Firm Survey 2006. Part A, Question 7, of the Firm Survey asked firms the year their company was founded. This paragraph summarizes 232 responses.

SURVEY ANALYSIS 41 2.2.2.2 Prior SBIR experience Fast Track Overall, the firms with Fast Track awards reported that they had significantly less prior experience with both Phase I and Phase II SBIR awards, including both awards related to the specific award being surveyed and other awards. Figure 2-3 shows that 48 percent of Fast Track surveys reported that they had no Phase I awards prior to the award that led to the Fast Track Phase II, compared with 23 percent for the control group. Nine percent of the Fast Track surveys reported that the firm had received more than five Phase I awards prior to the surveyed Fast Track Phase I. four times as many projects in the control group reported that they had received more than five Phase I awards prior to the surveyed Phase I.43 Only 36 percent of Fast Track responses reported a prior Phase II and only six percent reported receiving more than five prior Phase II awards.44 The high percentage of Fast Track awards to firms who had no prior Phase II is not surprising. Such firms need only obtain one matching third-party dollar for four dollars of SBIR funds. Firms with one or more prior Phase II must find dollar for dollar matching funds to be eligible for Fast Track.45 Phase II Enhancement These projects reported prior experience with SBIR that was similar to that of the control group (see Figure 2-3 and 2-4). Seventy percent of the Phase II Enhancement surveys and 63 percent of the control group surveys reported receiving a prior Phase II award. As shown in figure 2-4, substantially more firms in these groups reported receipt of five or more prior Phase II awards.46 2.2.2.3 Growth in the Number of Employees The average firm in each group reported a larger number of current employees than at the time of the surveyed award. (See Figures 2-2 and 2-5.) Fast Track The number of firms reporting more than twenty employees doubled between the time of the award and the time of the survey, while the percentage reporting more than 50 employees grew from 11 to 20 percent. The percentage 43 See NRC Project Survey 2006, Question 19. See Appendix C. 44 Ibid. 45 In the past, some DoD components, like BMDO, in some cases allowed a company with fewer than five Phase II awards to get a two for one match. Also in the past, some companies were able to get more than two 4:1 matching Phase II Fast Track awards of the Fast Track application. In some cases the companies that had multiple Phase I awards in their first solicitation participation were able to get multiple Fast Track awards at very competitive matching rates. 46 See NRC Project Survey 2006, Question 19. See Appendix C.

42 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE Had no prior 100 Phase I awards Percent of Respondents 90 80 Had 1-5 prior Phase I awards 70 60 Had more than 5 50 Phase I awards 40 30 20 10 0 Fast Track Phase II Control Group Enhancement FIGURE 2-3 Prior Phase I experience: Number of prior Phase I awards. Figure 2-3.eps SOURCE: NRC Project Survey 2006, Question 19. See Appendix C. of Fast Track responses currently reporting five or fewer current employees has dropped from 43 percent to 16 percent durign that period.47 Control group employment has grown more slowly than FT, with thirty-one percent of Control Group firms reporting more than 50 employees, up from 22 percent at the time of the surveyed award. The percentage of Control Group firms reporting fewer than five employees has declined from 36 percent to 21 percent.48 Had no prior 100 Phase II awards Percent of Respondents 90 Had 1-5 prior 80 Phase II awards 70 Had more than 5 60 Phase II awards 50 40 30 20 10 0 Fast Track Phase II Control Group Enhancement FIGURE 2-4 Prior Phase II experience:2-4.epsof prior Phase II awards. Figure Number SOURCE: NRC Project Survey 2006, Question 19. See Appendix C. 47 Ibid. 48 See NRC Project Survey 2006, Question 16. See Appendix C.

SURVEY ANALYSIS 43 >50 100 21-50 90 6-20 1-5 80 Percent of Respondents 0 70 60 50 40 30 20 10 0 Fast Track Phase II Control Group Enhancement FIGURE 2-5 Current firm size: Number of employees. SOURCE: NRC Project Survey 2006, Question 16b. See Appendix C. Figure 2-5.eps Thus FT firms have closed some of the gap with the Control Group in building larger firms, with 20 percent now reportign at least 50 employeedm, compared with 31 percent for the CG. The FT has dramatically reduced the number of very small firms, and now has fewer such firms than the Control Group (26 percent vs. 21 percent). It is worth noting that the smaller percentage of FT firms that responded to the sdurvey may indicate a higher percentage for firms that are out of business (and hence have zero employees). Phase II Enhancement These firms were larger at the time of award (see above). Still, the percentage of responses currently reporting more than 50 employees (40 percent) has nearly doubled since the receipt of the surveyed award (22 percent). The number of PIIE firms reporting fewer than five employees has declined from 24 percent to 11 percent.49 PIIE firms now have a bigger share of large firms (more than 50 employees) than do the Control Group (40 percent vs. 31 percent). This was not the case at the time of the award, when both groups had simialr shares. The share of firms reporting fewer than five emopoloyees has declined at about the same rate for PIIE and Control Group firms.50 49 Ibid. 50 Ibid.

44 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE 2.2.2.4 Firm Sales and Revenues Along with employment, respondents also identified firm size using the metric of annual sales. Firms were asked whether they had ever applied for Fast Track and whether they had ever applied for Phase II Enhancement. Fifty-four of the 204 responding firms had not applied for either. Thirty-five applied only for Fast Track and 63 applied only for Phase II Enhancement. Forty-eight firms had applied for both programs. The four pie charts in Figure 2-6 show the average firm annual revenue for each group.51 Figure 2-6 shows that the annual revenue for largest percentage of firms in each group is $1,000,000 to $5,000,000. • The group that applied only for Fast Track has the smallest percentage over $5,000,000 in annual revenue and the highest percentage under $500,000. • The group that applied for both Fast Track and Phase II Enhancement has the highest percentage over $5,000,000 in annual revenue. • The Phase II Enhancement only group has the second highest percentage of firms with annual revenue in excess of $5,000,000. Survey results suggest that larger firms are more likely to apply for Phase II Enhancement and smaller firms more likely to apply for Fast Track. 2.2.2.5 Participation by Women and Minorities The original 1982 SBIR legislation specifically identifies “participation by minority and disadvantaged persons in technological innovation” as an objective of the program. Participation by women was not addressed in the legislation until the reauthorization in 1992; hence, the 1992 GAO study did not address woman-owned firms.52 Since the 1999 NRC study was specific to Fast Track, it did not consider firm ownership.53 51 Firms are grouped by their responses as to whether they applied, not by whether they received the award. It would not be appropriate to categorize the firms solely by their projects in the sample as, for example, a firm might have won a Phase II Enhancement that was not sampled or won either a Fast Track or an Enhancement after 2002. Revenue was as reported for 2005, not at the time of award. 52 U.S. General Accounting Office, Small Business Innovation Research Shows Success But Can Be Strengthened, GAO/RCED-92-37, Washington, DC: U.S. Government Printing Office, March 1992. 53 Neither the initial legislation, nor the 1992 reauthorization, established percentage goals for SBC that are minority- or woman-owned. Legislation focused on encouraging participation and dissemination of information. PL 102-564 states as a purpose “to improve the Federal Governments dissemination of information concerning the small business innovation research program, particularly with regard to program participation by woman-owned small business concerns and by socially and economically disadvantaged small business concerns.”

SURVEY ANALYSIS 45 Fast Track Only Neither Phase II Enhancement Only Both <$100,000 $5,000,000–$19,999,000 $100,000–$499,000 $20,000,000–$99,999,000 $500,000–$999,000 $100,000,000+ $1,000,000–$4,999,000 FIGURE 2-6 Firm annual revenue. SOURCE: NRC Firm Survey 2006, Question 8. See Appendix B. Figure 2-6.eps The analysis below identified firms by woman and minority ownership through analysis of the DoD Submissions Database. It then analyzed survey response in light of those data.

46 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE TABLE 2-2 Ownership of Firm Survey Responses: Minority Status (Number) Program Woman Minority Both Neither Total Control Group 10 7 1 89 107 Fast Track 9 2 3 50 64 PHII Enhancement 7 5 0 70 82 Total 26 14 4 209 253 Survey Responses: Percentage of Awards in Category Program Woman Minority Both Neither Total Control Group 9.3 6.5 0.9 83.2 100.0 Fast Track 14.1 3.1 4.7 78.1 100.0 PHII Enhancement 8.5 6.1 0.0 85.4 100.0 SOURCE: DoD SBIR Submissions Database. NOTE: The DoD SBIR Submissions Database, which contains proprietary information on the firms, was made available to NRC on a nondisclosure basis. Note also that the fast Track and Phase II Enhancement categories each also include first that received both fast Track and PIIE awards. • Ownership. Since ownership by women and minorities could be a discriminator, the current NRC assessment determined if the owner was a woman or a member of a minority group. (See Table 2-2.) Fast Track Given the limited number of responses in each category, there were no significant differences between FT and the Control Group in terms of woman and minority ownership overall, although FT awards were less likely than the control group to go to minority-owned firms and more likely to go to woman-owned firms. PIIE PIIE firms closely tracked the control group in terms of woman and minority ownership. • Principal Investigators. Ownership is not the only means of participation in SBIR. The current survey also examined the Phase III performance of projects where women and minorities had the opportunity to serve as Principal Investigators (PI) on the Phase II awards. Figure 2-7 displays whether the PI was a woman or minority.

SURVEY ANALYSIS 47 Although there is no information concerning which cases where the owner was the PI, a cross comparison of the ownership with the PI question shows that women and minorities are more likely to be the PI in a firm that is owned by a woman or a minority than in one where the ownership is neither woman nor minority. The small percentage of women and minorities as owners or PI did not support a detailed analysis of responses by firms that are owned by members of a minority group or by women.54 2.2.3 Would Small Firms Have Commercialized Without SBIR? When we attribute sales or other economic activity to SBIR, we must assess whether these sales would have occurred in the absence of SBIR. To do so, we asked the companies. Figure 2-8 displays responses to the question of whether the company would have undertaken the project in the absence of SBIR. Firms could respond that they definitely or probably would have undertaken the research had they not received the SBIR award, or that they probably or definitely would not have done the research without SBIR. The fifth possible response was that they were not certain what they would have done. Neither woman 100 nor minority PI 90 Percent of Respondents 80 Minority PI 70 Both woman 60 and minority PI 50 Woman PI 40 30 20 10 0 Fast Track Phase II Control Group Enhancement FIGURE 2-7 Principal investigators: Participation by women and minorities. SOURCE: NRC Project Survey 2006, Question 17. See Appendix C. Figure 2-7.eps 54 DoD SBIR Submissions Database cross-referenced with NRC Project Survey 2006, Question 17.

48 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE 60 Percent of Responses by Program 50 Control Group Fast Track Fast Track and Phase II Enhancement 40 Phase II Enhancement 30 20 10 0 Definitely Probably Uncertain Probably Definitely Yes Yes Not Not FIGURE 2-8 Commercializations absent the SBIR award. Figure 2-8.eps SOURCE: NRC Project Survey 2006, Question 13. See Appendix C. N= 107 (control group), 51 (Fast Track), 13 (FT and PIIE), 69 (PIIE) Fast Track As shown in Figure 2-8, two-thirds (66 percent) of the Fast Track award recipients surveyed reported that they definitely or probably would not have undertaken the project without SBIR. Seventeen percent of the Fast Track respondents said that they definitely or probably would have undertaken the project in the absence of the SBIR. This percentage is surprisingly low given that all Fast Track award winners surveyed were able to raise third-party funding for their projects during or before Phase I. The large number of uncertain answers for Fast Track may indicate that they were not sure the third-party would have invested without the SBIR award. These data are closely comparable to the 70 percent of the control group that reported that they would not have undertaken the project absent SBIR funding. Phase II Enhancement Seventy-five percent of the Phase II Enhancement group report that they would not have undertaken the project absent SBIR funding. Sixteen of the 84 Project Survey respondents who said that they definitely would not have undertaken the work in the absence of SBIR, reported sales of more than $1 million resulting from the SBIR effort.55 55 Cross-reference of individual survey responses to NRC Project Survey 2006, Questions 9a and 13. See Appendix C.

SURVEY ANALYSIS 49 2.2.3.1 Delay in the Absence of a Phase II Awardees who responded that they would have or probably would have undertaken the research and development if they had not received the SBIR award were asked about the impact of that undertaking had they not received the Phase II award.56 All groups reported that the start of the work would have been delayed. • Fast Track participants anticipated a 13-month delay. This compares with 6 months for the control group. This again is surprising, as FT respondents might have been able to access the matching funds they had already generated.57 • Phase II Enhancement participants predicted an average delay of 5 months, while the control group anticipated a 6-month delay.58 2.2.4 Actual and Expected Sales in Phase III 2.2.4.1 Defining a Sale The survey definition of sales attempted to cover all of the possible types of sales and customers that a small business might develop in Phase III.59 • As defined in the questionnaire, sales included all sales of products(s), process(es), service(s), or other sales to federal or private sector, or to other customers, resulting from the technology associated with the specific Phase II project. • A sale could also include the sale of technology or rights, which was counted as part of the total sales activity. Additionally firms were asked to report licensee sales, when that information was known. 2.2.4.2 Sales to Date The initial question simply concerns the percentage of projects reporting that they reached the commercial market-place—that they recorded at least $1 in sales. They survey asked this question both directly and indirectly by asking about total dollar sales to date. Fast Track Data from the latter question indicates that the sales rates for the FT and the Control Group are sufficiently similar that it is not possible to identify any significant differences, as illustrated in Figure 2-9. Forty-seven percent of 56 See NRC Project Survey 2006, Question 15. See Appendix C. 57 Ibid. 58 Ibid. 59 See NRC Project Survey 2006, Question 3. See Appendix C.

50 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE 58 56 55.4 That Reported Some Sales to Date 54 Percent of All Respondents 52 50.0 50 48 46.7 46 44 42 Control Group Fast Track Phase II Enhancement (n=107) (n=64) (n=83) FIGURE 2-9 Percentage of all responding projects reaching the market. SOURCE: NRC Project Survey Figure 2-9.epsSee Appendix C. 2006, Question 4. Control Group projects reached the market, while 50 percent of FT projects did so. Phase II Enhancement The same data do suggest that there may be an association between PIIE and an increased sales rate, reflecting a higher percentage of projects that reached the market, as measured by this metric: 55 percent of PIIE projects reported some sales, while 47 percent of Control Group projects did so. 2.2.4.3 Expected Sales Two-thirds of the Phase II Enhancement awards that do not yet have sales, expect to achieve sales. For the other two groups, about 55 percent of those without sales expect to achieve sales.60 60 Ibid.

SURVEY ANALYSIS 51 Export markets 100 Other 90 Percent of Respondents State or local 80 governments 70 Other agencies 60 50 NASA 40 Prime contractors 30 for DoD 20 DoD 10 Domestic private 0 sector Fast Track Phase II Control Group Enhancement FIGURE 2-10 Distribution of Figure 2-10.eps III sales going to each sales: Percent of Phase sector. and also SOURCE: NRC Project Survey 2006, Question 4. See Appendix C. Figure 2-13.eps 2.2.4.4 Sales Distribution by Size of Sales Overall, approximately 54 percent of the projects responding to the survey reported sales.61 They reported $268.5 million in actual sales through August 2006.62 This amounts to an average sale of $1,118,600 for the 240 responding projects.63 Given the high degree of skew in SBIR outcomes, it is important to review the distribution of sales by size in order to make relevant comparisons. Fast Track The distribution of sales for the Fast Track and control groups are—as Figure 2-11 indicates—quite similar. FT did not report a project with sales greater than $10 million, but it did report more projects with sales between $5 million and $10 million. Phase II Enhancement The distribution of awards by size indicates that the PIIE group generated more awards with larger revenues, as reflected in Figure 2-12. 61 Ibid. 62 See NRC Project Survey 2006, Question 4, in Appendix C. The percentages going to each sector are percentages based on sales by the firm. Firms were not asked about sector sales by their licensees as it was felt that they would not have good information on the customers of licensees. Sales by the firm, depicted in Figure 2-11 totaled $231 million. Sales by licensees, not depicted in Figure 2-11 totaled $37 million. 63 NRC Project Survey 2006, Question 4. See Appendix C.

52 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE 80 70 Percent of Projects Reporting 60 Control Group 50 Fast Track 40 30 20 10 0 >10 Million 5 Million to 1 Million to <1 Million 10 Million <5 Million Sales (Dollars) FIGURE 2-11 Distribution ofFigure size, Fast Track and control group. sales by 2-11.eps SOURCE: NRC Project Survey 2006, Question 4. See Appendix C. N=31 (Fast Track), n=47 (control group). NOTE: Data for the Fast Track group includes the six projects that were awarded both FT and PIIE. The PIIE group generated two awards with more than $10 million in cumulative revenues, and six with revenues of $5 million to 10 million, compared to one and two respectively for the control group. Given the very small numbers of awards involved, caution should be employed when drawing conclusions from these data. They are however suggestive. 2.2.4.5 Sales Distribution by Customer Sector Figure 2-13 shows the distribution of total sales achieved by SBIR Phase II projects to key customers as of August 2006.

SURVEY ANALYSIS 53 80 70 Percent of Projects Reporting 60 Control Group 50 Phase II Enhancement 40 30 20 10 0 >10 Million 5 Million to 1 Million to <1 Million 10 Million <5 Million Sales (Dollars) FIGURE 2-12 Distribution of sales by size, Phase II Enhancement and control group. Figure 2-12.eps SOURCE: NRC Project Survey 2006, Question 4. See Appendix C. N=43 (PIIE), n=47 (control group). NOTE: Data for the PIIE group includes the six projects that were awarded both FT and PIIE. Export markets 100 Other 90 Percent of Respondents State or local 80 governments 70 Other agencies 60 50 NASA 40 Prime contractors 30 for DoD 20 DoD 10 Domestic private 0 sector Fast Track Phase II Control Group Enhancement FIGURE 2-13 Distribution of Figure 2-10.eps III sales going to each sales: Percent of Phase sector. and also SOURCE: NRC Project Survey 2006, Question 4. See Appendix C. Figure 2-13.eps

54 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE Customers purchasing the result of SBIR activity in Phase III included the private sector, export markets, DoD, DoD Primes64, NASA, other federal agencies, and other public sector such as state and local governments. The information on the distribution of sales to these customers relates to both the achievement of private-sector commercialization and to meeting agency R&D needs in Phase III. “Other” customers identified included educational, medical, and research institutions.65 2.2.4.6 Expected vs. Realized Sales It is interesting to compare the reported sales for Fast Track in the current survey to the expected sales for Fast Track reported in the first NRC study.66 • In the first survey, Fast Track awardees responded that they expected average sales in 2001 (four years after the Phase II award) to be in excess of $8 million.67 • All current Fast Track respondents have had at least four years since their Phase II award, some as many as nine years,68 but the average reported sales is only slightly more than $1 million.69 This suggests that when asked to indicate expected sales, Fast Track companies (and probably all SBIR companies) tend to take a highly optimistic view. 64 The category “DoD Primes” was not used in the 1991 GAO survey. It was added in 1999 for the NRC Fast Track study to differentiate between sales made directly to DoD and those made to private contractors largely for incorporation into products ultimately sold to DoD. The term is used in gathering commercialization data on the DoD SBIR submission site. In 1991, these sales may have been categorized as private sector or DoD. 65 See NRC Project Survey 2006, Question 4, in Appendix C. The high percentage of sales to the export market for the control group merits explanation. The survey response, which reported the largest company sales ($35 million), was a control group award, which reported 65 percent of its sales to the export market. The sales from this award were more than double the next highest reported sales for any award. The export share of the sales from this award amounted to $22.7 million of the $29 million in export sales reported by the control group. 66 See National Research Council, The Small Business Innovation Research Program: An Assessment of the Department of Defense Fast Track Initiative, Charles W. Wessner, ed., Washington, DC: National Academy Press, 2000. 67 See Figure 15 in Peter Cahill, “Fast Track: Is It Speeding Commercialization of the Department of Defense Small Business Innovation Research Projects?” in National Research Council, The Small Business Innovation Research Program: An Assessment of the Department of Defense Fast Track Initiative, op. cit. 68 Only 10 of the 45 Fast Track awards that responded in 1999 responded to this survey. Many of the non-responding firms no longer exist. Some firms, which were apparently successful, were acquired by larger firms that did not respond to the survey. 1999 survey results are reported in National Research Council, The Small Business Innovation Research Program: An Assessment of the Department of Defense Fast Track Initiative, op. cit. 69 NRC Project Survey 2006, Question 4. See Appendix C.

SURVEY ANALYSIS 55 1,800 Other sales Sale of product, 1,600 processes and services Sales (Thousands of Dollars) 1,400 1,200 1,000 800 600 400 200 0 Fast Track Phase II Control Group Enhancement FIGURE 2-14 Near-term expected sales: Expected sales through end of 2007. Figure 2-14.eps SOURCE: NRC Project Survey 2006, Question 9. See Appendix C. 2.2.4.7 Projected Sales Projects that reported sales or reported that they expected their first sale to occur before the end of 2007 were asked to estimate the amount of sales they expected between the completion of the survey and the end of 2007. (See Figure 2-12.) Other sales were defined as including rights to technology and sale of spin-off companies. Based on the survey conducted for the 2008 NRC SBIR study, it is not unreasonable to project eventual average sales of in excess of one and a half million dollars per DoD SBIR project. To expect the above levels of average additional sales in the next year and a half is optimistic. Such optimism, however, is normal among SBIR participants. Optimism about projected SBIR sales also appeared in the 1991 GAO study and was found again in the 1999 NRC study.70 2.2.4.8 Initial Public Stock Offerings (IPO) Three surveyed firms made IPO, which they attribute to the SBIR program. One of these firms made their IPO in 1996, prior to the Fast Track and Phase II Enhancement initiative. That firm completed surveys on two subsequent Phase II Enhancement awards. Another firm had an IPO in 2000, which was after the control group award—that is, an award that is selected for 70 See National Research Council, The Small Business Innovation Research Program: An Assessment of the Department of Defense Fast Track Initiative, op. cit.

56 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE analysis and neither a Fast Track nor a Phase II Enhancement award—and before the Phase II Enhancement award on which they completed surveys. The third firm, which had a 2005 IPO, completed a survey on a control group award. Each of these three firms had over a dozen Phase II awards before its IPO.71 2.2.4.9 Skew in Outcomes SBIR sales are typically dominated by a few awards that produce most of the sales. Sixty-four percent of the total sales estimated between 2006 and 2007 were estimated for nine awards. These nine highest estimates (all at least $10 million) were in the Phase II Enhancement or the Control group.72 These high estimates seemed credible in that four of five Phase II Enhancement projections and two of four of the control group highest estimates had reported a high level of completed sales. Firms were asked about the basis for their estimates with choices ranging from market research, consultants, educated guesses, and past experience to ongoing negotiations, existing contracts and projection from current sales. All of the high estimated sales were based on existing contracts or projections from current sales.73 2.2.5 Impact of Funding Gaps Begun in 1996, Fast Track was an initiative of the 1995 DoD Process Action Team on the SBIR Program. Fast Track was one of two reforms that affected the funding gap between Phase I and Phase II. The requirement for a Fast Track contractor to submit his Phase II proposal during Phase I and for the agency to expedite evaluation and award of Fast Track proposals was designed to reduce the Fast Track funding gap between the two phases. The second reform affecting the gap was the establishment of a new standard of six months for the average time interval between receipt of SBIR proposal and award in Phase II. Long gaps in funding make it difficult for a small company to keep its research team together. Studies have shown that delay in time to market decreases the value of an innovation. 74 The funding gap is addressed on three charts. Figure 2-14 compares the percentage of Fast Track firms that experienced a gap with those from the Control Group. Figure 2-13 shows the magnitude of the gap, and Figure 2-15 shows the number of firms experiencing a gap and how the firms dealt with the gap. 71 NRC Firm Survey 2006, Question 12. Cross-reference with Question 4. See Appendix B. 72 See NRC Project Survey 2006, Question 9. See Appendix C. 73 Ibid. Individual responses to Question 9a were cross-referenced to responses to Question 9c. 74 Innovation speed can confer a significant competitive advantage to a firm. See H. Sonnenberg, Balancing Speed and Quality in Product Innovation, Canadian Business Review, 17(3):19-22, 1993. Also see Gideon D. Markman , Peter T. Gianiodis, Phillip H. Phan, and David B. Balkin, “Innovation Speed: Transferring University Technology to Market,” Research Policy, 34(7):1058- 1075, September 2005.

SURVEY ANALYSIS 57 BOX 2-2 Funding the Gap Between Phase I and Phase II Normally there are three components to the gap between Phase I and II of an SBIR award. These are: • The time required by the firm to prepare and submit a proposal; • The time needed by agencies or services to evaluate and select proposals; and • The period needed for the federal contracting process, which includes the time for the firm to provide additional information and certifications, DCAA audit of the firm, contract negotiations and, finally, the award. During contracting process, the Navy provides bridge funding for up to six months and the Army for four months. The Air Force funds all Phase I for an additional three months. The survey responses suggest that the Fast Track Program has had an impact in reducing the percentage of firms experiencing a delay between Phase I and Phase II. 2.2.5.1 The Size of the Gap Figure 2-16 displays the average funding gap (in months) reported for each group, for those projects that reported experiencing a funding gap. For Fast Track, only 22 of the 64 respondents experienced a gap75; thus, the average gap for all Fast Track respondents was only 1.5 months; however, the number reporting no gap has decreased and the average gap has increased in recent years.76 This coincides with a general relaxation by DoD components in how soon the Phase II proposal must be submitted. When Fast Track originated in 1996, proposals for Fast Track Phase II had to be submitted within five months of the Phase I award. The general deadline for Fast Track proposals is now six months after award of Phase I, and Army, CBD, DARPA, and OSD allow even more time. For the Control group, 33 projects reported no gap in funding, resulting in an average gap for all Control group respondents of six months.77 Responses to the gap by firms in each of the three groups are shown in Figure 2-17. 75 See NRC Project Survey 2006, Question 31, in Appendix C. 76 Information from the DoD Fast Track Database. 77 See NRC Project Survey 2006, Question 31, in Appendix C.

58 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE 80 Percent of All Projects Reporting Delays 70 Between Phase I and Phase II 60 50 40 30 20 10 0 Control Group Fast Track FIGURE 2-15 Funding gap between Phase I and Phase II: Percentage of firms experiencing delays. Figure 2-15.eps SOURCE: NRC Project Survey 2006, Question 31. See Appendix C. N=64 (FT) and n=107 (CG). 9.0 Average gap of those that had a gap 8.0 Average gap of all projects 7.0 Number of Months 6.0 5.0 4.0 3.0 2.0 1.0 0.0 Fast Track Phase II Control Group Enhancement Figure 2-16.eps FIGURE 2-16 Funding gap between Phase I and Phase II: Average length of funding gap in months. SOURCE: NRC Project Survey 2006, Question 31. See Appendix C.

SURVEY ANALYSIS 59 70 Fast Track 60 Phase II Enhancement Percent of Respondents 50 Control Group 40 30 20 10 0 Stopped Continued Continued Received Company Had No Work Work at Work at Bridge Ceased All Gap Reduced Same or Funding Operations Pace Greater Pace FIGURE 2-17 How firm copedFigure 2-17.eps with funding gap. SOURCE: NRC Project Survey 2006, Question 33. See Appendix C. As shown in Figure 2-14, only 12 Fast Track projects (19 percent) reported that the funding gap caused them to stop work. This compares to 40 percent for the control group. The significant number of projects still experiencing a gap, and the number stopping work due to the gap, may indicate that the agency should consider increasing the duration of gap funding. 2.2.5.2 Limits to Reducing the Gap Agencies typically create a gap between submission of the Phase II proposal and the award of the Phase II contract by fixing the times when a company can submit a Phase II proposal.78 Some of the gap, however, is due to the time the firm takes to submit the proposal. A question in the project survey asked for the date that the Phase I award period ended and the date the Phase II proposal was submitted.79 Almost half of the respondents left the dates blank. Only 33 percent of the 78 The DoD components usually control when the company can submit a Phase II proposal. SDIO/BMDO was the only component that had an open submission for Phase II awards and allowed companies to submit for a Phase II award at the company’s discretion. As such, any delay that occurred between Phase I and the submission of a Phase II proposal was driven by the company and not the government. 79 NRC Project Survey 2006, Question 32. See Appendix C.

60 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE Fast Track sample could identify these dates. Both the Control Group (68 percent) and the Phase II Enhancement (64 percent) samples were more responsive. Eighteen percent of the respondents claimed to have submitted their Phase II proposals the same day as they completed Phase I. Thirty-three percent reported that they had submitted their Phase II proposals prior to completion of Phase I work. Phase II proposals reported to be submitted prior to the end of Phase I work ranged from an average of 68 days early for the Control group to 94 days early for Fast Track. (Some of these were probably reporting errors.)80 Air Force Phase I awards last for nine months, but require a report at the sixth month. The other agencies provide six-month Phase I awards with the report due at the end of the 6th months. Proposal evaluation cannot be accomplished until the Phase I report has been received. In general a proposal submission prior to the end of Phase I will not speed the Phase II decision. Half of the proposals were reported to have been submitted after Phase I was completed. This self imposed funding gap appeared to range from 197 days for Phase II-Enhancement proposals to 232 days for the Control group. This part of the funding gap is largely under the firm’s control. The agency cannot begin the evaluation and award process until the proposal is submitted.81 2.2.6 Additional Developmental Funding SBIR does not provide federal funds to commercialize a product.82 And many projects require further development, testing, standardization, producibility, engineering, Beta testing of software, and similar activities after Phase II. Activities related to market research, trade show attendance, advertising and sales are not allowable costs under SBIR. Thus additional funding may be an important component of commercialization, and its presence is therefore a good indicator that commercialization is under way. It is not unusual for an SBIR firm to use a Phase II developed prototype to attract investors or customers who place an order for the further development and delivery of a product. Some needed development activities may occur 80 Ibid. 81 It should be noted that some of the data that used to compute the delay is limited and, in some cases, may be flawed. Seven projects reported submission over one year after Phase I. Normally, only the Army would allow this much delay and none of these projects were Army awards. For example, the longest reported delay (5 years between Phase I completion and Phase II proposal) is probably erroneous in that the Phase I was reported as being completed four years prior to the topic year that was identified. 82 The purpose of Phase II Enhancement funding is to encourage the transition of SBIR research. Navy and Army focus on funding additional research and development, and the Air Force focuses on overcoming unforeseen technological barriers. All three services and MDA direct their enhancement programs to transition into acquisition programs.

SURVEY ANALYSIS 61 Personal and Other domestic private company investment company investment State and local Other private equity government + US venture capital university investment Non-SBIR federal Foreign investment funds 1,200,000 Average Investment (Dollars) 1,000,000 800,000 600,000 400,000 200,000 0 Fast Track Phase II Control Group Enhancement FIGURE 2-18 Average non-SBIR investments. Figure 2-18.eps SOURCE: NRC Project Survey 2006, Question 23. See Appendix C. concurrently with early sales, with the revenue used to upgrade later versions of the product: however, most additional investment must happen early in the sales cycle. The reported additional non-SBIR development funding to date is shown in Figure 2-18. 2.2.6.1 Non-SBIR Investments Most private additional developmental funding is invested in anticipation of a return on investment. Thus, the additional investment is a leading indicator of ultimate commercial sales.83 Similarly early additional funding by non-SBIR federal funds is generally an indicator of Service interest in the technology of the award and is a leading indicator of sales to DoD or its primes. Greater investment in Fast Track projects and Phase II Enhancement projects than in the control group ought to be a foregone conclusion at this point in the projects. Fast Track projects had to bring third-party money to the table and have it invested early in Phase II. Phase II Enhancement projects brought in third-party money and matching funds. The control group projects did not. It would appear that in addition to occurring earlier, the ultimate investment in 83 See Peter Cahill, “Fast Track: Is it Speeding Commercialization of Department of Defense Small Business Innovation Research Projects?” op. cit., p. 68.

62 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE Personal and Other private equity company investment US venture capital State and local government + Non-SBIR federal university investment funds Foreign investment SBIR federal funds Other domestic private company investment 2,500,000 Investment per Phase II Award 2,000,000 1,500,000 1,000,000 500,000 0 Fast Track Phase II Control Group Enhancement FIGURE 2-19 Average investment including SBIR. SOURCE: NRC Project Survey 2006,2-19.eps See Appendix C. Figure Question 23. Fast Track Phase II and Phase II Enhancement Awards will be larger than in projects that are neither.84 To get a complete picture of the funding that goes into the research, development, and transition into commercialization, we need to consider the SBIR funding for the award as well. Fast Track Fast Track awards reported an average Phase II SBIR award that is $100,000 higher than the control group.85 84 The 2005 survey for the NRC study found that 920 responding DoD projects awarded Phase II from 1992 to 2001 averaged $795,734 in additional non-SBIR developmental funding, a larger average than the control group, but below that of Fast Track in the current survey (NRC Project Survey 2006, Question 23, in Appendix C). The average responding project in the 2005 survey had three more years in this study to acquire additional funding. The 2005 survey results can be found in National Research Council, An Assessment of the SBIR Program at the Department of Defense, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2009, Appendix B. 85 Phase II Enhancement Awards averaged $1,333,061 in SBIR funding. Fast Track awards averaged $890,374. Control group awards averaged $794,292. The apparent advantage of Fast Track over the control group is an illusion. Fourteen of the Fast Track awards received subsequent Phase II

SURVEY ANALYSIS 63 Phase II Enhancement As Phase II Enhancement can obtain up to a half million dollars in additional SBIR money by obtaining appropriate matching funds, Figure 2-16 adds the SBIR funding to the funding shown in Figure 2-15. The average SBIR funding for the responding awards was $500,000 higher for Phase II Enhancement awards than for Control Group. Returning to the comparison of additional funding between the Phase II Enhancement and the control group on Figure 2-15, we can examine the totals without the input of Phase II Enhancement by third parties. As discussed above, Phase II Enhancement awards received, on average, $400,000 more in awards than the control group. Since Phase II Enhancement provides a dollar for dollar match of SBIR funding to third-party funding, it can be presumed that at least $400,000 of the additional average funding for Phase II Enhancement (displayed in Figure 2-15) was the funding that was executed during the additional year of Phase II. This $400,000 is less than 40 percent of the difference in average additional funding of Phase II Enhancement compared to the control group.86 2.2.6.2 Venture Capital Impacts Venture capital investment is often suggested as an alternative to SBIR.87 However, less than two percent of the awards reported any venture funding. Four Fast Track, one Control Group, and three Phase II Enhancement awards reported venture funding.88 As can be seen from Figure 2-15, the impact of venture capital on available funding was marginal. The average venture capital investment on these few projects was less than $1,000,000. The number of projects reporting VC funding was too small to allow substantive comparisons between groups. Enhancement, thus adding to their SBIR funding. When these 14 are removed from the Fast Track averages, the average SBIR for the remaining 50 Fast Track awards ($797,267) is comparable to that of the control group. 86 A similar direct comparison for Fast Track is not simple. Fast Track awards receive no additional SBIR funding compared to control group. The required third-party funding may be a dollar for dollar match to the Phase II award, or a one dollar to four SBIR dollars match dependent on whether this is the firm’s first SBIR Phase II. Since almost two-thirds of Fast Track respondents said this was their first Phase II, the average third-party match would be 1/3 x $800k +2/3 x $200k = $400k; however, almost one fourth of the Fast Track also received Phase II Enhancements, raising this average for third-party funding to about one half million dollars. Continuing this logic, one fifth of the Phase II Enhancement respondents were also Fast Track awards, hence the full average third- party funding for Phase II Enhancement awards was also almost a half million dollars. 87 The limitation on majority VC owned firms was implemented only beginning in 2002. For a review of the impact of the SBA rule on the SBIR program at NIH, see, National Research Council, Venture Capital Funding and the NIH SBIR Program, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2009. 88 One of the awards reporting venture capital was both a Fast Track and a Phase II Enhancement.

64 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE 2.2.7 Knowledge Effects: Patents and Scientific Publications The congressional objectives for the SBIR program include additions to the nation’s scientific and technical knowledge. Patents, copyrights, and scientific publications are all evidence that this objective is being addressed by the SBIR program. The number of patents and copyrights applied for and issued is a measure of the intellectual property being generated. Since most scientific journals are refereed, the number of publications submitted and published measures to some degree the scientific merit of the SBIR. The numbers of patents, copyrights, trademarks, and scientific publications to date were measured by the survey.89 (See Figure 2-20.) Fast Track A concern expressed by some opponents of Fast Track is that, in order to attract third-party investors, there can be no true research in a Fast Track SBIR. The project must be much further down the development path. They contend that to obtain third-party financing, innovation must be complete or nearly complete and the SBIR merely serves to validate. If this is the case, one might expect a lower level of patent and copyright activity for Fast Track. 1.80 Average Number per Phase II Award 1.60 Fast Track 1.40 Phase II Enhancement 1.20 Control Group 1.00 0.80 0.60 0.40 0.20 0.00 Patents Copyrights Trademarks Publications Received Received Received Published FIGURE 2-20 Disclosure activity: Average number of patents, copyrights, Figure 2-20.eps trademarks, and scientific publications received for the technology developed as a result of project. SOURCE: NRC Project Survey 2006, Question 18. See Appendix C. 89 NRC Project Survey 2006, Question 18. See Appendix C.

SURVEY ANALYSIS 65 The data reported in the figure above indicate that Fast Track and Control Group projects tend to patent at much the same rate. Both Fast Track and Control Group projects reported 0.42 patents per Phase II award. Eighty six percent of FT projects reported zero patents, compared with 77 percent of Control Group projects.90 Phase II Enhancement The average patents reported received per phase II award varies from 0.30 for Phase II Enhancement to 0.42 for the control group.91 Once again, only a few firms reported receiving patents: 13 percent of PIIE firms and 23 percent of the Control Group.92 It therefore appears that patenting is a somewhat less important tool for protecting intellectual property among PIIE firms. It is also worth noting that, as in other area, skew is important: one firm accounted for a fifth of all Control group patents (9). 2.2.7.1 Scientific Publications The largest disclosure of research results appears to be in scientific publications. Three hundred and sixty-seven of the 380 papers reported as submitted to scientific publications were published. The control group included one award that reported 50 publications and another that reported 12 publications. Five awards (three Phase II Enhancement and one each Fast Track and Control Group) reported ten publications apiece. Most awards (74 of 107) that reported publication, had three or fewer papers published in scientific journals. Overall, there was little difference in the reported intellectual property/disclosure results for the vast majority of awards in each group. Slight differences in averages were due to single high performers in each category. 2.2.8 Impact of the Use of Fast Track and Phase II Enhancement on Responding Awards Respondents were asked several questions that were answered only by Fast Track awards or only by Phase II Enhancement Awards. The first question dealt with the source of matching funds in the Fast Track proposal or in the Phase II Enhancement proposal. (See Figure 2-21.) Since respondents could identify more than one source, the percentage totals of responses for the sources in Figure 2-18 add up to more than 100 percent. The number of respondents indicating their own firm as the source of matching funds in the proposal is quite unusual. Nothing prevents a firm from investing in its own SBIR research; however, DoD criteria for Fast Track 90 N=56 and n=107 respectively. 91 One control group Phase II award received 11 patents, which accounts for its advantage in averages. Ibid. 92 N=77 and n=107 respectively.

66 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE 100 Own firm 90 Federal agency 80 Other firm Percent of Respondents 70 Angel investor or other private funding source 60 Venture capital 50 Other (mostly state) 40 30 20 10 0 Fast Track Phase II Enhancement FIGURE 2-21 Proposal matching funds: Sources of matching or co-investment Figure 2-21.eps funding included in the proposal. SOURCE: NRC Project Survey 2006, Question 25. See Appendix C. specifically require that the matching funds in the proposal have to come from an outside third-party source. Fast Track The requirement to find third-party financing for Phase II and to obtain finalized agreements within 150 days of the award of Phase I has been a deterrent to participation in Fast Track. One-fourth of the Fast Track respondents reported that it took six months or more to obtain an agreement for third-party financing, indicating that they had to begin the search for financing prior to obtaining the Phase I award. The reported average time to obtain third- party financing for Fast Track was 3.5 months. The longest reported time was 12 months. Phase II Enhancement For Phase II Enhancement, the individual component programs have differing criteria for outside investors, some specifying only acquisition programs qualify and others allowing private sector. The tilt toward acquisition programs for Phase II Enhancement can be seen in the high percentage of federal agency shown as the source of matching funds for that program.

SURVEY ANALYSIS 67 For Phase II Enhancement, the average time to find and obtain an agreement for third-party financing was 4.1 months.93 Eleven percent of the respondents reported times of 12 or more months. Phase II Enhancement proposals are submitted on the schedule required by the individual components, but never prior to the completion of the first year of Phase II. Hence there is much more time available to locate and negotiate with the third-party source.94 The survey asked for comparison of the performance of Phase II Fast Track Awards and awards that also received Phase II Enhancement to performance of a standard Phase II award. As shown in Figure 2-22, the surveyed awards were judged to perform better, worse, or the same as a standard Phase II. Note that the relevant comparison in each case is between the control group and either Fast Track or PIIE. There is no relevant comparison between the latter groups. Note also that in the case of each variable, both Fast Track and PIIE were reported to compare favorably with the control group. 2.2.8.1 Speed to Market Over 50 percent of Fast Track and Phase II Enhancement respondents contended that the initial products reached the market faster than products of the average standard Phase II award. For Phase II Enhancement, which includes an extra year of Phase II development, seven percent stated that the time for the initial product to reach the market was longer than would have occurred absent the Phase II Enhancement.95 2.2.8.2 Sales to Date A substantial percentage of Fast Track (38 percent) and Phase II Enhancement (48 percent) projects claimed that sales to date were greater than would have occurred with a standard Phase II award. Similarly, both Fast Track (52 percent) and Phase II Enhancement (47 percent) projects claimed that investment to date was greater than would have occurred with a standard Phase II award. Seven percent (four awards) reported smaller investment to date than would have occurred absent the Phase II Enhancement.96 2.2.8.3 Potential Sales Potential sales were expected to be greater for Fast Track (45 percent) and for Phase II Enhancement (69 percent) projects than these groups would have expected to occur with a standard Phase II award. Investment in Phase II Enhancement may ease the transition into a DoD procurement. Seventy-five percent of the awards that reported such investment also reported improved transition. Similarly, 72 percent of the Fast Track projects that reported 93 NRC Project Survey 2006, Question 26. See Appendix C. 94 Ibid. 95 NRC Project Survey 2006, Question 27 and 29. See Appendix C. 96 See NRC Project Survey 2006, Question 27 and 29. See Appendix C.

68 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE Worse Same Improved 100 Percent of Respondents 90 80 70 60 50 40 30 20 10 0 Enhancement Enhancement Enhancement Enhancement Enhancement Phase II Phase II Phase II Phase II Phase II Fast Track Fast Track Fast Track Fast Track Fast Track Time to Sales of Investment Potential Transition Reach Market Product Sales into DoD Procurement FIGURE 2-22 Comparisons to standard SBIR awards: Impact Fast Track or Figure 2-22.eps Phase II Enhancement had as compared to a standard Phase II proposal. SOURCE: NRC Project Survey 2006, Questions 27 and 29. See Appendix C. investment by a federal agency as their source of matching funds also reported improved transition.97 2.2.8.4 Satisfaction with Fast Track and the Phase II Enhancement Figure 2-23 shows the bottom line satisfaction by the firms with both the Fast Track program and the Phase II Enhancement program. Respondents were asked “In retrospect, knowing the outcome, are you satisfied with your decision to use” the program (Fast Track or Phase II Enhancement) “on this Phase II”? High percentages of both groups expressed satisfaction with their decision to use those programs (90 percent of the Fast Track group and 95 percent Phase II Enhancement group). These questions about both Fast Track and Phase II Enhancement were only asked of participants in those programs. The next series of questions elicited opinions from firms who had not or would not in the future participate in one of these programs. 97 Ibid.

SURVEY ANALYSIS 69 Dissatisfied Satisfied 100 90 Percent of Respondents 80 70 60 50 40 30 20 10 0 Fast Track Phase II Enhancement FIGURE 2-23 Customer satisfaction with programs: Decision to use Fast Figure 2-23.eps Track or Phase II Enhancement. SOURCE: NRC Project Survey 2006, Questions 28 and 30. See Appendix C. 2.2.9 Reasons for Not Submitting Fast Track and Phase II Enhancement Proposals The following information was determined from questions on the firm survey.98 Fast Track Firms were asked if they had ever submitted a Fast Track proposal.99 Thirty-eight percent of those responding had submitted a Fast Track proposal. Of those who had submitted a Fast Track proposal, 38 percent said that they were likely to do so in the future. However, firms did not express strong opinions in responding as to whether they would submit future Fast Track proposals. Nineteen percent of all respondents said yes; one-third said no; and the rest did not answer this question. Ten percent of the firms that had submitted for Fast Track in the past said that they would not do so in the future. Firms that had not submitted, or were not likely to submit a Fast Track proposal, were asked to identify why. Their responses are shown in Figure 2-24. 98 The project survey is not the basis for categorizing a firm as belonging to Fast Track, Phase II Enhancement, or control groups. This is because a given firm (1) could have been surveyed in more than one award category and/or (2) could have award(s) in another category but not sampled in that category. 99 NRC Firm Survey 2006, Question 15. See Appendix B.

70 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE Difficulty obtaining timely third-party funding Insufficient benefits provided by Fast Track Other Concern about dilution of equity / control Bureaucratic requirements of Fast Track Government sponsor opposed Fast Track Dissatisfied with prior Fast Track experience 0 10 20 30 40 50 60 70 80 Percent of Respondents FIGURE 2-24 Reasons for Figure 2-24.epsTrack proposals not submitting Fast SOURCE: NRC Firm Survey 2006, Question 16. See Appendix B. Difficulty in finding third-party funding in time to submit a Fast Track application100 was by far the primary reason for not submitting or planning to submit. Firms that responded “other” offered a variety of reasons. The most frequent response (4 of 29 who responded other) was lack of knowledge or understanding of Fast Track. The second most frequent response (3 of 29) was that the firm was no longer eligible for SBIR. Only one other response, dealing with lack of commercial maturity of the technology, was given by more than one firm. Phase II Enhancement The firms were asked the same questions about participation in Phase II Enhancement.101 Fifty-one percent of firms reported that they had submitted a Phase II Enhancement proposal. Of these, 61 percent reported that they were likely to do so in the future. Although many firms indicated that they were likely to submit future Phase II Enhancement proposals, again many expressed no opinion. Forty-six percent of all respondents said that they were likely to submit future Phase II Enhancement proposals; eight percent said that they were 100 Although Fast Track Phase II proposals are not due for six or more months (component dependent) after the Phase I award, Fast Track applications are due 150 days after the Phase I award. 101 NRC Firm Survey 2006, Question 17. See Appendix B.

SURVEY ANALYSIS 71 unlikely; and the rest did not answer this question. Six percent of the firms that had submitted for Phase II Enhancement in the past reported that they would not do so in the future. Firms that had never submitted for Phase II Enhancement or which said they would not do so in the future were asked to identify why. Their responses are shown in Figure 2-25. Because some components require that third-party funding be provided by acquisition programs, the response dealing with such funding was split to differentiate between private sector and acquisition program funding. Many respondents (26 of 38) who identified difficulty in obtaining acquisition program funding also identified difficulty in obtaining private sector funding. For firms reporting “other,” the most frequent response (over half of the 32 firms who responded “other”) was lack of knowledge or understanding of Phase II Enhancement. This may indicate a need to better publicize the program. The second most frequent response (5 of 32) was that the firm was no longer eligible for SBIR. No “other” response was given by more than one firm. 2.2.10 Dilution of Ownership Some SBIR contractors may be reluctant to seek third-party financing, particularly from venture capital companies, fearing that they will lose control of their firms. Others may welcome such cash infusions, preferring to have partial control over a potentially large firm to full control of a small one.102 Some in this latter group may intend to sell their interests as the firm gets large and roll the profits into starting a new firm. Those leery of equity funding are often heavily involved in advancing technology and less interested in production. They are afraid that outside investors would change the fundamental nature of the firm. The business expertise that venture capital insists on putting in place (if not already there) creates an environment likely to produce commercial success, but such success may not be the principal goal of the owners.103 Third-party investors may also be larger firms who want to merge with or partially or completely own the SBIR firm after the award. Finalized agreements and ongoing negotiations are shown in Figure 2-26. 102 Other sources of funding include family, angel investors, and state programs, though the role of these diverse sources changes as the firms evolves from the seed and early stages of development through to the later states. 103 For a qualitative study of entrepreneur perspectives on conceptualizing and starting successful ventures, see, F. G. Crane and J. Sohl, “Imperatives for Venture Success: Entrepreneurs Speak,” The International Journal of Entrepreneurship and Innovation, May 2004, pp. 99-106. For an overview of the growth, characteristics, and challenges of the venture capital industry in the United States, see Paul Alan Gompers and Joshua Lerner, The Venture Capital Cycle, 2nd Edition, Cambridge, MA: MIT Press, 2004.

72 REVISITING THE DEPARTMENT OF DEFENSE SBIR FAST TRACK INITIATIVE Difficulty obtaining third-party private-sector funding Difficulty obtaining DoD Acquisition Program funding Insufficient benefits provided by Phase II Enhancement Other Concern about dilution of equity / control Bureaucratic requirements of Phase II Enhancement Government sponsor opposed Phase II Enhancement Dissatisfied with prior Phase II Enhancement experience 0 10 20 30 40 50 60 Percent of Respondents FIGURE 2-25 Reasons for not submitting Phase II Enhancement proposal. Figure 2-25.eps SOURCE: NRC Firm Survey 2006, Question 18. See Appendix B. Merger negotiations 40 Partial sale negotiations 35 Percent of Phase II Reporting Sale negotiations 30 Finalized merger 25 Finalized partial sale Finalized sale 20 15 10 5 0 Fast Track Phase II Control Group Enhancement FIGURE 2-26 Dilution of firm ownership SOURCE: NRC Project SurveyFigure 2-26.eps See Appendix C. 2006, Question 12.

SURVEY ANALYSIS 73 Fast Track The figure indicates that overall FT firms were much more likely than control group firms to be engaged in equity-related activities. Given the need for matching funds this is of course not surprising. Figure 2-23 portrays the “cost” to firm owners of Fast Track firms. To obtain third-party funding, they may be limiting their personal share of the potential gain from their innovation, by selling a share of their firm. But they may also be increasing the ultimate gain from the innovation by infusing cash and business expertise at the critical point in development. Fourteen percent of the Fast Track awards have resulted in finalized agreements for sale of ownership, partial sale of ownership, or merger. Negotiations are ongoing for another 22 percent. Whether this is a cost or an opportunity is very much a personal evaluation. This compares with one percent of firms (one firm) reporting a finalized agreement, in this case for partial sale of the firm. Phase II Enhancement The impact of equity-related activities is lower for Phase II Enhancement firms, where only 6 percent report finalized activity in sale or merger. This is still considerably greater than for the control group.

Next: 3 Findings and Recommendations »
Revisiting the Department of Defense SBIR Fast Track Initiative Get This Book
×
Buy Paperback | $64.00 Buy Ebook | $49.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In October 1995, the Department of Defense launched a Fast Track initiative to attract new firms and encourage commercialization of Small Business Innovation Research (SBIR) funded technologies throughout the department. The goal of the Fast Track initiative is to help close the funding gap that can occur between Phase I and II of the SBIR program. The Fast Track initiative seeks to address the gap by providing expedited review and essentially continuous funding from Phase I to Phase II, as long as applying firms can demonstrate that they have obtained third-party financing for their technology. Another program initiative, Phase II Enhancement, was launched in 1999 to concentrate SBIR funds on those R&D projects most likely to result in viable new products that the Department of Defense and others will buy.

The current volume evaluates the two SBIR Program initiatives--Fast Track and Phase II Enhancement--and finds that both programs are effective. Ninety percent of Fast Track and 95 percent of Phase II Enhancement reported satisfaction with their decision. This book identifies the successes and remaining shortcomings of the programs, providing recommendations to address these issues.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!