as well as potentially irredeemable data loss; the fact that some firms and principal investigators (PIs) are no longer in place; and fading memories.

Award Numbers

While adding the annual awards numbers of the five agencies would seem to define the larger sample, the process was more complicated. Agency reports usually involve some estimating and anticipation of successful negotiation of selected proposals. Agencies rarely correct reports after the fact. Setting limitations on the number of projects to be surveyed from each firm required knowing how many awards each firm had received from all five agencies. Thus, the first step was to obtain all of the award databases from each agency and combine them into a single database. Defining the database was further complicated by variations in firm identification, location, phone numbers, and points of contact within individual agency databases. Ultimately, we determined that 4,085 firms had been awarded 11,214 Phase II awards (an average of 2.7 Phase II awards per firm) by the five agencies during the 1992-2001 time frame. Using the most recent awards, the firm information was updated to the most current contact information for each firm.

Sampling Approaches and Issues

The Phase II Survey used an array of sampling techniques to ensure adequate coverage of projects, to address a wide range both of outcomes and potential explanatory variables, and also to address the problem of skew. That is, a relatively small percentage of funded projects typically account for a large percentage of commercial impact in the field of advanced, high-risk technologies.

  • Random Samples. After integrating the 11,214 awards into a single database, a random sample of approximately 20 percent was sampled. Then a random sample of 20 percent was ensured for each year; e.g., 20 percent of the 1992 awards, of the 1993 awards, etc. Verifying the total sample one year at a time allowed improved ability to adapt to changes in the program over time, as otherwise the increased number of awards made in recent years might dominate the sample.

  • Random Sample by Agency. Surveyed awards were grouped by agency; additional respondents were randomly selected as required to ensure that at least 20 percent of each agency’s awards were included in the sample.

  • Firm Surveys. After the random selection, 100 percent of the Phase IIs that went to firms with only one or two awards were polled. These are the hardest firms to find for older awards. Address information is highly



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement