2. An Overview of the Study Process

Following its approval of the broad study parameters of the study in October 2002, the Committee set out an overall roadmap to guide the research process. Tasks included are the development a set of operational definitions, the identification of detailed metrics, the review existing data sources, and the development of primary research methodologies. Closely interrelated, these tasks will be addressed iteratively. (These iterative tasks are represented in the box within Figure 1.) Following completion of field research, the Committee will conduct its analysis and assessment and will issue its findings and recommendations.

Figure 1



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 9
2. An Overview of the Study Process Following its approval of the broad study parameters of the study in October 2002, the Committee set out an overall roadmap to guide the research process. Tasks included are the development a set of operational definitions, the identification of detailed metrics, the review existing data sources, and the development of primary research methodologies. Closely interrelated, these tasks will be addressed iteratively. (These iterative tasks are represented in the box within Figure 1.) Following completion of field research, the Committee will conduct its analysis and assessment and will issue its findings and recommendations. Figure 1 NRC SBIR Study: The Logic of Analysis Congressional requirements Project Objectives Operational definitions Potential metrics Existing data sources Iterative Primary processes research Assessm ent and analysis Conclusions and recomm endations 9

OCR for page 9
The elements of this multi-step process are detailed below: 1. Agree on initial guidelines. These initial guidelines are based on the legislation, the Memorandum of Understanding, and contracts. 2. Clarify objectives. What central questions must the study answer? What other interesting but optional questions should be addressed? What questions will specifically not be considered? This is discussed further in Section 3 of this chapter. 3. Develop operational definitions: For example, while Congress has mandated that the study address the extent to which SBIR supports the agencies’ missions, the Committee needs to develop operational definitions of “support” and “agency mission,” in collaboration with agency managers responsible for program operations. This is a necessary step before developing the relevant metrics. This is discussed further in Section 4 of this chapter. 4. Identify metrics for addressing study objectives. The Committee will determine extent of commercialization fostered by SBIR—measured in terms of products procured by agencies, commercial sales, licensing revenue, or other metrics. This is discussed further in Section 5 of this chapter. 5. Identify data sources. Implementation of agreed metrics requires data. A wide mix of data sources will be used, so the availability of existing data and the feasibility of collecting needed data by different methods will also condition the selection of metrics, and the choice of study methods. The existence or absence of specific methodologies and data sets will undoubtedly lead to the modification, adoption, or elimination of specific metrics and methods. This is discussed further in Section 6 of this chapter. 6. Develop primary research methodologies. The study’s primary research components will include interviews, surveys, and case studies to supplement existing data. Control groups and counterfactual approaches will be used where feasible and appropriate to isolate the effects of the SBIR program. Other evaluation methods may also be used on a limited basis as needed to address questions not effectively addressed by the principal methods. This is discussed further in Section 7 of this report. 7. Complete Phase I. Phase I of the NRC study will be formally completed once a set of methodologies is developed and documented, is approved by the Committee, and passes successfully through the Academy’s peer review process. 8. Implement the research program (NRC Study Phase II). The variety of tasks involved in implementing the research program is previewed in Annex I of this report. 9. Prepare agency-specific reports. Results from the research program will be presented in five agency- specific reports—one for each of the agencies. Where appropriate, agency-specific findings and recommendations will be formulated by the relevant study subcommittee for review and approval by the full Committee. 10. Prepare overview report. A separate summary report, buttressed by the relevant commissioned work and bringing together the findings of the individual agency reports, along with general recommendations, will be produced for distribution. This final report will also draw out, as appropriate, the contrasts and similarities among the agencies in the way they administer SBIR. It will follow the approval procedure outlined above. 11. Organize public meetings to review and discuss findings. Following report review, findings and recommendations will be presented publicly for information, review, and comment. 12. Submit reports to Congress. 13. Disseminate findings broadly. 10