Annex J:
Template for Individual Agency Reports98

Introduction

Program history at the agency

Basic demographics of the agency program – number of awards, $ awarded, trends over past 20 years [drawing on program manager survey]

Organization – who runs what, under what offices, etc.

Short description – number of competitions annually, use of topics etc.

General methodology problems and approaches*

Comparability

Measurement

Bias

Multiple methodology approach, plus brief description of major instruments and approaches

Success stories box

Outcomes at the Agency

Commercialization Strategies and Outcomes

Difficulties of evaluating*

Research methodology

P2 survey data for the agency

Selected case study data for the agency

Agency-initiated analysis and data

Conclusions

Mission support

Short intro on agency differences, and procuring vs. non-procuring agencies*

Identification of specific agency as procuring/non-procuring

Research methodology

98  

Note: starred (*) items are expected to be identical across all agency reports.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 81
Annex J: Template for Individual Agency Reports98 Introduction Program history at the agency Basic demographics of the agency program – number of awards, $ awarded, trends over past 20 years [drawing on program manager survey] Organization – who runs what, under what offices, etc. Short description – number of competitions annually, use of topics etc. General methodology problems and approaches* Comparability Measurement Bias Multiple methodology approach, plus brief description of major instruments and approaches Success stories box Outcomes at the Agency Commercialization Strategies and Outcomes Difficulties of evaluating* Research methodology P2 survey data for the agency Selected case study data for the agency Agency-initiated analysis and data Conclusions Mission support Short intro on agency differences, and procuring vs. non-procuring agencies* Identification of specific agency as procuring/non-procuring Research methodology Note: starred (*) items are expected to be identical across all agency reports. 98 81

OCR for page 81
Agency interviews [within SBIR program and at more senior levels] Procuring agencies only: Agency contracts analysis (DoD, NASA, parts of DoE) Conclusions Small business support Problems of measuring additionality, and impact* Research methodology* If available, comparative impact study for specific agency (funded vs. unfounded applicants to SBIR) Basic demographics of agency support for small business Small business research $ as share of agency R&D spending; SBIR share of all agency R&D spending on small businesses Conclusions Knowledge base Largely indirect character of knowledge expansion in the commercial environment; other “knowledge base” issues; comparability issues* Research methodology* Patents and trademarks Licensing Citation analysis Equity sales [why is this here?] Partnerships Indirect path: indications, not measurement* Conclusions Program management Topic development and selection Sources for topic ideas Agency-driven vs. investigator driven approach to topics Topic decision-making 82

OCR for page 81
Outreach (before successful application) Agency outreach objectives (e.g. reaching low-application states and regions; educating very likely applicants.) Outreach programs that the agency runs or participates in (including brief description) Agency support for P1 applicants Agency outreach benchmarks and metrics Grant selection Description of selection processes for P1 and P2 Peer review panels – membership, selection, qualifications Fairness – any issues and changes in procedures to meet problems; (P2 winner and non-winner views from surveys) [make sure there is a “fairness” question in the P2 survey] Scoring procedures Role of program manager Re-submission procedures and outcomes Training (after successful application) Training programs for agency P1 and P2 grantees Benchmarks used to evaluate effectiveness Take-up rates and projections Constraints Phase 2+ program Description of agency’s P2+ program Use of matching funds Application and selection procedures Role of program manager Funding “gaps.” P1 – P2 gap. Average size, drivers. Options for addressing. After P2. Data on subsequent agency funding/take-up of technologies. Agency perceptions of its role. Bridge funding programs (after P2). Program description and objective Limitations (e.g. matching funding requirements) What is the average extent of the gap What is the primary cause of the gap 83

OCR for page 81
Does the agency think there is a problem Is the agency finding ways to address the problem Reporting requirements Reports submitted to the agency by SBIR winners Report utilization, and utility to the agency Evaluation and assessment Annual and intermittent agency evaluations of its SBIR program Operational benchmarks for the agency SBIR program Evaluators (internal and external) Annual evaluation and assessment budget, and funding sources Flexibility Program managers discretion (project selection, project size, program management) Program manager tools (e.g. shifting funding from one sub-agency to another; adjusting scoring criteria; changing the size of awards) Agency manager perceptions of constraints Size Formal and effective limits on size and duration of awards Distribution of P1 and P2 awards within those limits Availability of additional funding. Amounts and distributions. Online capabilities and plans Components of the grant application, management, and reporting process NOT yet online Plans and timelines for putting these remaining elements online Barriers obstructing implementation of these plans Administrative funding Funding of program administration Control of administration budget Administration budget as percent of agency SBIR funding Evaluation and assessment funding Recommendations Appendices 84

OCR for page 81
Appendix A. Agency data about its program (incl. demographic data) Appendix B. Agency P2 survey results Appendix C. Agency P1 survey results Appendix D. Selected case studies Appendix E. Selected agency reports and data 85