Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Appendixes
An Assessment of NASA 's Pioneering Revolutionary Technology Program information, and (4) new and wicle-ranging concepts. Another important criterion, which should go without saying, is that the concept floes not violate the laws of ohYsics (Troutman, 2002~. While the technology pull criterion attempts to relate RAS(: concepts to the NASA technology portfolio, it could be strengthened by overtly specifying its - -- r--~--- direct relationship to the NASA technology database. No criteria for the reviser! project had been presented to the pane! by time of publication of this report. Recommendation: RASC should improve its relationship to the NASA technology database. Better integration is necessary to ensure an actual connection exists between RASC and the NASA technology database.. RASC studies tend to be concept studies within a certain NASA enterprise area. The pane! notes that although efforts have been made to distinguish current studies from past work, further effort is necessary. The pane! suggests that RASC should emphasize work that crosses enterprise lines to strengthen the idea that it is wicle ranging. Examples of such work might include unclerstanding the synergy between human and robotic missions, NASA-wide future communication needs, and the synergy between high-speed aeronautics and launch vehicle technology. The pane} felt that NASA shouict revisit its 20+ years timeframe since in some cases this criterion might have unintentionally directed ideas too far into the future. Almost all projections tend to underestimate how soon a project will begin. If this change is adopted, it may address some minor criticism about RASC relevance. The pane! notes that the 20-year time frame was subsequently dropped during program reformulation in early 2003. Recommendation: RASC should reconsider the criteria it uses to select studies. giving more weight to cross-enterprise studies. Care should be taken with the long- term focus so as not to make RASC projects so far off that they become irrelevant. r - -a ~ NASA Institute for Advanced Concepts Element NIAC's purpose is to be an independent source of revolutionary aeronautical and space concepts that could dramatically impact how NASA develops ant! conducts its mission (NIAC, 2001~. Its ultimate goal is to infuse NIAC-funcled concepts into future NASA plans and programs. NIAC is operated by the Universities Space Research Association (USRA) as a virtual institute using Internet technology to distribute its solicitations, receive proposals ant! reports, and review proposed projects. In 2002, NIAC's fifth year of existence, funcling was provident at a level of $4 million per year. NIAC solicits proposals only from non-NASA sources and strives to use non-NASA reviewers to maintain inclepenclence. The panel was impressed by the diversity and experience of the reviewers as expressed in general statistics.6 If one agrees with the purpose and premise of NIAC namely, to be a technology incubator then NIAC has had some success infusing interesting new ideas into NASA. 6 Reviewer names and specific affiliations are held in confidence by USRA and were unavailable to the review panel. Information on general affiliations and experience was provided to the panel. 100
A Statement of Task This project will produce biennial assessments of the programs within NASA's Aerospace Technology Enterprise the Pioneering Revolutionary Technology (PRT) program, the Aviation program, and the Space Transportation program. The first review in the series will be of the PRT program group; other reviews will follow in subsequent years. Programs within the PRT group are the Enabling Concepts and Technologies (ECT) program, the Computing, Information, and Communications Technology (CICT) program, and the Engineering of Complex Systems (ECS) program. The committee will assess the overall scientific and technical quality of the PRT program elements. These assessments will include findings and recommenda- tions related to the quality and appropriateness of NASA's internal and collaborative research, develop- ment, and analysis. While its primary objective is to conduct peer assessments that provide scientific and technical advice, the committee may offer program- matic advice when it follows naturally from technical considerations or is requested by the NASA Associate Administrator for Aerospace Technology. The committee will be assisted by three NRC pan- els that each focus on one of the three elements of the PRT program listed above. Each panel will assess the scientific and technical quality of selected programs in the element under its purview. Each panel will provide input to the committee's report via internal working draft reports to the committee. Panels will meet twice during the study to receive technical presentations 101 about the projects under review by their group and for- mulate final findings and recommendations. Panel members will also make site visits as deemed neces- sary in formulating the assessment. Portions of each meeting will be highly interactive with NASA person- nel. After completion of its deliberations and investi- gation, the panel will report to the committee on its findings via internal privileged correspondence and working papers. The main committee will meet twice during the review: once to plan the review process, meet with the panel members, and discuss the charge to the commit- tee and panels, and a second time to discuss in a closed session the working papers and findings and recom- mendations. This meeting will also involve interactive discussions with NASA personnel from the program. A final report will be developed from discussion at this final meeting. Before the final report is published, com- mittee and panel members may revisit select programs within the PRT group during a short reevaluation pro- cess. This reevaluation will assess progress made by individual programs within the PRT that were initially deemed to be problematic. While the committee's observations will follow the broad themes of technical and scientific quality and appropriateness of the research, the research perform- ers, and the research plan, the panel assessments should use specific criteria, where appropriate. These criteria are discussed next.
102 Research Portfolio Is the balance between fundamental and user- driven research proper? Is research being conducted in the proper ar- eas? Are there plausible hypotheses supporting each of the research plans? Is far-term research at the forefront of science and determined to be a world-class endeavor? Is the proper amount of high-risk, high-payoff research being pursued? Is the application of fundamental science to solve real-world problems adequate? Formu/ation of the Research Plan . . AN ASSESSMENT OF NASA 'S PIONEERING REVOLUTIONARY TECHNOLOGY PROGRAM Connections to the Broader Community Are the program's goals and objectives clearly defined and consistent with relevant docu- ments such as NASA's Strategic Plan? Is there evidence of a clear understanding of the need by NASA's enterprises, other organi- zations (e.g., the FAA, DOD, etc. ), or the aero- space community at large for the R&D or analysis and the potential benefits? Are the program's deliverables to those organizations Methodo/ogy clearly articulated, and are those organizations adequately involved in the planning and review process? Can the expected benefits be accomplished by the proposed research? If not, is the path for adequately maturing the research clear? Is this planning well supported by sufficient decision points, downselects, customer agreements, and/or unallocated outyear funding? Are there sufficient near-term deliverables or progress metrics by which the program can be regularly assessed? Are there sufficient off- ramps or sunsets to ensure that funding is real- located within the program or to other pro- grams if the program does not make adequate progress toward one or more of its goals and objectives? Are the program's plans for inde- pendent and/or external reviews adequate and appropriate? Are appropriate scientific and technical objec- tives being posed, taking into consideration program goals, NASA's strengths, and the time horizon for the project? Are the critical per- sonnel and facilities required to support the program well defined? What programs or program elements should be performed in-house at NASA and be exempt from competition with industry or academia? Is there evidence that the research plan for the area under review reflects a broad understand- ing of the underlying science and technology and of comparable work within other NASA units as well as industry, academia, and other federal laboratories? Is there evidence that the research builds ap- propriately on work already done elsewhere? Does it leverage the work of leaders in the field? Is the strategy for out-of-house work (competitions, partnerships, etc.) well chosen and managed? Is the research being accomplished with a proper mix of personnel from NASA, academia, industry, and other government agencies? Is the program using high-quality research performers, or is there untapped tal- ent outside the program that can be brought to bear? How well crafted are the research plans for the areas under review? In general, is the use of laboratory experiment, modeling, simulation, and/or field testing appropriate? How well are these methods integrated? Have the appropriate supporting system-level assessments been conducted? Do both the researchers and managers under- stand and manage the risks involved to an ap- propriate level? Are the plans for further study reasonable and justifiable? Overa// Capabi/ities Is the scientific or engineering quality of the work (including work performed in academia and industry) comparable to similar world- class efforts at other institutions, and is it ap- propriate for the goal? Are the qualifications of the scientific and en- gineering staff (including researchers in academia and industry) sufficient to achieve program goals?
APPENDIX A . 103 Are the capabilities, quantity, and state of readiness of equipment and facilities sufficient to achieve program goals? Are personnel, equipment, and facilities sup- plied by support contractors used efficiently? Do they fill gaps in government capabilities without duplication? The selection of criteria for each assessment and the relative weights given to each criterion are within a panel's discretion and can vary from program to pro- gram. Neither the committee nor the panels will make explicit budget recommendations to NASA but will instead comment on program content, gaps in technol- ogy, and other issues outlined above.
