Evaluation of U.S. Air Force Preacquisition Technology Development

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
Committee on Evaluation of U.S. Air Force Preacquisition Technology Development Air Force Studies Board Division on Engineering and Physical Sciences R01861 AF PTD--CS4 final.indd 1 2/18/11 2:25 PM

OCR for page R1
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, N.W. Washington, DC 20001 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This is a report of work supported by Grant FA9550-09-1-0653 between the U.S. Air Force and the National Academy of Sciences. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the view of the organiza- tions or agencies that provided support for the project. International Standard Book Number-13: 978-0-309-16275-3 International Standard Book Number-10: 0-309-16275-0 Limited copies of this report are Additional copies are available from: available from: Air Force Studies Board The National Academies Press National Research Council 500 Fifth Street, N.W. 500 Fifth Street, N.W. Lockbox 285 Washington, DC 20001 Washington, DC 20055 (202) 334-3111 (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area) Internet, http://www.nap.edu Copyright 2011 by the National Academy of Sciences. All rights reserved. Printed in the United States of America R01861 AF PTD--CS4 final.indd 2 2/18/11 2:25 PM

OCR for page R1
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to as- sociate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council. www.national-academies.org R01861 AF PTD--CS4 final.indd 3 2/18/11 2:25 PM

OCR for page R1
R01861 AF PTD--CS4 final.indd 4 2/18/11 2:25 PM

OCR for page R1
COMMITTEE ON EVALUATION OF U.S. AIR FORCE PREACQUISITION TECHNOLOGY DEVELOPMENT RICHARD V. REYNOLDS, The VanFleet Group, LLC, Chair DONALD C. FRASER, Charles Stark Draper Laboratory (retired), Vice Chair CHARLES E. ADOLPH, Albuquerque, New Mexico BRIAN A. ARNOLD, Raytheon Company FRANCIS J. BAKER, Wright State University THOMAS W. BLAKELY, Lockheed Martin Aeronautics Company CLAUDE M. BOLTON, Defense Acquisition University THOMAS J. BURNS, SET Corporation LLEWELLYN S. DOUGHERTY, Raytheon Company RICHARD B.H. LEWIS, Lockheed Martin Corporation ELLEN M. LORD, AAI Corporation CHRISTOPHER E. MANUEL, Sierra Nevada Corporation MATT L. MLEZIVA, Wildwood Strategic Concepts RONALD E. MUTZELBURG, Alexandria, Virginia RICHARD L. RUMPF, Rumpf Associates International, Inc. Staff CARTER W. FORD, Program Officer and Study Director SARAH M. CAPOTE, Research Associate ZEIDA PATMON, Program Associate v R01861 AF PTD--CS4 final.indd 5 2/18/11 2:25 PM

OCR for page R1
AIR FORCE STUDIES BOARD GREGORY S. MARTIN, GS Martin Consulting, Chair PAMELA A. DREW, TASC, Inc., Vice Chair MARSHA J. BERGER, New York University THOMAS J. BURNS, SET Corporation THOMAS DARCY, EADS North America Defense Company KENNETH E. EICKMANN, U.S. Air Force (retired) JOHN V. FARR, Stevens Institute of Technology RAND H. FISHER, Aerospace Corporation MICHAEL J. GIANELLI, Boeing Company (retired) JACQUELINE GISH, Northrop Grumman Corporation LESLIE GREENGARD, New York University KENNETH C. HALL, Duke University WESLEY L. HARRIS, Massachusetts Institute of Technology PAUL G. KAMINSKI, Technovation, Inc. LESLIE KENNE, LK Associates LESTER L. LYLES, The Lyles Group DEBASIS MITRA, Bell Laboratories MATT L. MLEZIVA, Wildwood Strategic Concepts GERALD F. PERRYMAN, JR., Raytheon Company GENE W. RAY, GMT Ventures MARVIN R. SAMBUR, Headquarters, U.S. Air Force (retired) J. DANIEL STEWART, University of Tennessee Staff MICHAEL A. CLARKE, Director JESSICA R. BROKENBURR, Financial Assistant KAMARA E. BROWN, Research Associate SARAH M. CAPOTE, Research Associate GREGORY EYRING, Senior Program Officer CARTER W. FORD, Program Officer CHRIS JONES, Financial Manager ZEIDA PATMON, Program Associate MARGUERITE E. SCHNEIDER, Administrative Coordinator DANIEL E.J. TALMAGE, JR., Program Officer SHANNON C. THOMAS, Program Associate vi R01861 AF PTD--CS4 final.indd 6 2/18/11 2:25 PM

OCR for page R1
Preface The enactment of the Weapon Systems Acquisition Reform Act of 2009 (Public Law 111-23) and the recent revision of Department of Defense Instruction 5000.02 have served to highlight the complexity of the Department of Defense acquisition process.1 This report serves as a follow-on study to the 2008 National Research Council (NRC) report Pre-Milestone A and Early-Phase Systems Engineering: A Retrospective Review and Benefits for Future Air Force Systems Acquisition, which emphasized the role of systems engineering in the acquisition life cycle.2 This complementary report focuses specifically on the role of maturing technologies and inserting them at the appropriate time in the acquisition cycle. Leaders in the Air Force responsible for science and technology and acquisition are trying to determine the optimal way to utilize existing policies, processes, and resources to properly document and execute pre-program of record technology development efforts, including opportunities to facilitate the rapid acquisition of revolutionary capabilities and the more deliberate acquisition of evolutionary capabilities. The Committee on Evaluation of U.S. Air Force Preacquisition Technology Development (see Appendix A for biographical sketches), appointed by the NRC to conduct this study, acknowledges and appreciates the contribution of the mem- 1 For additional information, see http://www.dtic.mil/whs/directives/corres/pdf/500002p.pdf. Ac- cessed January 14, 2011. 2 NRC. 2008. Pre-Milestone A and Early-Phase Systems Engineering: A Retrospective Review and Benefits for Future Air Force Systems Acquisition. Washington, D.C.: The National Academies Press. vii R01861 AF PTD--CS4 final.indd 7 2/18/11 2:25 PM

OCR for page R1
Preface viii bers of the Air Force Studies Board (AFSB) of the NRC in developing the study’s statement of task (see Box 1-1 in Chapter 1) in concert with the Air Force sponsor. The AFSB was established in 1996 as a unit of the NRC at the request of the United States Air Force. The AFSB brings to bear broad military, industrial, and academic scientific, engineering, and management expertise on Air Force technical challenges and other issues of importance to senior Air Force leaders. The board discusses potential studies of interest, develops and frames study tasks, ensures proper project planning, suggests potential committee members and reviewers for reports produced by fully independent ad hoc study committees, and convenes meetings to examine strategic issues. The board members were not asked to endorse the committee’s conclusions or recommendations, nor did they review the final draft of this report before its release, although board members with appropriate expertise may be nominated to serve as formal members of study committees or as report reviewers. The committee thanks the many people who provided information to the committee, including the guest speakers shown in Appendix B, their organizations, and supporting staff members; and others, including the study sponsor Dr. Steven Walker, Deputy Assistant Secretary of the Air Force for Science, Technology, and Engineering, Office of the Assistant Secretary of the Air Force for Acquisition, and his staff members. The committee is also grateful to the NRC staff members who provided their dedicated support throughout the study. Richard V. Reynolds, Chair Donald C. Fraser, Vice Chair Committee on Evaluation of U.S. Air Force Preacquisition Technology Development R01861 AF PTD--CS4 final.indd 8 2/18/11 2:25 PM

OCR for page R1
Acknowledgment of Reviewers This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures ap- proved by the National Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: Vinton G. Cerf, Google, Inc., Pamela A. Drew, TASC, Inc., Charles B. Duke, Xerox Corporation (retired), Annette J. Krygiel, Great Falls, Virginia, Paul E. Nielsen, Maj Gen, USAF (retired), Carnegie Mellon University, Robert E. Schafrik, GE Aircraft Engines, and Larry D. Welch, Gen, USAF (retired), Institute for Defense Analyses. Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the report before its release. The review of this report was overseen by Lawrence T. Papay, PQR, LLC, and Lawrence J. Delaney, Ti- tan Corporation (retired). Appointed by the National Research Council, they were ix R01861 AF PTD--CS4 final.indd 9 2/18/11 2:25 PM

OCR for page R1
acknowledgment reviewers x of responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review com- ments were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution. R01861 AF PTD--CS4 final.indd 10 2/18/11 2:25 PM

OCR for page R1
Contents SUMMARY 1 1 PREACQUISITION TECHNOLOGY DEVELOPMENT FOR AIR 11 FORCE WEAPON SYSTEMS Statement of Task and Committee Formation, 12 The Parameters of This Study, 13 Committee Approach to the Study, 16 Three Domains of the Air Force, 16 Air, 16 Space, 18 Cyberspace, 21 Air Force Science and Technology Strategic Planning, 24 The “Three R” Framework, 25 Requirements, 25 Resources, 27 The Right People, 29 Report Organization, 31 2 THE CURRENT STATE OF THE AIR FORCE’S ACQUISITION 33 POLICIES, PROCESSES, AND WORKFORCE Current and Historical Policies and Processes Related to Technology Development, 33 Planning, Programming, Budgeting, and Execution System, 34 xi R01861 AF PTD--CS4 final.indd 11 2/18/11 2:25 PM

OCR for page R1
contents xii Department of Defense Instruction 5000.02, 34 Air Force Acquisition Improvement Plan, 37 Joint Capabilities Integration and Development System, 37 Competitive Prototyping, 37 Historical Governance Related to Technology Development, 38 The Trust “Death Spiral,” 38 The “Three R” Framework, 44 Requirements, 44 Resources, 54 The Right People, 58 Concluding Thoughts, 60 3 GOVERNMENT AND INDUSTRY BEST PRACTICES 62 Best Practices, 62 Examples of Government Best Practices, 63 Joint Improvised Explosive Device Defeat Organization, 63 Capabilities Development for Rapid Transition, 65 Big Safari, 66 Future Naval Capabilities Process, 67 Army S&T Objective Process, 70 Dragon Eye, 73 Examples of Joint Government and Industry Cooperation, 76 The Rapid Reaction Technology Office and the VADER System, 76 DARPA’s Adaptive Execution Office, 78 Ground Robotics Consortium, 79 The National Small Arms Center, 80 Sierra Nevada Corporation and the Commercial Space Sector, 81 The Naval Center for Space Technology, 82 Examples of Industry Best Practices, 83 High Technological/Manufacturing/Integration Readiness Levels Pay Off: Ford, Jaguar, and Adaptive Cruise Control, 83 The Innovation Culture at 3M, 84 Technology Networks at Raytheon, 86 Concluding Thoughts, 87 4 THE RECOMMENDED PATH FORWARD 89 Key Issue 1, 91 Freezing Requirements Too Early or Too Late in the Technology Development Phase Can Lead to a Mismatch Between Technology-Enabled Capabilities and Requirement Expectations That Significantly Reduces the Probability of Successful Technology Transitions, 91 R01861 AF PTD--CS4 final.indd 12 2/18/11 2:25 PM

OCR for page R1
contents xiii Key Issue 2, 92 The Lack of an Air Force-Level Science and Technology Strategy Leads to AFRL Efforts That May Not Support Desired Strategic Air Force Capabilities, and to the Fragmented Prioritization and Allocation of 6.4 Technology Transition Funds, 92 Key Issue 3, 96 Current Air Force Funding and Business Practices for Pre- Milestone B Activities Are Inconsistent with Department of Defense Instruction 5000.02, 96 Key Issue 4, 97 Technology Readiness Levels Must Be Accurately Assessed to Prevent Programs from Entering the Engineering and Manufacturing Development Phase with Immature Technology, 97 Key Issue 5, 98 Developing Technologies and Weapon Systems in Parallel Almost Inevitably Causes Cost Overruns, Schedule Slippage, and/or the Eventual Reduction in Planned Capabilities, 98 Key Issue 6, 99 Weak Ties and Lack of Collaboration Within and Between Government and Industry Lead to Lack of Awareness of Government Priorities and of Industry’s Technology Breakthroughs, 99 Key Issue 7, 100 A Much Reduced and Inexperienced Development Planning Workforce Has Weakened the Technology Transition Bridge Between Laboratories, Product Centers, and Major Commands, 100 Conclusion, 101 APPENDIXES A Biographical Sketches of Committee Members 105 B Meetings and Participating Organizations 115 C Background Information on Policies and Processes Related to 123 Technology Development D Background Information on the Vanguard Process and Applied 132 Technology Councils R01861 AF PTD--CS4 final.indd 13 2/18/11 2:25 PM

OCR for page R1
R01861 AF PTD--CS4 final.indd 14 2/18/11 2:25 PM

OCR for page R1
Acronyms A&AS advisory and assistance services ACC adaptive cruise control ACD&P Advanced Component Development and Prototypes AEO Adaptive Execution Office AFI Air Force Instruction AFMC Air Force Materiel Command AFRL Air Force Research Laboratory AFSB Air Force Studies Board AFSC Air Force Systems Command AFSPC Air Force Space Command AIP Acquisition Improvement Plan AoA Analysis of Alternatives ARDEC Armament Research, Development, and Engineering Center ATC Applied Technology Council ATO Army Technology Office CDRT Capabilities Development for Rapid Transition CJCS (J8) Chairman of the Joint Chiefs of Staff (Force Structure, Resources, and Assessment Directorate) COCOM (United States) Combatant Command CP competitive prototyping CPM Critical Path Method CSAF Chief of Staff of the Air Force xv R01861 AF PTD--CS4 final.indd 15 2/18/11 2:25 PM

OCR for page R1
acronyms xvi C/SCSC Cost/Schedule Control System Criteria CTE critical technology element DAG Defense Acquisition Guidebook DAPA Defense Acquisition Performance Assessment DARPA Defense Advanced Research Projects Agency DDR&E Director, Defense Research and Engineering DMEA Defense Microelectronics Activity DoD Department of Defense DP Development Planning EFP explosively formed penetrator EMD Engineering and Manufacturing Development (phase) EPP “Enhanced” Planning Process EUREKA European Research Cooperation Agency EVM Earned Value Management FAR Federal Acquisition Regulation FFRDC Federally Funded Research and Development Center FNC Future Naval Capabilities GAO Government Accountability Office GPS Global Positioning System ICD Initial Capabilities Document IED improvised explosive device IPT Integrated Product Team IR&D independent research and development ISET Industry System Engineering Team JCA Joint Capability Area JCAAMP Joint IED Defeat Capability Approval and Acquisition Management Process JCIDS Joint Capabilities Integration and Development System JGRE Joint Ground Robotics Enterprise JHU/APL Johns Hopkins University/Applied Physics Laboratory JIEDD Joint Improvised Explosive Device Defeat JIEDDO Joint Improvised Explosive Device Defeat Organization JIPT JIEDDO Integrated Process Team JPG Joint Programming Guidance JR2AB JlEDDO Requirements, Resources, and Acquisition Board R01861 AF PTD--CS4 final.indd 16 2/18/11 2:25 PM

OCR for page R1
acronyms xvii JRAC Joint Rapid Acquisition Cell JROC Joint Requirements Oversight Council JSF Joint Strike Fighter JUONS Joint Urgent Operational Needs Statement LCC life-cycle cost MAJCOM Major Command MDA Milestone Decision Authority MDAP Major Defense Acquisition Program MDD Material Development Decision MRL Manufacturing Readiness Level NASA National Aeronautics and Space Administration NDAA National Defense Authorization Act NLOS-LS Non-Line-of-Sight Launch System NRC National Research Council NRL Naval Research Laboratory NSAC National Small Arms Center O&S operations and support OMB Office of Management and Budget ONR Office of Naval Research OSD Office of the Secretary of Defense OTA Other Transactions Agreement OUSD (AT&L) Office of the Under Secretary of Defense (Acquisition, Technology, and Logistics) OUSD (C) Office of the Under Secretary of Defense (Comptroller) PDR Preliminary Design Review PE Program Element PERT Program Evaluation Review Technique POM Program Objective Memorandum PPBE Planning, Programming, Budgeting, and Execution PPBES Planning, Programming, Budgeting, and Execution System R&D research and development RAM Requirements Analysis and Maturation RDT&E Research, Development, Test, and Evaluation ROVER Remotely Operated Video Enhanced Receiver RRTO Rapid Reaction Technology Office R01861 AF PTD--CS4 final.indd 17 2/18/11 2:25 PM

OCR for page R1
acronyms xviii S&T science and technology SAF Secretary of the Air Force SAF/AQ Secretary of the Air Force (Acquisition) SAF/AQX Secretary of the Air Force (Acquisition Integration) SMC Space and Missile Systems Center SNC Sierra Nevada Corporation SOTVS Special Operations Tactical Video System SPG Strategic Planning Guidance SPO System Program Office SRR System Requirements Review TARDEC Tank Automotive Research, Development, and Engineering Center TDS Technology Development Strategy TIG technology interest group TRA Technology Readiness Assessment TRADOC Training and Doctrine Command TRL Technology Readiness Level TSPR Total System Performance Responsibility USAF United States Air Force USN United States Navy VADER Vehicle and Dismount Exploitation Radar WSARA Weapon Systems Acquisition Reform Act XR Product Centers’ Development Planning Organization R01861 AF PTD--CS4 final.indd 18 2/18/11 2:25 PM