Click for next page ( R2


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
~ ~ Post-Challenger Evaluation of Space Shuttle Risk Assessment and Management Prepared by the Committee on Shuttle Criticality Review and Hazard Analysis Audit of the Aeronautics and Space Engineering Boars! with staff support from the Space Applications Boars] Commission on Engineering ant! Technical Systems National Research Council 6 NATIONAL ACADEMY PRESS January 1988

OCR for page R1
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The National Academy of Sciences is a private, nonprofit, self- perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Frank Press is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its admin- istration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal gov- ernment. The National Academy of Engineering also sponsors engi- neering programs aimed at meeting national needs, encourages edu- cation and research, and recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Samuel O. Thier is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engi- neering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Frank Press and Dr. Robert M. White are chairman and vice chairman, respectively, of the National Research Council. This study was conducted under Contract No. NASW-4003 between the National Academy of Sciences and the National Aeronautics and Space Administration. Available from: Aeronautics and Space Engineering Board National Research Council 2101 Constitution Avenue, N.W. Washington, D.C. 20418. Printed in the United States of America

OCR for page R1
COMMITTEE ON SHUTTLE CRITICALITY REVIEW AND HAZARD ANALYSIS AUDIT ALTON D. SLAY, Gen., (USAF, Retired), Slay Enterprises, Inc., McLean, VA (former Com- mancler, USAF Systems Commancl), Chairman GERARD W. ELVERUM, JR., Vice President and General Manager, Applied Technology Division, TRW, Inc., Redonclo Beach, CA. B. JOHN GARRICK, President, Pickard, Lowe ant] Garrick, Newport Beach, CA (from September 22, 1986 to February 19, 1987) GRANT L. HANSEN, retired Vice President, Sys- tems Development Corporation, San Diego, CA WILLIS M. HAWKINS, Senior Advisor, Lockheec! Corporation (former Senior Vice President), Cal- abasas, CA T. GRANT HEDRICK, Senior Management Con- sultant, Grumman Corporation (former Senior Vice President), Bethpage, NY BRUCE HOADLEY, Division Manager, Analytical Methocls and Software Systems, Bell Commu- nications Research, Recibank, N] WILLIAM B. LENOIR, Principal, Space Systems Practice, Booz-Allen & Hamilton (former astro- naut), Bethesda, MD , 6 . . . 111 ARTUR MAGER, Consultant (retired Group Vice President, The Aerospace Corporation), Los An- geles, CA NORMAN R. PARMET, retired Vice Presicient- Engineering & Quality Assurance, Trans World Airlines, Fairway, KS ROBERT E. UHRIG, Distinguished Professor of Engineering, Department of Nuclear Engineer- ing, University of Tennessee, Knoxville, TN JAMES J. KRAMER, Manager, A(lvance(l Tech- nical Programs, General Electric Company, Washington, DC (Ex Officio Member, Chair- man, Aeronautics and Space Engineering BoarcI) Staff Davis] S. Johnson, Study Director Robert H. Korkegi, Director, Aeronautics and Space Engineering Boar(l William H. Michael, ir., Director, Space Applica- tions Board CourtIanc! S. Lewis, Consultant Vki Marrero, Administrative Assistant Amy Janik, Administrative Secretary

OCR for page R1

OCR for page R1
PREFACE The President of the United States approved the Space Shuttle program in ~ 972, to become the heart of the National Space Transportation System (NSTS) and provide routine, economical access to space. The launch of Columbia in 1981 the first reusable vehicle to be launcher! ant! orbit the earth opener! a new era. The clevelopment of the Space Shuttle ant] its operation ant! maintenance have involves] several National Aeronautics and Space Administration (NASA) centers, their indus- trial prime contractors, and scores of subcontrac- tors, including tens of thousands of people. This must be considered one of the most complex technical undertakings of all time. After 24 successful Shuttle flights, the Space Shuttle Challenger accident of lanuary 28, 1986, stunner! the entire nation and indeed the world. In response to the accident President Reagan estab- lishecI the Presidential Commission on the Space Shuttle Challenger Accident (frequently called the Rogers Commission, after its chairman) to inves- tigate the accident and make recommendations for the safe recovery of the Space Transportation System (STS). Among its recommendations, the Rogers Commission caller] upon NASA to review certain aspects of its STS risk assessment effort and to "identifyvithose items that must be improved prior to flight to ensure mission success and flight safety." It further recommended that an audit pane! be appointed by the National Research Coun- ci! (NRC) to verify the adequacy of the effort anc! report directly to the Administrator of NASA. The Committee on Shuttle Criticality Review and Haz- ard Analysis Audit was established in response to the recommendation. Beginning with the Commit- tee's first meeting on September 22, 1986, this report is the culmination of 14 months of investi- gation, stucly, and deliberation. While the Committee recognizes that it is not possible, a priori, to guarantee mission success and flight safety, we hope the Committee's conclusions and recommendations will assist NASA in taking those prudent acIditional steps which will provide a reasonable and responsible level of flight safety for the Space Shuttle. As the Challenger accident made painfully obvious, no probe into space is - Report to the President by the Presidential Commission on the Space Shuttle Challenger Accident, William P. Rogers, Chairman (June 1 986). V routine, ant! the Space Shuttle is still a clevelop- mental vehicle. The risks of space flight must be accepted] by those who are asked to participate in each flight as well as by those who are responsible to the nation for achieving its goals in space. Such risks shouIcl also be recognized by Executive Branch officials ant! Congress in their review and oversight of NASA endeavors. The Committee has been favorably impressed by the dedicated effort ant! beneficial results obtained thus far by NASA ant! its contractors from the STS risk assessment and risk management system. The Committee is also gratified by the progress NASA is making in strengthening this system. We appre- ciate the close collaboration the Committee had with NASA and contractor personnel, the interest they showed, and their responsiveness to the Com- mittee's suggestions. Nevertheless, although our general impressions are favorable, we do have suggestions for improvement. It is against this background that the recommendations in this re- port should be judged. The Committee recognizes that the NSTS risk assessment and risk management activities, both existing and with the modifications proposed here, are large and complex. This means that change should be introduced with care. A systematic ex- amination of the entire set of processes supporting risk assessment and management in order to op- timize the total ensemble may be appropriate. Such an examination may be particularly useful in con- junction with implementation of a new program such as the Space Station. Although this report and its recommendations are directed to the NSTS Program, they are of broader applicability. It certainly would be wise to consider the lessons learned when structuring any risk assessment and management system for other programs having attributes similar to the NSTS Program, such as the Space Station Program. It, too, is a large program involving highly complex technology which requires the major participation of several NASA centers and prime contractors for . . its execution. Acknowledgments In conducting its work, the full Committee met an average of once a month for over a year, and individual and groups of members conducted ad-

OCR for page R1
ditional site visits, research, and writing on behalf of the Committee. This intense dedication and the resulting contributions of the highly competent members of the Committee are acknowledged with great appreciation. ~ also would like to express the Committee's appreciation for the excellent support of the National Research Council staff in all aspects of its work. While this report represents the con- tributions by and deliberations of all members of the Committee, ~ would especially like to note the contributions to its writing by David S. Johnson and CourtIand S. Lewis. Mr. Johnson, in particular, was extraordinarily effective as Study Director. His organizational skills, technical knowledge, and hard work were central to our effectiveness as a com- mittee. The peer review by the National Research , 6 Council also made a key contribution to the quality of our reports. In closing, we wish to thank the many NASA and contractor employees who facilitated the work of the Committee, often extending their already heavy workloads in the aftermath of the Challenger accident. Of special note was the assistance pro- vided during the study by the two NASA liaison persons, E. William Land, Jr. and Charles S. Harlan. Alton D. Slay Chairman, Committee on Shuttle Criticality Review and Hazard Analysis Audit V1

OCR for page R1
Contents 1. EXECUTIVE SUMMARY 1.1 NASA's Safety Policy and Process I.2 The Committee's View 1.3 Finclings and Recommendations 1.4 Closing Remarks INTRODUCTION 2.] Purpose of Study 2.2 Study Approach 2.2.1 Interpretation of Task 2.2.2 Plan ant:! Structure 2.2.3 Meetings anal Site Visits 2.2.4 Interim Reports of the Committee 2.3 Organization of the Report 3. NASA'S SAFETY PROCESS FOR THE NATIONAL SPACE TRANSPORTATION SYSTEM PROGRAM 3.1 Policy on Safety , - 3.2 Management Structure 3.2.1 Program Management 3.2.2 Review Boarcis Page 1 4 9 10 10 10 10 11 12 12 13 15 15 16 16 17 3.3 Organizational Roles 17 3.3.1 Engineering Project Offices 3.3.2 Safety, Reliability, Maintainability, and Quality Assurance 3.3.3 Engineering Integration Office 3.4 Safety Analyses 3.4.1 The Failure Modes ant] Effects Analysis and Critical Items List 3.4.2 Hazard Analysis 3.4.3 Element Interface Functional Analysis 3.4.4 Other Analyses 3.4.5 Overall Scope of Analyses 3.5 Post-5 1L Reevaluation/Review 3.5.1 NASA Management Directives 3.5.2 Process 3.5.3 Relation to Engineering Redesign Activity 3.5.4 Relation to Flight Reacliness Process 3.5.5 Data Input ant! Output . . V11 17 17 19 20 20 22 23 23 25 29 29 29 31 31 32

OCR for page R1
page 4. RISK ASSESSMENT AND RISK MANAGEMENT: THE COMMITTEE'S VIEW 4. ~ General Concept 4.2 NASA's Process: Overall Co~nrnents 4.2. ~ NASA Risk Assessment 4.2.2 N ASA Risk Management Sundry . t 33 33 34 34 37 ~ 7 SPACE TRANSPORTATION SYSTEM RISK ASSESSMENT AND RISK MANAGEMENT: DISCUSSION AND RECOMMENDATIONS 5.l Critical Items List Retention Rationale Review and Waiver Process 5.2 Critical Items List Prioritization and Disposition 5.3 Hazard Analysis and Mission Safety Assessment 5.4 Relationship of Formal Risk Assessment Process to Space Transportation System Engineering Changes 5.5 5.6 C 7 5.8 5.9 Timely Feedback of Data into the Risk Assessment and Management Processes The Neec! for Quantitative Measures of Risk The Need for Integrated Space Transportation System Engineering Analysis in Support of Risk Management Independence of the Space Transportation System Certification and Software Val idation and Verification I'rogram Operational Issues 5.9.1 Launch C:omrnit Criteria Waiver Policy - 5.9.2 Human Factors as a Contributor to Risk 5.9.3 Cannibalization c:' f Spare Parts 5.10 Other Weaknesses in Risk Assessment and Management 1 ~ 5.10.3 Software Issues 5.10.4 Differences in Procedures among NASA Centers 5.10.5 Use of Non-Destructive Evaluation Techniques 40 40 45 47 51 52 55 57 59 63 63 64 65 68 5.10.1 The Apparent Reliance on Boards and Panels for Decision Making 68 5.10.2 Adequacy of Orbiter Structural Safety Margins 70 71 72 73 5.11 Focus on Risk Management 6. LESSONS LEARNED 6.l Elements of and Responsibilities for Risk Assessment and Risk Management 6.2 Establishment of Responsibility for Program Direction and Integration 6.3 The Need for Quantitative Measures of Relative Risk 6.4 The Need for Integrated Review and Overview in the Assessment of Risk, and in Independent Evaluation of Retention Rationale 6.5 Indepencience of the Certification of Flight Hardware and of Software Validation ant! Verification 6.6 Safety Margins for Flight Structures 6.7 Other . . . vain 74 79 79 80 81 81 81 81 81

OCR for page R1
APPENDICES: B C D Page Acronyms and Definitions Establishing Reports and Documents Letter Reports to the Aclministrator of NASA and NASA Response Probabilistic Risk Assessment An Improved Critical Item Risk Assessment Procedure for the National Space Transportation System Description of Proposed Systems Safety Engineering Functions in Support of the National Space Transportation System Risk Assessment and Risk Management 1X 83 87 97 115 125 139