Click for next page ( 11

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 10
Intr ~ ti " Criticality Review and Hazard Analysis. NASA and the primary Shuttle contractors should review all Criticality 1, JR, 2, and 2R items and hazard analyses. This review should iden- tify those items that must be improved prior to flight to ensure mission success and flight safety. An Audit Panel, appointed by the National Research Council, should verify the adequacy of the effort and report directly to the Administrator of NASA." 2.1 PURPOSE OF STUDY The Space Shuttle Challenger disaster of lanuary 28, 1987, stunned NASA and the entire nation. As the shock of the accident began to subside, NASA initiated a wide range of actions designed to ensure greater safety in various aspects of the Shuttle system and an improved focus on safety throughout the National Space Transportation System (NSTS) Program. A number of these actions were prompted by recommendations of the Presidential Commis- sion on the Space Shuttle Challenger Accident (also known as the Rogers Commission). Recommendation Ill of the Presidential Com- mission (see box above) directed NASA to review certain safety-critical items on the Shuttle as well as the existing analyses of hazards that could affect Shuttle operations and system safety, and to identify needed improvements in the Shuttle system. It also recommended the establishment of an audit panel, under the auspices of the National Research Coun- ci! (NRC), to monitor that review effort and verify its adequacy. At NASA's request, the NRC formed the Committee on Shuttle Criticality Review and Hazard Analysis Audit to conduct this audit. The Committee consisted of 12 people with expertise in a range of relevant areas: space system devel- opment and operations, aircraft development and 1 operations, propulsion systems, avionics, struc- tures, statistics, reliability and safety, and risk assessment and management of complex techno- Togical systems. They were asked to evaluate NASA's effort in response to the Rogers Commis- sion recommendation and to report their findings and recommendations directly to the NASA Ad- mlnlstrator. See Appendix B for the full text of the pertinent establishing documents. 2.2 STUDY APPROACH 2.2.1 Interpretation of Task Following its charge from the Rogers Commis- sion and NASA, the Committee planned initially to focus its audit strictly on certain specific features of the NASA safety process: the Critical Items List (CIL) and the NASA review of those Shuttle primary and backup units whose failure might result in loss of life, the Shuttle vehicle itself, or the mission (i.e., the Criticality 1, 1R, 2 and 2R items41; the Failure Modes and Effects Analyses (FMEA) on which the criticality determinations are largely based; and the hazard analyses and their review. (See Section 3 for a description of these activities and their interrelationships.) 4 See Table 3-l for definitions of Criticality levels. 10

OCR for page 10
Early in its study, the Committee recognized that to fulfill its charge to "verify the adequacy of the effort" it must broaden the scope of its audit to include an assessment, from a risk management point of view, of NASA's overall process for iclentifying, assessing, reviewing, and implementing changes in the Space Shuttle system. That broacler scope would inclucle not only other safety analyses and functions, but also the relationship of safety elements ant] organizations to the continuing proc- ess of Space Shuttle design and engineering. (See Appendix B for the resulting Statement of Task.) Thus, in the context of evaluating NASA's pro- cedures for detecting, assessing, ant] dealing with hazards ant] potential failure modes in the Shuttle system, the Committee would seek to determine: What has NASA clone in the past? What is it doing differently now? How adequate are these procedures? Where are the flaws in the process, if any? 2.2.2 Plan and Structure The Committee began with a general review of NASA's policies and procedures for reviewing safety- critical items and analyzing hazards. This process overview, provides] in briefings by and discussions with NASA officials and managers of the NSTS Program and its component projects, provided not only a general overview but also the status of the reevaluation which NASA hac! undertaken of the FMEA/CIL ant] hazard analyses. The general re- view also includes] briefings and studies on the ways in which other organizations and industries (e.g., U.S. Air Force, nuclear power, and commer- cial aviation) accomplish similar safety analyses anc! reviews. The Committee decided to conduct its audit of the reevaluation on several levels. First, it would conduct a detailed review of one or two major Space Transportation System (STS) elementsS, and the reevaluation process and its results. The Space Shuttle Main Engine (SSME) and the Solic! Rocket Booster/Solic! Rocket Motor (SRB/SRM) were se- lectec! for this auclit, since the Committee felt that 5 NASA terminology generally refers to the entire Space Shuttle as a "system" composed of four major flight "elements": Orbiter, Space Shuttle Main Engines, Solid Rocket Boosters/Solid Rocket Motors, 6 and External Tank. Each of these elements is composed of major systems which are, in turn, made up of subsystems, units, and components or piece parts. the greatest hazards are in propulsion. During its work, the Committee identifier] other areas of concern which lee] to a cletailed examination of a number of different aspects of the STS safety- relatec] activities. Each of these audits was con- clucted through a series of meetings with NASA and contractor personnel on-site at contractor facilities anct NASA centers. Concern about the potential weakness of NASA's "top-down" analyses to complement the "bottom- up" FMEA/CILs (which seemed to be the dominant safety evaluation tool) led the Committee to initiate auclits related to the integrates] system safety as- sessments across all of the elements of the STS. For example, it examiner! interactions arising from the generation and distribution of electrical power ant] fresh water aboarc! the STS, and the generation anc! distribution of hydraulic power in the Orbiter and the SRB. This work is reflected particularly in Section 5.7 of this report. The 17-inch diameter fuel and oxidizer cliscon- nect valves between the Orbiter ant! the External Tank (ET) were selected for cletailed examination of the preparation ant! role of hazard analyses in STS risk assessment to complement the broader, more general treatment of this subject obtainer! in briefings, discussions, and written answers to Com- mittee questions. This audit contributes] signifi- cantly to Sections 5.3 and 5.11. The Committee cliscoverecl early in its work that the large number of Criticality 1 ant! 1R items on the STS are not ranker! by priority of their impor- tance and that NASA did not appear to be making much use of moclern analytical techniques in quan- titatively assessing probabilities of failures ant! their effects, and levels of risk in the program. This lee] to a special investigation of the extent to which such techniques are used in the NSTS program, and of methods which might be of special value to the program. (See especially Sections 5.2 and 5.6, and Appendices D and E.) Since the STS structure was excluclecl by NASA from the FMEA/CIL process, and since there were concerns about the actual margins of safety, the Committee examined in some detail the past history and current activity of NASA in this critical area (see Section 5.10.21. The safety/risk assessment for Orbiter software also is handled in a very different manner than hardware (e.g., no FMEA/CIL). Therefore, it too was subjected] to a special audit, the results of which are reflected! primarily in Sections 5.8 and 5.10.3. 11

OCR for page 10
Finally, because of significant problems in the past, the Committee examiner] in some detail, from a safety standpoint, the history ant! current redesign of the Orbiter nose wheel steering system, anc! the main wheels anc! brakes. These more cletailect auclits of selectee! subsys- tems, when couplet! with the broacler investigations of the SSME ant! SRB elements and the STS as a whole, proviclec! the basis for the Committee's finclings, conclusions, and recommendations in Sec- tion 5 anc! supporting material in Appendices D through F. The Committee clicI not examine the interfaces between the STS and its payloads to the extent that the members were comfortable in mak- ing any specific conclusions anc! recommendations beyonc! those for the NSTS Program in general. 2.2.3 Meetings and Site Visits Apart from the meetings and site visits concluctec3 by incliviclual and groups of Committee members, the full Committee helc! a total of 12 meetings. Nine meetings were largely fact-fincling with NASA anc! contractor personnel; three were clevotec! to formulating conclusions and recommendations, and preparation of this final NRC report (see Table 2-~. The Committee met with a large number of NASA personnel representing Headquarters man- agement, as well as program and project manage- ment at all three of the NASA fielc! centers having primary involvement in the NSTS Program. Safety, Reliability, and Quality Assurance (SR&QA) organizations6 were heavily represented among those presenting briefings anc! working with the Committee. Prime contractors for STS elements, anc! contractors for several subsystems anc! STS . . Integration activities were a so extensive y repre- sentecI, both at NASA centers anc! at their own facilities. In acIclition, inclepenclent contractors in- volvec! in the FMEA/CIL reevaluation were heart! from. In aciclition to the meetings and site visits, input was proviclec! by NASA in two other very important ways. First, two NASA liaison persons representing Headquarters management anc! the NSTS Program (SR&QA Office) facilitated the Committee's auclit anc! proviclect direct input on specific questions on 6 As of September 1987, the NASA Headquarters organization is called Safety, Reliability, Maintainability, and Quality Assurance (SRM&QA), while the similar organizations at the NASA centers are still named SR&QA. In this report, SR&QA also is used to refer generically to this function. an ongoing basis. SeconcIly, a series of documents were proviclec! giving cletailec} answers to lists of questions clevelopec! by the Committee on a wicle range of subjects. These "Q&A" documents were supplementecI by substantial reports from NASA r on certain points of concern. It shouIc:l be noted here that the Committee was at all times impressed anc! gratifier! by the excellent support that was consistently provicled by NASA management and staff to accommodate the Com- , . . . .. mlttee s auc It ant Its loqulrles. 2.2.4 Interim Reports of the Committee In accordance with its charge, the Committee issuec! two interim progress reports in the form of letters to the NASA Administrator (see Appendix C). The first letter report was dated January 13, 1987, some four months after the Committee first met. Presenter] in person by Committee Chairman Alton D. Slay to the Administrator anc! his key deputies, it presented four specific suggestions for improvement in aspects of the FMEA/CIL and hazard analysis processes, based on the initial phase of the Committee's audit. The Administrator dis- cussed these matters with Chairman Slay, and then responder] formally to SCRHAAC on April 22, 1987, to describe actions taken with regard to the Committee's concerns. As following sections will detail, specific changes in procedure ant! approach have already been made in response to two of the four suggestions (see NASA response to the first letter report, in Appendix C). In aclclition, Committee Chairman Slay appearec! before the House Subcommittee on Space Science and Applications (Committee on Science, Space and Technology) on April 29, 1987, to discuss the findings contained in the first letter report. The Committee's second letter report was issued July 22, 1987, and was again cleliverec! personally by the Chairman and discussed with the Adrr~in- istrator. It summarizer! SCRHAAC's continuing activities and findings, also commenting on the actions taken by NASA in response to the first letter report. In this second report, eight new topics were aclclressed, some of them expressing approval of particular aspects of the STS risk assessment and management process, and planned changes, and others highlighting areas of concern on the part of the Committee. Some of the concerns expressed in the interim reports have been resolver] since the reports were 12

OCR for page 10
TABLE 2-1 Meetings of the Committee on Shuttle Criticality Review and Hazard Analysis Audit Date Location Participants Purpose 1. 9122-23186 NRC, Washington, DC 2. 1 0127-28186 Rockwell STS Div. Rocketdyne Div. Los Angeles, CA 3. 11/10/86 NRC, Washington, DC 4. 12/15-16/86 5. 1/1 (1 6/87 NASA JSC, Houston MSFC Huntsville, AL KSC FL 6. 2/1~11/87 NRC, Washington, DC 7. 3/18/87 Rocketdyne Div. Canoga Park, CA 8. 4124-25187 NRC, Washington, DC 9. 512~29187 NRC, Washington, DC 10. 7/13-14/87 NRC, Woods Hole, MA Executive session 11. 913-4187 NRC, Washington, DC Executive session 12. 10/12/87 NRC, Washington, DC Executive session ACRONYMS: C I L C ritical Items List FMEA Failure Modes and Effects Analysis HQ Headquarters (of NASA) JSC Johnson Space Center KSC Kennedy Space Center MSFC Marshall Space Flight Center NASA National Aeronautics & Space Administration NRC National Research Council presented; others remain at issue. All of the con- cerns identifies! in those reports are discussed in Section 5 of this report. It shouic! be noted that NASA's safety process in general, and the current reevaluation in particular, have been undergoing consiclerable change following the Challenger ac- cident anc! during the Committee's audit. Indeed, some of the changes have resulted from the Com- mittee's discussions with NASA officials and from its interim reports. Thus, many of the subjects coverer! by this report have been "moving targets" that continuer! to change as this report was being prepared. However, the Committee believes that the report reflects the facts and circumstances as of: September 1987. 2.3 ORGANIZATION OF THE REPORT Following this introduction is Section 3, which presents an overview of NASA's safety process for 13 NASA Headquarters, JSC, MSFC & KSC staff Boeing Comm'l Aircraft representatives Rockwell STS Div., Rocketdyne Div. NASA HQ, JSC, MSFC, USAF Space Div. and Aerospace Corp. staff NASA Assoc. Admins. for Space Flight & SRM&QA, NSTS Program Manager NSTS and JSC personnel (including Mission Operations & Astronaut personnel) MSFC and KSC leaders and staff related to STS MSFC & JSC Indpndnt contractor staff, Quant. Risk Assess. (QRA) consultants Rockwell STS Div., Rocketdyne Div., NASA HQ, JSC, and MSFC staff NASA HQ & JSC NSTS personnel NASA HQ SRM&QA personnel NSTS Dep. Dir., Operations JSC, HQ personnel f Process overview, Committee planning SSME, Orbiter FMEA/CIL & hazard analysis audit Discussion of concerns; draft first interim report Review STS risk mangement and operations Overview of MSFC & KSC FMEA/CILs & hazard analyses QRA, Independent contractor FMEA/CIL reviews SSME; STS integration activities SRM&QA status and functions STS integration & software STS oprns, payloads, PCASS, system engineering, draft second interim report Review & discuss information collected Formulate conclusions, rec- ommendations; review drafts Review & approve final text NSTS National Space Transportation System PCASS Program Compliance Assurance and Status System QRA Quantitative Risk Assessment SRM&QA Safety, Reliability Maintainability & Quality Assu rance SSME Space Shuttle Main Engine STS Space Transportation System USAF United States Ai r Force the NSTS Program as the Committee unclerstancis it. That section is provider] as a tutorial for those who may not be familiar with this complex process. Section 4 briefly describes the Committee's con- ception of modern risk management, including the essential element of objective risk assessment, and contrasts it with NASA's safety process in general terms. The heart of the report is Section 5, which presents discussion, finclings, and recommendations regarding particular aspects of NASA's STS safety assurance process. It comprises the results of the Committee's audit. The section is clivicled into 11 subsections, each dealing with a different aspect of the process (with some encompassing relatecl but distinct topics). Section 6 is a brief summary of the main "lessons learned" by SCRHAAC in the course of its auclit. These lessons, derivecl from the STS review, are

OCR for page 10
considered to be applicable to other large and complex technological systems which, by their size and complexity, require the involvement of several major centers and organizations for their execution. Finally, ~ series of appendices are provided. Some, like Appendix A 6'Acronyms and DeEnE Honshu, are intended as useful tools for the reader. Others are provided as amplification or background on various subiccts addressed in the report See the Table of Contents for ~ complete Usting. 14