Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 10
Intr ~ ti
" Criticality Review and Hazard Analysis. NASA
and the primary Shuttle contractors should
review all Criticality 1, JR, 2, and 2R items
and hazard analyses. This review should iden-
tify those items that must be improved prior
to flight to ensure mission success and flight
safety. An Audit Panel, appointed by the
National Research Council, should verify the
adequacy of the effort and report directly to
the Administrator of NASA."
2.1 PURPOSE OF STUDY
The Space Shuttle Challenger disaster of lanuary
28, 1987, stunned NASA and the entire nation. As
the shock of the accident began to subside, NASA
initiated a wide range of actions designed to ensure
greater safety in various aspects of the Shuttle
system and an improved focus on safety throughout
the National Space Transportation System (NSTS)
Program. A number of these actions were prompted
by recommendations of the Presidential Commis-
sion on the Space Shuttle Challenger Accident (also
known as the Rogers Commission).
Recommendation Ill of the Presidential Com-
mission (see box above) directed NASA to review
certain safety-critical items on the Shuttle as well
as the existing analyses of hazards that could affect
Shuttle operations and system safety, and to identify
needed improvements in the Shuttle system. It also
recommended the establishment of an audit panel,
under the auspices of the National Research Coun-
ci! (NRC), to monitor that review effort and verify
its adequacy. At NASA's request, the NRC formed
the Committee on Shuttle Criticality Review and
Hazard Analysis Audit to conduct this audit. The
Committee consisted of 12 people with expertise
in a range of relevant areas: space system devel-
opment and operations, aircraft development and
· 1
operations, propulsion systems, avionics, struc-
tures, statistics, reliability and safety, and risk
assessment and management of complex techno-
Togical systems. They were asked to evaluate
NASA's effort in response to the Rogers Commis-
sion recommendation and to report their findings
and recommendations directly to the NASA Ad-
mlnlstrator.
See Appendix B for the full text of the pertinent
establishing documents.
2.2 STUDY APPROACH
2.2.1 Interpretation of Task
Following its charge from the Rogers Commis-
sion and NASA, the Committee planned initially
to focus its audit strictly on certain specific features
of the NASA safety process:
· the Critical Items List (CIL) and the NASA
review of those Shuttle primary and backup
units whose failure might result in loss of life,
the Shuttle vehicle itself, or the mission (i.e.,
the Criticality 1, 1R, 2 and 2R items41;
· the Failure Modes and Effects Analyses (FMEA)
on which the criticality determinations are
largely based; and
· the hazard analyses and their review.
(See Section 3 for a description of these activities
and their interrelationships.)
4 See Table 3-l for definitions of Criticality levels.
10
OCR for page 11
Early in its study, the Committee recognized that
to fulfill its charge to "verify the adequacy of the
effort" it must broaden the scope of its audit to
include an assessment, from a risk management
point of view, of NASA's overall process for
iclentifying, assessing, reviewing, and implementing
changes in the Space Shuttle system. That broacler
scope would inclucle not only other safety analyses
and functions, but also the relationship of safety
elements ant] organizations to the continuing proc-
ess of Space Shuttle design and engineering. (See
Appendix B for the resulting Statement of Task.)
Thus, in the context of evaluating NASA's pro-
cedures for detecting, assessing, ant] dealing with
hazards ant] potential failure modes in the Shuttle
system, the Committee would seek to determine:
· What has NASA clone in the past?
· What is it doing differently now?
· How adequate are these procedures?
· Where are the flaws in the process, if any?
2.2.2 Plan and Structure
The Committee began with a general review of
NASA's policies and procedures for reviewing safety-
critical items and analyzing hazards. This process
overview, provides] in briefings by and discussions
with NASA officials and managers of the NSTS
Program and its component projects, provided not
only a general overview but also the status of the
reevaluation which NASA hac! undertaken of the
FMEA/CIL ant] hazard analyses. The general re-
view also includes] briefings and studies on the
ways in which other organizations and industries
(e.g., U.S. Air Force, nuclear power, and commer-
cial aviation) accomplish similar safety analyses
anc! reviews.
The Committee decided to conduct its audit of
the reevaluation on several levels. First, it would
conduct a detailed review of one or two major
Space Transportation System (STS) elementsS, and
the reevaluation process and its results. The Space
Shuttle Main Engine (SSME) and the Solic! Rocket
Booster/Solic! Rocket Motor (SRB/SRM) were se-
lectec! for this auclit, since the Committee felt that
5 NASA terminology generally refers to the entire Space Shuttle as a
"system" composed of four major flight "elements": Orbiter, Space
Shuttle Main Engines, Solid Rocket Boosters/Solid Rocket Motors,
6 and External Tank. Each of these elements is composed of major
systems which are, in turn, made up of subsystems, units, and
components or piece parts.
the greatest hazards are in propulsion. During its
work, the Committee identifier] other areas of
concern which lee] to a cletailed examination of a
number of different aspects of the STS safety-
relatec] activities. Each of these audits was con-
clucted through a series of meetings with NASA
and contractor personnel on-site at contractor
facilities anct NASA centers.
Concern about the potential weakness of NASA's
"top-down" analyses to complement the "bottom-
up" FMEA/CILs (which seemed to be the dominant
safety evaluation tool) led the Committee to initiate
auclits related to the integrates] system safety as-
sessments across all of the elements of the STS.
For example, it examiner! interactions arising from
the generation and distribution of electrical power
ant] fresh water aboarc! the STS, and the generation
anc! distribution of hydraulic power in the Orbiter
and the SRB. This work is reflected particularly in
Section 5.7 of this report.
The 17-inch diameter fuel and oxidizer cliscon-
nect valves between the Orbiter ant! the External
Tank (ET) were selected for cletailed examination
of the preparation ant! role of hazard analyses in
STS risk assessment to complement the broader,
more general treatment of this subject obtainer! in
briefings, discussions, and written answers to Com-
mittee questions. This audit contributes] signifi-
cantly to Sections 5.3 and 5.11.
The Committee cliscoverecl early in its work that
the large number of Criticality 1 ant! 1R items on
the STS are not ranker! by priority of their impor-
tance and that NASA did not appear to be making
much use of moclern analytical techniques in quan-
titatively assessing probabilities of failures ant! their
effects, and levels of risk in the program. This lee]
to a special investigation of the extent to which
such techniques are used in the NSTS program,
and of methods which might be of special value to
the program. (See especially Sections 5.2 and 5.6,
and Appendices D and E.)
Since the STS structure was excluclecl by NASA
from the FMEA/CIL process, and since there were
concerns about the actual margins of safety, the
Committee examined in some detail the past history
and current activity of NASA in this critical area
(see Section 5.10.21. The safety/risk assessment for
Orbiter software also is handled in a very different
manner than hardware (e.g., no FMEA/CIL).
Therefore, it too was subjected] to a special audit,
the results of which are reflected! primarily in
Sections 5.8 and 5.10.3.
11
OCR for page 12
Finally, because of significant problems in the
past, the Committee examiner] in some detail, from
a safety standpoint, the history ant! current redesign
of the Orbiter nose wheel steering system, anc! the
main wheels anc! brakes.
These more cletailect auclits of selectee! subsys-
tems, when couplet! with the broacler investigations
of the SSME ant! SRB elements and the STS as a
whole, proviclec! the basis for the Committee's
finclings, conclusions, and recommendations in Sec-
tion 5 anc! supporting material in Appendices D
through F. The Committee clicI not examine the
interfaces between the STS and its payloads to the
extent that the members were comfortable in mak-
ing any specific conclusions anc! recommendations
beyonc! those for the NSTS Program in general.
2.2.3 Meetings and Site Visits
Apart from the meetings and site visits concluctec3
by incliviclual and groups of Committee members,
the full Committee helc! a total of 12 meetings.
Nine meetings were largely fact-fincling with NASA
anc! contractor personnel; three were clevotec! to
formulating conclusions and recommendations, and
preparation of this final NRC report (see Table
2-~. The Committee met with a large number of
NASA personnel representing Headquarters man-
agement, as well as program and project manage-
ment at all three of the NASA fielc! centers having
primary involvement in the NSTS Program. Safety,
Reliability, and Quality Assurance (SR&QA)
organizations6 were heavily represented among
those presenting briefings anc! working with the
Committee. Prime contractors for STS elements,
anc! contractors for several subsystems anc! STS
. .
Integration activities were a so extensive y repre-
sentecI, both at NASA centers anc! at their own
facilities. In acIclition, inclepenclent contractors in-
volvec! in the FMEA/CIL reevaluation were heart!
from.
In aciclition to the meetings and site visits, input
was proviclec! by NASA in two other very important
ways. First, two NASA liaison persons representing
Headquarters management anc! the NSTS Program
(SR&QA Office) facilitated the Committee's auclit
anc! proviclect direct input on specific questions on
6 As of September 1987, the NASA Headquarters organization is
called Safety, Reliability, Maintainability, and Quality Assurance
(SRM&QA), while the similar organizations at the NASA centers are
still named SR&QA. In this report, SR&QA also is used to refer
generically to this function.
an ongoing basis. SeconcIly, a series of documents
were proviclec! giving cletailec} answers to lists of
questions clevelopec! by the Committee on a wicle
range of subjects. These "Q&A" documents were
supplementecI by substantial reports from NASA
r
on certain points of concern.
It shouIc:l be noted here that the Committee was
at all times impressed anc! gratifier! by the excellent
support that was consistently provicled by NASA
management and staff to accommodate the Com-
, . . . ..
mlttee s auc It ant Its loqulrles.
2.2.4 Interim Reports of the Committee
In accordance with its charge, the Committee
issuec! two interim progress reports in the form of
letters to the NASA Administrator (see Appendix
C). The first letter report was dated January 13,
1987, some four months after the Committee first
met. Presenter] in person by Committee Chairman
Alton D. Slay to the Administrator anc! his key
deputies, it presented four specific suggestions for
improvement in aspects of the FMEA/CIL and
hazard analysis processes, based on the initial phase
of the Committee's audit. The Administrator dis-
cussed these matters with Chairman Slay, and then
responder] formally to SCRHAAC on April 22,
1987, to describe actions taken with regard to the
Committee's concerns. As following sections will
detail, specific changes in procedure ant! approach
have already been made in response to two of the
four suggestions (see NASA response to the first
letter report, in Appendix C).
In aclclition, Committee Chairman Slay appearec!
before the House Subcommittee on Space Science
and Applications (Committee on Science, Space
and Technology) on April 29, 1987, to discuss the
findings contained in the first letter report.
The Committee's second letter report was issued
July 22, 1987, and was again cleliverec! personally
by the Chairman and discussed with the Adrr~in-
istrator. It summarizer! SCRHAAC's continuing
activities and findings, also commenting on the
actions taken by NASA in response to the first
letter report. In this second report, eight new topics
were aclclressed, some of them expressing approval
of particular aspects of the STS risk assessment
and management process, and planned changes,
and others highlighting areas of concern on the
part of the Committee.
Some of the concerns expressed in the interim
reports have been resolver] since the reports were
12
OCR for page 13
TABLE 2-1 Meetings of the Committee on Shuttle Criticality Review and Hazard Analysis Audit
Date Location Participants Purpose
1. 9122-23186 NRC, Washington, DC
2. 1 0127-28186
Rockwell STS Div.
Rocketdyne Div.
Los Angeles, CA
3. 11/10/86 NRC, Washington, DC
4. 12/15-16/86
5. 1/1 (1 6/87
NASA JSC, Houston
MSFC Huntsville, AL
KSC FL
6. 2/1~11/87 NRC, Washington, DC
7. 3/18/87 Rocketdyne Div.
Canoga Park, CA
8. 4124-25187 NRC, Washington, DC
9. 512~29187
NRC, Washington, DC
10. 7/13-14/87 NRC, Woods Hole, MA Executive session
11. 913-4187 NRC, Washington, DC Executive session
12. 10/12/87 NRC, Washington, DC Executive session
ACRONYMS:
C I L C ritical Items List
FMEA Failure Modes and Effects Analysis
HQ Headquarters (of NASA)
JSC Johnson Space Center
KSC Kennedy Space Center
MSFC Marshall Space Flight Center
NASA National Aeronautics & Space Administration
NRC National Research Council
presented; others remain at issue. All of the con-
cerns identifies! in those reports are discussed in
Section 5 of this report. It shouic! be noted that
NASA's safety process in general, and the current
reevaluation in particular, have been undergoing
consiclerable change following the Challenger ac-
cident anc! during the Committee's audit. Indeed,
some of the changes have resulted from the Com-
mittee's discussions with NASA officials and from
its interim reports. Thus, many of the subjects
coverer! by this report have been "moving targets"
that continuer! to change as this report was being
prepared. However, the Committee believes that
the report reflects the facts and circumstances as
of: September 1987.
2.3 ORGANIZATION OF THE REPORT
Following this introduction is Section 3, which
presents an overview of NASA's safety process for
13
NASA Headquarters, JSC, MSFC & KSC staff
Boeing Comm'l Aircraft representatives
Rockwell STS Div., Rocketdyne Div. NASA
HQ, JSC, MSFC, USAF Space Div. and
Aerospace Corp. staff
NASA Assoc. Admins. for Space Flight &
SRM&QA, NSTS Program Manager
NSTS and JSC personnel (including Mission
Operations & Astronaut personnel)
MSFC and KSC leaders and staff related to
STS
MSFC & JSC Indpndnt contractor staff,
Quant. Risk Assess. (QRA) consultants
Rockwell STS Div., Rocketdyne Div., NASA
HQ, JSC, and MSFC staff
NASA HQ & JSC NSTS personnel NASA HQ
SRM&QA personnel
NSTS Dep. Dir., Operations JSC, HQ
personnel
f Process overview, Committee
planning
SSME, Orbiter FMEA/CIL &
hazard analysis audit
Discussion of concerns; draft
first interim report
Review STS risk mangement
and operations
Overview of MSFC & KSC
FMEA/CILs & hazard analyses
QRA, Independent contractor
FMEA/CIL reviews
SSME; STS integration
activities
SRM&QA status and functions
STS integration & software
STS oprns, payloads, PCASS,
system engineering, draft
second interim report
Review & discuss information
collected
Formulate conclusions, rec-
ommendations; review drafts
Review & approve final text
NSTS National Space Transportation System
PCASS Program Compliance Assurance and Status System
QRA Quantitative Risk Assessment
SRM&QA Safety, Reliability Maintainability & Quality
Assu rance
SSME Space Shuttle Main Engine
STS Space Transportation System
USAF United States Ai r Force
the NSTS Program as the Committee unclerstancis
it. That section is provider] as a tutorial for those
who may not be familiar with this complex process.
Section 4 briefly describes the Committee's con-
ception of modern risk management, including the
essential element of objective risk assessment, and
contrasts it with NASA's safety process in general
terms.
The heart of the report is Section 5, which
presents discussion, finclings, and recommendations
regarding particular aspects of NASA's STS safety
assurance process. It comprises the results of the
Committee's audit. The section is clivicled into 11
subsections, each dealing with a different aspect of
the process (with some encompassing relatecl but
distinct topics).
Section 6 is a brief summary of the main "lessons
learned" by SCRHAAC in the course of its auclit.
These lessons, derivecl from the STS review, are
OCR for page 14
considered to be applicable to other large and
complex technological systems which, by their size
and complexity, require the involvement of several
major centers and organizations for their execution.
Finally, ~ series of appendices are provided.
Some, like Appendix A 6'Acronyms and DeEnE
Honshu, are intended as useful tools for the reader.
Others are provided as amplification or background
on various subiccts addressed in the report See the
Table of Contents for ~ complete Usting.
14
Representative terms from entire chapter:
nsts program
