Below are the first 10 and last 10 pages of uncorrected machineread text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapterrepresentative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 125
APPENDIX E
AN IMPROVED CRITICAL ITEM RISK ASSESSMENT PROCEDURE
FOR THE
NATIONAL SPACE TRANSPORTATION SYSTEM
(With an Example of Application to the 51L Field Joints)
1. INTRODUCTION
]
On May 28, 1987, a NASA representative made
a presentation to the Committee on Shuttle Criti
cality Review ant! Hazard Analysis Auclit entitled,
`'Critical Items List (CIL) Prioritization." The methoc!
discusser! was subsequently issued in moclifiec! form
as NSTS Instruction 2249 1, Reference F34. This
Instruction for the preparation of Critical Item Risk
Assessments (CTRA) provides a methoc! for prior
itizing the failure modes in the CIL. It contains
many excellent ideas and is a significant step
forward. However, the Committee has some con
cerns and some related suggestions on how to
simplify ant! clarify the method.
This Appendix also contains in Section 5 an
example of the application of trend analysis and
Probabilistic Risk Assessment (PRA) to the pre
Challenger Orings. This application, include(l here
only as an example of some applicable analysis
techniques, makes heavy use of moclern statistical
science ant] Bayesian ideas.
2. CONCERNS WITH THE CURRENT
METHOD
The Committee's concerns with the CIRA method,
as currently formulated, can be summarized as
follows:
I. In Table ~ of Reference t31 (shown here in
Attachment I) the column labeled "SEVER
ITY" DEFINITIONS really contains worst
case ciamage states.
2. In Table I, the columns labeler! SUCCESS
PATHS and STATUS CODE FOR REDUN
DANCY/BACKUP are really descriptions of
system or subsystem architectures. They affect
risk by affecting the probabilities in the last
Failure
Mode
1
2
Severity
Definition
(A) Loss
of Life
(A) Loss
of Life
Success
Paths
o
o
Redundancy/
Backup
(a) None
(a) None
125
two columns. However, the relevant informa
tion is in the probabilities themselves not in
the architecture. Any guidelines written on
how to assess the probabilities, either empir
ically or subjectively, should contain much
discussion on how success paths, redundancy
structure, ant] periodic checking strategy af
fects the probabilities in columns 4 and 5.
3. The probabilities in the last two columns of
Table ~ are qualitative ant] open to interpre
tation as to what the terms "Very Likely,"
"Likely," "Unlikely," ant! "Very Unlikely,"
mean. The two columns, which have the same
qualitative scale, appear to have different
quantitative scales associates] with them. In
column 4, '~Very Unlikely" appears to mean
something like c ~ 0  6 and "Very I=ikely"
means something like 10 i. In column 5, the
scale depends on whether or not there is
reclundancy . If there is no reclundancy, then
"Very Unlikely" means something like 102
and "Very Likely" means something like
greater than .95. But if there is redundancy,
then "Very Unlikely" may mean ]06. With
the qualitative definitions of probability, it is
quite possible that two engineers working on
two failure modes with the same severities
and probabilities would assign them to dif
ferent probability categories and therefore
produce inconsistent priorities. It is very im
portant that the probabilities have opera
tional clefinitions. Terms like "Unlikely" are
not operational definitions.
4. There is no way to produce a unique priority.
Suppose there are two failure modes, and
Table 1 is filled out as follows:
Design
Conf idence
Likelihood of
Worst Case
(iv) Unlikely
(ii) Likely
OCR for page 125
Which one should have the highest priority?
Suppose that the last two columr~s were
replaced by the following structure:
Failure
Mode
2
Probability of
Failure
Likely= 01
Unlikely = 00001
Probability of
Worst Case
Given Failure
.
Unlikely= 01
! ikely = 5
Probability of
Worst Case
0001
000005
Now it is clear that failure mode ~ presents
a higher risk.
3. PROPOSED IMPROVEMENTS
As an improvement to Reference f3I, the Com
mittee proposes the procedure described in Tahie
E1 below:
All failure modes with the same Worst Damage
State Given Lack of Redundancy or Reclunclancy
Failure wouic] be ranked by column Z.
The probabilities shown in Tahie El are for
illustration only anc! do not reflect any specific
example. In actual application, it wouIc! he highly
desirable for the analyst to include confidence limits
(or the equivalent) for each of the probabilities
listed in the tahies produced through the CTRA.
The Committee recommends strongly that such
probabilities be documented by a rationale. Many
of the facts mentioned in the current CIL "Rationale
for Retention" would be cites! in the probability
rationale but in the quantitative manner illus
trated by the example in Section 5. In addition,
facts that imply higher probabilities wouIc! also be
analyzecl. For example, the longrun frequency of
catastrophic failure for solid rocket motors of a
TABLE E1 Improved Risk Assessment Procedure
Failure
Mode
.
2
3
u
Criticality
OR
.
OR
V
Probability of
Primary Failure
During Mission

00
00
w
Probability of
Redundancy Failure
Given
Primary Failure
001 999
1
0505
01
1
126
mature design is I/50; ant! therefore I/25 for two
solic! rocket motors. A disaggregation of this
frequency by ~ aiTure mode wouic! be a useful
baseline for an analysis. How are our clesign and
failure modes different from history? For example,
the field joint is similar to Titan ITI, but also
different. The redundant Oring points to a smaller
probability, but the insulation geometry points to
a higher probability.
In Table El, failure mode 3 has the most risk,
even though it is only a Criticality JR item. For
this case, the computation of column W uses the
following estimates:
(i) There is one success path remaining after
the primary failure.
(ii) The availability of the backup is not readily
detectable ant] is checked every thirc! flight;
and the estimated availability is .99.
(iii) The probability of a secondary failure is
.05.
The formula for column W is
W= PrlBackup Available) x PrlSeconclary Failure)
+ PrlBackup not Available)
= (.99) (.05) + (.01)
= .0595
For failure mode I, there is no backup; but, it
is a relatively rare (probability = .001~ failure
mode and infrequently (probability = .01) causes
the worst damage state.
Failure mode 2 is much less risky. The compu
tation of column W uses the following estimates:
(i) There is one success path remaining after
the first failure.
x
Worst
Damage State
Given Lack of
Redundancy or
Redundancy Failure
(A)—Loss of Life
and/or Vehicle
(A) Loss of Life
n~l/~r V~hi~.l~
(A) Loss of Life
and/or Vehicle
1
y
Probability of
Worst
Damage State,
Given Lack of
Redundancy or
Redundancy Failure
01
.1
Z = (V)(W)(Y)
.
Probability of
Worst
Damage State
Event
.
00001
0000001 999
000595
OCR for page 125
(ii) The backup is readily detectable and fixed
when failed and the availability of the backup
is .999.
(iii) Given the backup, the probability of sec
ondary failure is .001—the same as the
primary.
Use of equation (~) in this case yields
W= (.9991~.001) + (.001)
= .001999
4. RELATIONSHIP BETWEEN IMPROVED
PROCEDURE AND TABLE E1
There is a strong relationship between the im
provements described in Section 3 and NASA's
Table ~ (Attachment 1 here). From the "SEVER
ITY" DEFINITIONS in column ~ of Table I, we
can deduce the following Worst Damage States:
A. Loss of Life and/or Vehicle
B. Mission is Aborted
C. Degraded Operational Capability or Early
Mission Termination or Damage to a Vehicle
System
D. Loss of Some Operational Capability of Ve
hicle, but Full Mission Duration.
E. No Operational Effect
The probability scales couic! be set up as categories
with the definitions given in Table E2.
The Committee urges the use of quantitative
definitions of probability. Even though for some
failure modes the probabilities will be assessed
subjectively, it is very important that the analyst
have art operational clefinition. To reiterate, terms
like "Unlikely" are not operational definitions. In
TABLE E2 Probability Scales For Improved Risk Assessment Procedure
aciclition, use of a quantitative probability scale
will augment the pure engineering judgment ap
proach.
The factors in Reference F3], Section 3.4, are
very relevant to assessing the Probability of Primary
Failure During Mission in Table E1. Other factors
include:
Product design certification test results
Manufacturing process qualification test re
sults
· Engineering analytical models
· Related industry data
· Etc.
The number of SUCCESS PATHS ant] the
REDUNDANCY/BACKUP scenarios given in
NASA's Table 1 (Attachment 1 to this appendix)
are very relevant to assessing the Probability of
Redundancy Failure Given Primary Failure in Table
E1.
The factors relevant to assessing the Probability
of Worst Damage State Event in Table E1 are very
similar to those listed in Reference f3], Section 3.5.
As part of the exercise of assessing this probability,
one could list all the events subsequent to redun
dancy failure that do not lead to the worst damage
state.
5. APPI ICATION TO THE DRINGS
Only as an example to illustrate the foregoing
proposal, consicler the field joint Orings prior to
the Challenger flight 51L at a joint temperature
of 31°F, which was predicted for the Challenger
flight. It is based only on a limited knowledge of
the subject derived from References A] and t2],
Center Point of Ranges of Probability Values
1 1
Description
Very Likely
Likely
Possible
Unlikely
Very Unlikely
Probability of
Primary Failure
During Mission
10 1
10 2
10 3
10 5
10 7
Probability of
Redundancy Failure
Given
Primary Failure
10 1
10 2
10 3
10 5
10 7
127
Probability of
Worst
Damage State
Given Lack of
Redundancy
or Redundancy Failure
0
5
0~
2
03
OCR for page 125
and thus must be viewed ONLY AS AN ILLUS
TRATION OF A PROCESS.
To keep things simple only one failure scenario is
considerecI. In the language of Table E1 we have:
TABLE E3 Application of Table E1 to the SAM
Field Joint
Language of Table E1
Primary failure
during mission
Redundancy failure given
primary failure
Worst damage state
Application to Field Joint
Erosion and blowhy
of the primary Oring
Failure of the secondary
Oring given erosion and
blowhy of the primary Oring
Loss of life and vehicle
The reason for considering this scenario is that
ciata are readily available. Also in Reference FI]
p. 135 it is stated that bypass erosion or blowby
was considered much more serious than just im
. .
plngement erosion.
The data set used in this analysis (see Attachment
2) is taken from pages 129131 of Reference FI].
The subset of these data user! here involves only
the actual flights and only the field ant! nozzle
joints. A useful organization of this subset is shown
in Attachment 3. In the columns labeled erosion
blowby and erosion or blowby the blanks
mean that the event (lid not occur. In the column where
labelect blowhy given erosion the blank means
there was no erosion and the zero means that there
was erosion but no blowby. Most of the data are
for the primary Orings; but the data with an
asterisk are for the secondary Orings.
5.1 Primary Failure
For primary Oring failures we consider the
scenario of erosion ant] blowby. The primary failure
probability is:
PrlPrimary Failure) = Pr{Primary Erosion)
Pr mary Primary
x Pr{Blowhy Erosions. (2)
The vertical bar in the probability expression (2)
reacls conditional on. So for example
Pr{Blowhy ~ Erosion)
would read probability of the event Blowhy
conditional on the event Erosion occurring. For
two events A ant! B a funciamental law of prob
ability is
PrlA and B) = PrlA) x Pr{B ~ A) .
5.1.1 Primary Erosion
A plot of the incidents of field joint primary O
rings with erosion is shown in Attachment 4. For
example flight 51C in January 1985 had two
field] joints with primary Oring erosion; this mis
sion experienced a joint temperature of 53° F and
a leak check pressure of 200 psi. The fitter! curves
are derived from a statistical model which allows
for possible joint temperature ant! leak check pres
sure effects.
Flight 5 lC experienced both erosion and blowhy
of the field joint. At a subsequent Flight Readiness
Review where 51C was discusser! there was a
conclucling statement I=o`v temperature enhancer!
probability of blowby (Reference tI] p. 1471.
On page H73 of Reference f21 it is statec! that
Frequency of Oring damage has increased since
the incorporation of . . . higher stabilization pres
sures in leak test procedures ... . So it is of interest
to statistically mocle! the effect of temperature ant!
leak check pressure on Oring anomalies.
pit, s') = Probability of erosion per field joint
primary Oring,
t= Joint temperature
s = Leak check pressure.
The assumptions for this statistical model are:
I. The model for pit, s) is:
{t /J(~'s)}
This is caller! a Logistic Regression moclel. The
variables a,F,~ are unknown parameters to be
estimated from the data. Different values of these
parameters represent different relationships be
tween erosion probability and (temperature,
pressure). For example, if ~ < 0, then probability
(lecreases with temperature; but if ,(3 ~ O. then
probability increases with temperature. We will
let the data determine which of these is most
likely.
2. Given pit, s'), the field joints are statistically
independent.
128
OCR for page 125
Let
aft, s) = Number of field joint primary Orings
with erosion for a launch with joint
temperature t ant] leak check pressure s.
Uncler these assumptions, the probability distri
bution of aft, s) given pit, s) is binomial with
parameters n = 6 (i.e., 6 fielc! joints) and p = pit,
s). So for k = 0, I, . . ., or 6,
Pr {X(t,s) = k j ptt,sy}
= {6) Spit s)]k;] — pit S)46k
Let the subscript i represent the ith launch in
Attachment 3. So i = I, 2, . . ., 23. Let
xi = Number of field joint primary
Orings with erosion
ti= loins temperature
si = Leak check pressure
Pi = P(ti, si)
Also let
t
X = (x I, X,, . . ., X, 3)
= (t,, t,, . . ., ten)
S = (S 1, S., . . ., S. 3).
The likelihood function, A, given the data x, is
defined as the probability of observing x conclitional
on t, s, and (a,Q,~). The variables t and s are
regarded as known variables (in standard] regression
analysis they are callec! inclepenclent variables); ant!
(`x,Q,~) are the unknown parameters. The likeli
hood function is regarcled as a function of (`x,h,~)
and is
Li a ~ A) = ~ ( 6 ) pxi ( } _ p )6 xi
Recall that Pi is a function of (`x,,B,~y).
The maximum likelihood estimates of the (a, [3,~y)
are those values that maximize the likelihood!
function. In effect, they are the values of (`x,h,~)
that make the observer] value of x the most probable
under our model.
There is a close relationship between maximum
likelihood] estimation ant! least squares. The least
squares estimates of (a,,(3,A) are those values that
. . .
mlnlmlze
where 6pi is the expected value of xi under our
moclel. If the xi's had a Gaussian (normal) distri
bution with common variance, then the maximum
likelihood estimates and the least squares estimates
would be the same. This is because the Gaussian
probability density would then be monotonically
related to the sum of squares above. However, the
probability densities of the xi's in our problem are
binomial ant] not Gaussian. Ant! it is a well
established fact in statistical science that maximum
likelihooc! estimation is usually more efficient (closer
to the truth) than least squares; so we use maximum
likelihoocI.
The results of a maximum likelihood analysis of
these ciata under the above mociel yields the values
in Table E4.
TABLE E4 Maximum Likelihood Analysis of the SRM
Field Joint Primary ORino Erosion Data
Parameter
cY
Maximum Likelihood
Estimate
78
.17
0024
_
— 90% Confidence
Interval
[ .1, 15 7]
[.28  06]
[  .01 2, 01 6]
The 90% Confidence Interval reveals the fact
that from our data we cannot learn the "true"
value of ((x,,B,A) with great precision. For example,
a Bayes interpretation of the interval [.28, .06]
for the temperature effect, A, is that given our data,
there is a .9 probability that the "true" value of
lies in the interval L.28, .064. Note that this
interval does not include the value ~ = 0 (i.e., no
effect). This means that the temperature effect is
"statistically significant;" or that there is only a
very small probability that the true value of ,l3 is
greater than or equal to zero.
Also note that there is no statistically significant
pressure effect on field joint erosion. That is because
most of the variation is explained by temperature
variation. This is curious, because in Reference L1],
blowholes caused by high pressure were cited as
a cause of erosion.
Plugging the maximum likelihood estimates into
equation (3) yields
A
(Xi 6pi)2,
i= 1
In[1 p(ti200) ] = 7.8  ( 17)t + ( 0024)(200)
= 8.3  (.17)t
129
OCR for page 125
~1 ~
1 nls 1rnplles
elf ~ (. 17)tl
p(t,200) = 1 + elf ~('7)tl (4)
The curve for 200 psi (plotted in Attachments 4
and S) is (6)p(t,200), because there are 6 field
1olnts.
The predicted probability per joint of primary
Oring erosion at 31° ~ joint temperature and 200
psi leak check pressure is
p(31,200~=.95 [probability of
 Primary Erosion
The 90 percent confidence interval for the "prob
ability of primary Oring erosion" is shown in
Attachment S and is t.S, 1.04. This shows that the
extrapolation to 3~° E; introduces considerable
uncertainty in the estimate. The propagation of
this uncertainty to the final result will be discussed
in Section 5.5.
S.1.7 Pri~7zar~' Blc''~by Gin Priorly Erosions
The frequencies per primary Oring of blowby
given erosion were extracted from Attachment 3
and are given in Table ES. An analysis of the
blowby given erosion data shows no statistically
significant effects of joint type, joint temperature,
or leak check pressure. So we use the estimate
p ~ Primary Blowby ~ Primary Erosion
r ~ for Field Joint I for Field Joint J
t primary Blowby Primary Erosion]
= Pr] for Field or for Field or
t Nozzle Joint Nozzle loins J
= .292
TABLE E5 Frequency per Primary ORing of
Blowhy Given Erosion
1 1
Joint .
Field
Plugging (5) and (6) into (2) yields
Pr{Primary Failure} = (.95) (.292)
= .277
It is revealing to look at the frequency of primary
Oring blowby, given no erosion, in Table E6.
TABLE E6 Frequency per Primary ORing of
Blowhy Given No Erosion
1 1
Joint

Field
Nozzle
Frequency
Joint per ORing
2
Field 7= 286
Nozzle ~ 7= 294
I, Field plus ! 7  .292
I Nozzle 1 24
Frequency
per ORing
2= 50
1= 20
5
Field plus 7 = 286
Comparison with Table ES shows that there is
a strong statistical dependence between primary
Oring erosion and blowby particularly for the
field joint. For the field joint, blowby was rare
(frequency = .015) when there was no erosion,
but not rare (frequency = .286) when there was
erosion. no
PrlBlowby ~ Erosion) >> PrlBlowby ~ No Erosionl,
which implies strong statistical dependence. If blowhy
and erosion were statistically indepenclent, then
these two conditional probabilities wouic! be the
same.
The strong statistical dependence shown above
suggests that erosion might be a causal factor for
blowhy. This idea is born out by field data ant!
various experiments. Experiments (reference L21, p.
H82) showed that an Oring will fad! to seal with
an erosion depth of 0.15 inches. In flights 51C
(6) an`] 5~B, there was both erosion and blowby of
the field primary Oring, and a heat effect or erosion
of the secondary Oring. In both cases, the erosion
of the primary Oring was among the worst ero
sions experienced (reference F2], p. H7l, H72) as
measured by crosssectioned depths of 0.038 and
0.171 inches, crosssectionecl perimeters of 130°
and 360°, and a top view of affected lengths of
58.75 anc! 12 inches. This implies that blowby can
be caused by excessive erosion. So our model that
the higher the probability of primary Oring ero
sion, the higher the probability of primary Oring
blowby, is plausible.
130
OCR for page 125
5.2 Probability of Secondary Failure
Next we consicler the Probability of Redundancy
Failure Given Primary Failure in Table E~. This
would be failure of the secondary Oring. Our
n~ode! of secondary failure is secondary erosion
and failure given primary erosion and blowhy.
Therefore
Pr ~ Secondary ~ Primary Erosion
Failure I end Blowby J
_ p ~ Secondary ~ Primary Erosion
r ~ Erosion I and Blowhy
x Pr{SeCondary  Secondaryl
Failure ~ Erosion i. (7)
A statistical analysis of secondary erosion given
primary erosion and blowby shows no statistically
significant effects of joint type joint temperature
or leak check pressure. So we use the estimate from
Table E7 below:
~ . Primary Erosion and]
Pr] Secondary Erosion Blowhy ~ 2
t for Field joint for Field Joint
Secondary Erosion Primary Erosion and]
= Pr] for Field or Blowhy for Field
t Nozzle loins or Nozzle Joint J
Joint
Field
(8)
TABLE E7 Frequency per SRM Joint of
Secondary ORing Erosion Given Erosion and
Blowhy of the Primary ORing in 23 Flights Prior to
Challenger 51L
1
Secondary Erosion
Given Primary Erosion and Blowbv
..._ ~
Nozzle i
Field plus 2 = 286 1 of Secondary ORing
Time After ignition
Ignition Transient:
O to 170 ms
1 70 to 330 ms
330 to 600 ms
The estimation of
Pr  Secondary  Secondary  Steady State:
Failure  Erosion J 60 ms to 2 min
131
in equation (7) presents some difficulties because
there were no secondary failures before 51~. So
we shall express the solutions parametrically in
terms of the parameter
A4 = PrISecondary Failure~Secondary Erosion) (9)
The state of knowledge curve (described in Appen
dix D) for A4 could be determined on the basis of
engineering information. Examples of relevant en
gineering information which was available before
51L are:
loins rotation created doubt about the ability
of the secondary Oring to seal. In fact the
Oring failure mode was considered Critical
ity 1, not Criticality 1R. So, officially, the
FMEA did not recognize the secondary 0
rings as providing redundancy. However, ac
cording to Reference f 1 l, p. 126, NASA
management and Thiokof still considered the
joint to be a redundant seal because there
were flights where the primary Oring failed
and the secondary Oring sealed in accord
ance with its design intent.
In July 1985, a ThiokoT engineer, in light of
the 51B nozzle joint secondary Oring ero
sion, expressed his concern that if the same
scenario should occur in a field joint (and he
believed it could), then it would be a "jump
ball" as to the success or failure of the joint
because the secondary Oring could not re
spond to the clevis opening rate and might
not be capable of pressurization (i.e., in the
51E design, which has been changed in the
redesigned joint). (See Reference F1], p. 139.)
3. The qualitative assessment (Reference t2], p.
H84, Chart 166) of the probability that the
field joint secondary Oring will fait given
erosion penetration of the primary Oring
seal is listed in Table E8.
TABLE E8 Qualitative Probability of SRM
Secondarv ORino Failure Given Erosion Penetration
Qualitative Probability of
Secondary ORing Failure
low
medium
high
high
OCR for page 125
There were only two incidents of secondary
Oring erosion in a field] joint. So there was
no solid statistical evidence that the secondary
Oring would work given primary Oring
failure; i.e. nothing like ~ 000 successes with
out a failure. Also as seen in Table E8 the
probability of secondary Oring failure cle
F'ends on time after ignition.
r. The night before the Challenger launch a
chart provicled to NASA by a Thioko! engi
neer about the possible temperature effect on
the Orings (Reference tI] p. 89 Chart 22)
included concerns that: (i) lower temperature
of the Orings would result in a change in
their sealing timing function which would
result in higher Oring pressure actuation
time; (ii) if the actuation time increases
threshold of secondary seal pressurization
capability is approached; (iii) if threshold is
reached then secondary seal may not be
capable of being pressurized.
Plugging (8) and (9) into (7) yields
Pr ( Secondary
Failure J
= (.286 ~ As
Probability of ) ~ ~ 0)
Secondary Failure
5.3 Probability of Worst Damage State Given
Redundancy Failure
If the Felt! joint seal were to fad! there is some
possibility that the crew anal vehicle wouIcl survive.
For example the seal might fad! right before the
solid rocket motors completed their burn. How
ever the chances are very high that such a failure
shouic! it occur would be earlier in the flight. This
suggests a value approaching ~ for the probability
of Toss of life and vehicle given total seal failure.
Thus the closest probability value of ~ from Table
E2 column Probability of Worst Damage State,
is selectee! in this example.
5.4 Probability of Worst Damage State Event
Using the estimates derived above the value for
column Z in Table El is
Z= (.277~.286jA4 l'Probabilityper~oint:
~ of Worst Damage J A4 =
= (.0792)A4 .
5.5 Probability of At Least One Field Joint Failure
The estimated probability in Section 5.4 is for
only one field joint. The estimated probability of
held joint failure for the mission is
Pry Mission Field
~ joint Failure J
= ~ _ prtNO Field
= ~—t!—(.0792) A446
(Probability of Failure) (12)
It is clear from the statistical analyses that there
is uncertainty in the estimates of the probabilities
used. For example the 90 percent confidence in
tervals in Table E4 show that the parameter
estimates are uncertain. Also the .286 estimate in
equation (8) was based on two failures out of
seven and is therefore uncertain. The uncertainty
associated with equation (12) is quantified in At
tachment 6. The two almost linear curves form a
90 percent confidence interval for the "probability
of mission fielcl joint failure," conditional on the
value of As. So if the value of A4 is .25, for example,
then the conclitional 90 percent confidence interval
is t0.010,.1184.
A subject matter expert could analyze the rele
vant engineering information and assess a state of
knowledge curve for 4. If this curve were centered
on A4 = .25 with a considerable variance, then the
unconditional 90 percent confidence interval for
the "probability of mission field joint failure,"
would be much wider than the F.010, .118] interval
cited above.
The 90 percent confidence intervals in Attach
ment 6 were derived by a Bayesian analysis (see
Appendix D for more discussion). For the STL
environment (e.g., 31° F), we define the following
Tong run "true" frequency probabilities:
H
132
= Probability of mission field joint failure
per mission; and for a given field joint,
~ = Probability of failure
A I = Probability of primary Oring erosion
A, = Probability of primary Oring blowby
given primary Oring erosion
Probability of secondary Oring erosion
given primary Oring erosion and
blowhy
Probability of secondary Oring failure
given secondary Oring erosion.
OCR for page 125
Our mocie! is that ~ = ~  ~ ~  l)6
4
d)= 11 A.
i = I
Let /i =
A,A,A~
then D= 1  [1—AA4]6
/1 ~
(14)
.
(15)
(16)
In the Bayesian analysis we assume that, condi
tional on our data, Al, A,, and As are statistically
independent. This is reasonable because the Ai's
are successive conditional frequencies. The state of
knowledge curves for the inclividual Ai's were
derived from Bayesian analyses assuming "flat" a
priori state of knowledge curves. This means that
we die] not use much information external to the
ciata in Attachment 3. For example, we macie no
attempt to use the engineering models described
in, e.g., Reference F2], p. H60. This may have
been possible by modeling the uncertainties in the
variables of the engineering models. This idea was
curves for the Ai's through equation (151. This was
done by a discrete probability approximation tech
nique. The implied 90 percent confidence interval
for ~ is t.007, .0824.
The upper and lower curves in Attachment 6 are
clerived from equation (16) and are
6~(A4) = 1—f 1  (.082) A4] 6
6~(A4 ) = 1—f 1  (.007) A4] 6
REFERENCES
(17)
t1] Report of the Presiclential Commission on the
Space Shuttle Challenger Accident, Volume 1,
June 6, 1986, Washington, D.C.
t2l Report of the Presiclential Commission on the
Space Shuttle Challenger Accident, Volume 2,
June 6, 1986, Washington, D.C.
suggested by Feynman (Reference t2l, Appendix
F). The uncertainties in the engineering mociels are
a. l . . . f3l National Space Transportation System, "In
a possible explanation as to why the models old . . r ,~ . c, . A. .
not predict very well.
Finally, the state of knowledge curve for A was
clerive(l by propagating the state of knowledge
6
struct~ons tor Preparation ot critical item Mask
Assessment (CIRA)," NSTS 2249 1, June 19,
1 987.
OCR for page 125
ATTACHMENT 1 NASA's Proposed CIRA Technique.
0 uJ u
<: ~ ~ ~ `,, ,,, ,x, ~ ·r~
`1` O Off LL ~ L1~ ~ Us U ~ ~ <~ m ~ ~ us
Up O _' ~ U ~ > ~ U ~ ~ ~ Lo ~ ~ U ~ ~ ~ U Ud
~ ~ ~ . , ~ ~ .
LL .2 := := .>
I ~ he Z
~ Z I ~
Z ~ I 3 ~ Y
X ~ Z ~ ~ m us Z
_ Y O Z
> ~ ~ >
car 1 
car ~
Z up ~ ~ up up
uJ ~ ~ ~ ~ C)
Z ~ — — ~ ,x, ,~, LL
O LL Z Z Z Z Z
Z O O O O 0
O ~ U U U U'
~J L1J ~ L1J UJ L~
Z Z ~ ~ C) ~ ~ ~ O
1 ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
L1J ~ o ~ ~ ~ ~ :) v~ ~) I
,= ,~ _ u~ ~ L., ~
, L~ ~ O L~ O ~ J
~ . ~ ' .
LLI . = _
U
 O ~ ~, m ~ Z ~ ~ Z ~  z ,~ Z ,~ C ~ c, ~  U O
1: L~ O Z ~ ^ ~ I UJ ~ 0 z ~ ~ L~ ,= z ~ ^ ~ ~ ~ ~ ~ ~ Z
U C ~ ~ ~ Z ~ m ~ ~ ~ ~ ~ ~ = ~ ~ ~ ~ ~ O z ~ C ~
O ~ _ ~ ~ ~ O O ~ O ~ `~ ~ ~ O O ~ u~ ~ ~ c O ~ C) ~ uJ 2 I _ ~ C
Z ~ 0 m ~ m ~ u ~ ~ m ~ m Lu 1l u m ~ ~ ~ c u ~ 1l m c~ ,
, . .
Q
~ ~ V~ ~ ~
m ~ z u~
I ~ ~ _ c~
V ~ ~ V L~ ~( ~= 3
~ ~ O
U
— O 3 0 ~ ~ ' ~ · r . z I
<( ~ O ~ O < 0 z O 5 ~ O
v~ v~ I v~ ~ O ~ <~ ~ ~ ~' ~ <~ ~ ~ ~ ,= ~ U ~ O ~ ~ u ~ U ~I
L~ ~ O Lu C~ ~C > Z ~ ~ 4< LL ~ Z O L~ t~~ ~ t1~ Ud C~ O ~ ~ 11
m ~ ~ ~ _ ~ ~ Z ~ ll ~ ~ ~ ~ ~ O c~ ~ ·= O O
,
m
134
OCR for page 125
ATTACHMENT 2 0Ring Anomall" Compared with Jolnt Temperatur" and Leak Check Pressurc
Flight (Solid Prossurc Jolnt
or Rockot JolnV (In pal) Tomp.
Motor Date Booster) ORlaa Fleld Nozzlc Eroslon Blowbv °E
DM1 07/18m
DM2 01/18n8
DM3 10/19n8
DM4 OV17/79
QM1 07/13/79

NA NA   84
NA NA   49
NA NA   61
NA NA   40
NA NA   83
OM2 09/27179   NA NA   67
QM3 OV13/80   NA NA   45
STS1 04/1V81   50 50   66
STS2 11/1V81 (Right) AM FloldlP~ma~ 50 50 X  70
STS3 03m/82   50 50   69
STS4
DM5
STS5
QM4
STS6
STS7 06/18/83
STS8 08/30/83
STS9 11/28/83
STS 41 B OV03/84
06t27/82 unknown: hardware lost at "a
10t21/82
11/11/82
03/21/83
04/04/83
(RIS]ht)
(Len)
(Right)
(Len)
STS 41 C 04J06/84 (Right)
(LoR)
(Right)
(Right)
STS 41 D 08t30/84
STS 41 G 10/05/84
DM6 10125/84
STS 51A 11/08/84
STS 51 C 01/24/85

Nozzle/Prlmary
Nozzle/Prlmary
Nozzle/Prlmary

(LeR)
(Right)

(Rlght)
(Right)
(Right)
(LeR)
(Leh)
Nozzle/Prlmary
Forward Fleld/
Prlmary
Nozzle/Prlma~y
Att FleldfPrlmary
IgnNorfPrlmary
FonNard
Fleld/Prlmary
NozziolPrlmary
l~nNerfPrlma~y
Inncr GaskcV
Prlmary
Center Fleldt
Prlmary
Ccntcr Fleldt
Secondaly
Nozzle/Prlmary
Forward Fleld/
Prlmary
Nozzis/Prlmary
Dash () denotes no anomaly; NA denotes not appilcable.
See cnd of attachment for footnotes.
50 50
NA NA
50 50
NA NA
50 50
50 W
50 50
100 50
100(2) 100
200 100
NA NA
X
(1)
(1)
_ _
X
200 100 X
200 100 X
200 100 (3)
NA NA 
200
200
NA
200
NA
200
200
100 X
100 X
NA
100
NA
100
100
200 100
200 100
200
200
135
100
100

80
58
68
60
67
67
72
73
70
57


_ _
57
63
63
X 63
70
X 70
 X 70
78
X X
X X
(4) —
_ X
X X
X
52
67
53
53
53
53
53
OCR for page 125
ATTACHMENT 2 (continued)
Flight (Solid Pressure Jolnt
or Rocket JolnV (In p~l) Temp.
Motor Dato Booster) O~R~ Fleld Nozzle Eroslon Blowbv °£
STS 51D 04/1 V85 (Right) Nozzle/Prln~ary 200 200 X  67
(RIght) l~niter/Prlmary NA NA  X 67
(Loffl Nozzh/Prlmary 200 200 X  67
(Lett) l~niterIPrlmary NA NA  X 67
STS 51B 04/29/85 (RIght) Nozzh/Prlmary 200 100 X  75
(L.n) Nozzle/Prlmary 200 100 X X 75
(Len) Nozzl~Socondary 200 100 X  75
DM7 05/09/85 Nozzle/Prlmary NA NA X  61
STS 51G 06/17/85 (RIght) Nozzle/Prlmary 200 200 X (5) X 70
(Left) Nozzle/Prlmary 200 200 X X 70
(Lett) Igniter/Prlmary NA NA  X 70
STS 51F 07/29/85 (RIght) Nozzle/Prlmary 200 200 (6)  81
STS 511 08127/85 (Leff) Nozzle/Prlmary 200 200 X (7)  76
STS 51J 10/03/85  200 200   79
STS 61 A 10/30/85 (RIght) Nozzle/Prlmary 200 200 X  75
(Len) ARFlel~P~ma~ =0 ~  X 75
(Lett) Center Fleld/
Prlmary 200 200  X 75
STS 61B 11126/85 (Right) No~l~Prlma~ 2  200 X _ 76
(Len) No~l~Prlma~ 200 200 X X 76
STS 61C 01/1V86 (Right) Nozzlc/Prlmary 200 200 X  58
(Len) AM FleldJPrlmary 200 200 X _ 58
(LMt) NozzlelPrlmary 200 200 _ X 58
STS 51L 01/28/86 200 200 31
(1) On STS6, both nozzl" had a hot gas path detected In thc putty with an Indlcatlon of heat on thc
primary Orlng.
(2) On STS9, onc ot thc right Solid Rocket Boostcr fleld ~olnts was pressurized at 200 psl after a
destack.
(3) On STS 41C, btt aft flold had a hot gas path detected In thc putty wIth an Indlcatlon ot heat on
thc primary Orlng.
(4) On a center fleld Jolnt of STS 51C, soot was blown by thc prlmaty and there was a heat effect on
the secondary.
(5) On STS 51G, right nozzle had aroslon In ~o places on thc primary Orlng.
(6) On STS 51F, right nozzle had hot gas path detected In putty with an Indlcatlon of heat on thc
primary Orlng.
(7) On STS 51l, Ictt nozzle had croslon In two plac" on thc primary Orlng.
136
OCR for page 125
rat
oS88 8888 8
~ ~ ~ ~ ~ _ _ _
o o o o o o o o o o 8 o o o 8 o 8 g 8 8 8 g o 8
~ ~ ~ ~ In ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
~ 3~$~` ~~G ~~` ~~~ ~0~
81 ~ ~ ~ ~ ~ 0
=
~ _
o
4.
o
._
._
_
C o
._
o
~ 1
_
_
Io, o
. .
g In ~
o
o o
.
.
o
.
I. 8
i,

~ 6'sm~
o ~ o
_ ._ .
o
8 o o
~ _
1
o
._
8
~ ~ ~ CY ret rut rat ~ ~ ~ ~ ~ ~ ~ ~ us ~ us us us Ut us ~0 %0
0 0 co a~ oo oo a) ca 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
t~J ~J N ~ ~ 0 0 0 ~1 ~ O 1^ 0 ~ ~J ~ ~ ~ ~ ~ O ~ ~ 0
~ ~ ~ t~J ~ O ~ ~ I%J O O ~ O O ~J ~ ~J _  1 ~J O  1 ~ ~ ~J
C) O ~ O ~ O O O ~ O O O ~ ~ O O O O O O ~ ~ ~ O O
4~
CJI

_ ~ ~ ~ ~
o
O O 0 ~i
137
OCR for page 125
ATTACHMENT 4 Occurrence of Field Joint Primary Orings with Erosion.
3.0
2.5 _
In
z 2.0
LL
At
id 1.5
a
a:
m
1.0
0.5
0~0
.
.
\~e $
.
\
\
\ ~ ~
\
\ ~ ~
_ \ ~
\ ~ ~
.
v.
50 55
ATTACHMENT 5 Maximum Likelihood Estimate and 90%
Confidence Interval for the Number of Field Joint Primary
Orings with Erosion at 200 psi.
.,
5
4
Q
3
3
at
.
1
o
·.
·.
30 40 50
Pressure
— Data
50
100 ~
200 #
\~#` # #
.
.
\ ..
N."
me.
·.
a'
,~
60 65 70 75 80 85
TEMPERATURE
ATTACHMENT 6 90 Percent Confidence Interval for the
"Probability of Mission Field Joint Failure," as a Function
of A4.
.
6 _............
it\ : 
\
J _ \ ..
2 ~ . i.
. \
....
.25
.20


o
o.
In
~ .10
o

Q
o
.15 ~
/
90% Confidence
Interval
.05
70
Temperature
80
138
~4
Probability of Secondary Oring Failure
Given Secondary Oring Erosion