An Assessment of Space Shuttle Flight Software Development Processes

Committee for Review of Oversight Mechanisms for Space Shuttle Flight Software Processes

Aeronautics and Space Engineering Board

Commission on Engineering and Technical Systems

National Research Council

National Academy Press
Washington, D.C. 1993



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
An Assessment of Space Shuttle Flight Software Development Processes An Assessment of Space Shuttle Flight Software Development Processes Committee for Review of Oversight Mechanisms for Space Shuttle Flight Software Processes Aeronautics and Space Engineering Board Commission on Engineering and Technical Systems National Research Council National Academy Press Washington, D.C. 1993

OCR for page R1
An Assessment of Space Shuttle Flight Software Development Processes NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the panel responsible for the report were chosen for their special competencies and with regard for appropriate balance. This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Frank Press is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Frank Press and Dr. Robert M. White are chairman and vice-chairman, respectively, of the National Research Council. This study was supported by Contract NASW-4003 between the National Academy of Sciences and the National Aeronautics and Space Administration. Library of Congress Catalog Card Number 93-84549 International Standard Book Number 0-309-04880-X Available in limited supply from: The Aeronautics and Space Engineering Board 2101 Constitution Avenue, N.W. Washington, D.C. 20418 Additional copies available for sale from: National Academy Press 2101 Constitution Avenue, N.W., Box 285 Washington, D.C. 20055 1-800-624-6242 or (202) 334-3313 Copyright 1993 by the National Academy of Sciences. All rights reserved. Printed in the United States of America First Printing, June 1993 Second Printing, November 1993

OCR for page R1
An Assessment of Space Shuttle Flight Software Development Processes COMMITTEE FOR REVIEW OF OVERSIGHT MECHANISMS FOR SPACE SHUTTLE FLIGHT SOFTWARE PROCESSES Nancy G. Leveson, Chair, Boeing Professor of Computer Science and Engineering, University of Washington Robert N. Charette, Chairman, ITABHI Corporation, Fairfax, Virginia B. A. Claussen, Executive Vice President, CTA INCORPORATED, Denver, Colorado Carl S. Droste, Manager, Flight Control Systems, Lockheed Fort Worth Company, Fort Worth, Texas Roger U. Fujii, Operations Manager, Systems Technology Operation, Logicon, San Pedro, California John D. Gannon, Professor of Computer Science, The University of Maryland, College Park, Maryland Richard A. Kemmerer, Professor of Computer Science, The University of California, Santa Barbara, California Robert O. Polvado, Senior Scientist, Office of Research and Development, Central Intelligence Agency, Arlington, Virginia Willis H. Ware, Senior Member, Corporate Research Staff, The RAND Corporation, Santa Monica, California Wallace H. Whittier, Program Engineering Manager, Lockheed Missiles and Space Company, Sunnyvale, California Staff Martin J. Kaszubowski, Study Director JoAnn C. Clayton, Director, Aeronautics and Space Engineering Board Christina A. Weinland, Senior Project Assistant Maria M. Kneas, Project Assistant

OCR for page R1
An Assessment of Space Shuttle Flight Software Development Processes AERONAUTICS AND SPACE ENGINEERING BOARD Duane T. McRuer, Chairman, President and Technical Director, Systems Technology, Inc., Hawthorne, California Steven Aftergood, Senior Research Analyst, Federation of American Scientists, Washington, D.C. James M. Beggs, Senior Partner, J.M. Beggs Associates, Arlington, Virginia John K. Buckner, Vice President, Special Programs, Lockheed Fort Worth Company, Fort Worth, Texas Ruth M. Davis, President and Chief Executive Officer, Pymatuning Group, Inc., Alexandria, Virginia Wolfgang H. Demisch, Managing Director, UBS Securities, New York, New York Owen K. Garriott, Vice President, Space Programs, Teledyne Brown Engineering, Huntsville, Alabama John M. Hedgepeth, President, Digisim Corporation, Santa Barbara, California Takeo Kanade, Professor of Computer Science, Robotics and Electrical Engineering, Carnegie Mellon University, Pittsburgh, Pennsylvania Jack L. Kerrebrock, R.C. Maclaurin Professor of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, Massachusetts Bernard L. Koff, Executive Vice President, Engineering and Technology, Pratt & Whitney, West Palm Beach, Florida Robert G. Loewy, Institute Professor, Aeronautical Engineering and Mechanics, Rensselaer Polytechnic Institute, Troy, New York John M. Logsdon, Director, Center for International Science and Technology Policy, Space Policy Institute, George Washington University, Washington, D.C. Robert R. Lynn, Bell Helicopter Textron, Euless, Texas Frank E. Marble, Richard L. Hayman and Dorothy M. Hayman Professor of Mechanical Engineering and Professor of Jet Propulsion, Emeritus, California Institute of Technology, Pasadena, California Garner W. Miller, Retired Senior Vice President for Technology, USAir, Naples, Florida Harvey O. Nay, Retired Vice President of Engineering, Piper Aircraft Corporation, Marysville, Washington Frank E. Pickering, Vice President and Chief Engineer, Aircraft Engines, General Electric Company, Lynn, Massachusetts Anatol Roshko, Theodore von Karman Professor of Aeronautics, California Institute of Technology, Pasadena, California Alfred Schock, Director, Energy System Department, Fairchild Industries, Germantown, Maryland Thomas P. Stafford, Vice Chairman, Stafford, Burke, and Hecker, Inc., Alexandria, Virginia

OCR for page R1
An Assessment of Space Shuttle Flight Software Development Processes Martin N. Titland, Chief Operating Officer, CTA INCORPORATED, Rockville, Maryland John D. Warner, Vice President, Computing, The Boeing Company, Seattle, Washington Staff JoAnn C. Clayton, Director Martin J. Kaszubowski, Senior Program Officer Allison C. Sandlin, Senior Program Officer Noel E. Eldridge, Program Officer Paul J. Shawcross, Program Officer Anna L. Farrar, Administrative Associate Christina A. Weinland, Administrative Assistant Susan K. Coppinger, Senior Secretary Maria M. Kneas, Senior Secretary Maryann Shanesy, Senior Secretary

OCR for page R1
An Assessment of Space Shuttle Flight Software Development Processes This page in the original is blank.

OCR for page R1
An Assessment of Space Shuttle Flight Software Development Processes FOREWORD The National Aeronautics and Space Administration (NASA) not only leads the world in space exploration and space science, but, dating back to the early space flights in the 1960s, it has led the world in the use of computers to control complex systems. While others were struggling to automate relatively simple business applications, NASA was stretching the technological envelope to build real-time computer systems to control complicated spacecraft and their support systems in programs such as Gemini, Apollo, and the Space Shuttle. Just as the Shuttle stretched the limits of the technology of its time, current projects such as Space Station Freedom and the Earth Observing System stretch the limits of technology today. In order to successfully build these future space systems, NASA needs not only to be at the technological forefront but to go beyond the state of the art and lead the world in software engineering. After the Challenger accident, the Rogers Commission Report made many recommendations for change at NASA and suggested that, after a reasonable time, a National Research Council (NRC) Committee be formed to evaluate the progress that had been made toward implementation of those recommendations. This latter committee was formed in 1988 and recommended that NASA adopt Independent Verification and Validation (IV&V) of the Shuttle software. The NRC's recommendation was later echoed by other reports and NASA ultimately instituted a fairly robust IV&V effort. Over time, that effort was reduced due to resource constraints and because of the belief that the maturity of the software reduced the need for such a robust oversight activity. Our committee was formed at the beginning of 1992, at the request of NASA, to reevaluate the need for IV&V and to investigate other aspects of NASA's software development and oversight processes. It is, of course, easy to be critical; we want to stress that we found the software and software development procedures for the Space Shuttle to be, in the main, excellent. However, the requirements of space science, applications, and exploration demand that the software be as good as possible. This report describes some ways in which we feel NASA can improve its software oversight activities to continue the successful operation of the Space Shuttle for as long it continues to be a part of the nation's space launch infrastructure. Our committee met over a period of 12 months, conducting interviews, listening to presentations, submitting questions for NASA and its contractors to answer, and reading copious amounts of material. I would personally like to thank the members of the Committee for their hard work.

OCR for page R1
An Assessment of Space Shuttle Flight Software Development Processes I would also like to thank the NASA and contractor personnel who did their best to provide us with the information we needed for the investigation (see Appendix A). Finally, we could never have completed this project without the hard work and dedication of the staff of the Aeronautics and Space Engineering Board (ASEB). I would especially like to thank the Director of the ASEB, JoAnn Clayton; the senior project assistant, Christina Weinland; the project assistant, Maria Kneas; and the study director, Marty Kaszubowski, whose technical expertise, hard work, organizational skills, and sense of humor are responsible for the success of this study. Dr. Nancy G. Leveson Chair, Committee for Review of Oversight Mechanisms for Space Shuttle Flight Software Processes

OCR for page R1
An Assessment of Space Shuttle Flight Software Development Processes CONTENTS     Acronyms and Abbreviations   xi     Figures and Tables   xii     EXECUTIVE SUMMARY   1  PART 1:   OVERVIEW AND BACKGROUND       1.  Overview of the Study   19      Introduction,   19      The Committee's Task,   20      Contents of this Report,   21      Previous Studies,   22      The Flight Software Challenge,   25     2.  Independent Verification and Validation of Critical Software   29      Introduction,   29      Orientation,   30      Scope,   31      Independence,   31      IV&V in the Shuttle Program,   38     3.  The Space Shuttle Flight Software Development Process   39      Introduction,   39      The Software,   39      The Process,   40  PART 2:   FINDINGS AND RECOMMENDATIONS       4.  The Space Shuttle Flight Software Verification and Validation Process   53      Introduction,   53      NASA Guidelines and Standards,   54      Off-Nominal Cases,   55      System-Level Software V&V,   56      The Independence of IV&V,   58

OCR for page R1
An Assessment of Space Shuttle Flight Software Development Processes     5. The Silent Safety Program Revisited   61      Introduction,   61      Software System Safety,   63      Software Safety Standards,   65      Software Safety Procedures,   67      Personnel,   71      System-Safety Organizational Roles and Responsibilities,   72     6.  Organizational Issues   77      Introduction,   77      Documenting the Process,   77      Organizational Roles and Responsibilities,   79      Policies, Guidelines, and Enforcement,   84     7.  Final Thoughts and Future Considerations   87      Introduction,   87      Gathering the Lessons Learned,   87      Contract Reporting Requirements,   89      Organizational Learning,   89      Establishing State-of-the-Art Capabilities Within NASA,   91     Biographical Sketches of Committee Members,   93     Bibliography,   95     Appendixes    A.   Study Participants,   101  B.   Statement of Task,   105  C.   Interim Report of the Committee for Review of Oversight Mechanisms for Space Shuttle Flight Software Processes: Independent Verification and Validation for Space Shuttle Flight Software,   107  D.   Overview of ASET IV&V Methodology,   131  E.   Flight Software Verification and Validation Requirements,   139

OCR for page R1
An Assessment of Space Shuttle Flight Software Development Processes Acronyms and Abbreviations BFS Backup Flight Software — The software, developed by Rockwell/Downey, that monitors the progress of the primary software and intervenes in the case of a severe error that disables the primary system. Code Q Code Q — Another name for the headquarters Safety and Mission Quality (S&MQ) Office. Each NASA headquarters office is given a code designation along with its formal name (e.g., the Development Office is Code D, the Space Station Office is Code S). In this case Code Q is the designator that corresponds to the S&MQ Office. CR Change Request — An official request by a member of the Shuttle flight software community to change the software to add to, or simplify, its functionality. DR Discrepancy Report — An official request by a member of the Shuttle flight software community to change the software because an error has been identified. GPC General Purpose Computers — The set of five independent computers used to run the primary and backup software. IV&V Independent Verification and Validation JSC Johnson Space Center — The NASA center at which the bulk of the software development and assurance activity takes place. MSFC Marshall Space Flight Center — The Marshall Space Flight Center is responsible for developing and assuring the software that controls the Space Shuttle Main Engines. NASA National Aeronautics and Space Administration OI Operational Increment — A planned update to the flight software. Updates occur approximately every year and each OI requires approximately 28 months to completely develop and test. PASS Primary Avionics Software System — The primary on-board software developed by IBM. SASCB Shuttle Avionics Software Control Board — The NASA body that is ultimately responsible for the safety and effectiveness of the flight software. S&MQ Safety and Mission Quality — The headquarters office that is responsible for NASA wide safety and quality activities. SR&QA Safety, Reliability, and Quality Assurance — The safety offices at the Johnson Space Center and the Marshall Space Flight Center. SSMEC Space Shuttle Main Engine Controller — The software system used to control the actions of the Space Shuttle main engines. The SSMEC is developed by Rocketdyne for the Marshal Space Flight Center. V&V Verification and Validation

OCR for page R1
An Assessment of Space Shuttle Flight Software Development Processes Figures and Tables  Table 1-1 Functions Covered by IV&V, 24  Table 1-2   Operational Increment Change History,   27  Figure 2-1a   Classical IV&V,   34  Figure 2-1b   Modified IV&V,   35  Figure 2-1c   Internal IV&V,   36  Figure 2-1d   Embedded IV&V,   37  Figure 3-1   The Software Development Process,   41  Figure 3-2a   The Flight Software Definition Phase,   42  Figure 3-2b   The Flight Software Development Phase,   43  Figure 3-2c   The Flight Software Mission-Preparation Phase,   44  Figure 3-3a   Block 1 Space Shuttle Main Engine Controller Requirements Definition Roadmap,   45  Figure 3-3b   Block 1 Space Shuttle Main Engine Controller Software Development Roadmap,   46  Figure 3-3c   Block 1 Space Shuttle Main Engine Controller Verification/Validation/Certification Roadmap,   47  Figure 3-3d   Block 1 Space Shuttle Main Engine Controller Mission Readiness Roadmap,   48  Figure 5-1   The Office of Safety and Mission Quality,   74  Figure 6-1   The Requirements Definition Phase,   83