Advancing Aeronautical Safety: A Review of NASA's Aviation Safety-Related Research Programs (2010)

The first task in the request for this study calls for an assessment of whether NASA’s safety-related research programs have well-defined, prioritized, and appropriate research objectives. The committee interpreted this task as a congressional interest in knowing whether NASA has well-founded objectives to guide its aviation safety research. Addressing this task thus requires an understanding of what these objectives are, how they were established and prioritized, and how they are being used.

At the highest level, NASA’s aviation safety research is guided by the agency’s overarching mission for aeronautics research. This mission has changed in recent years, particularly in the safety domain. Having significant influence over the current mission is the National Aeronautics Research and Development Policy (2006)¹ and the National Plan for Aeronautics Research and Development and Related Infrastructure (2007),² both issued by the Office of Science and Technology Policy. The first, referred to as the “National Policy,” assigns roles to each federal agency engaged in aeronautics research and development (R&D), defining the timeframe and breadth of their research efforts. The second, referred to as the “National Plan,” identifies a series of near- to long-term aviation safety challenges that the FAA, NASA, and other federal government agencies are expected to address commensurate with their research roles.

NASA’s aviation safety research mission has evolved over the past dozen years in response to changing policy guidance, from the 1997 report by the White House Commission on Aviation Safety and Security to the more recent National Policy. The National Plan sets forth fundamental safety challenges intended to inform and guide NASA’s aviation safety research, as well as that of the FAA and other federal government agencies. In addition to these national policies and plans, NASA aviation safety research programs receive input and guidance from a number of other sources, including the National Research Council’s (NRC’s) 2006 Decadal Survey of Civil Aeronautics,³ the Joint Program Development Office (JPDO), and the Commercial Aviation Safety Team (CAST). The committee

examined and assessed all these sources, as well as the process employed by NASA to prioritize safety needs for the programming and resourcing of its safety research programs.

NASA has an important role in aeronautics research that dates back to its predecessor agency, the National Advisory Committee for Aeronautics (NACA).⁴ In particular, early NACA research and test facilities were instrumental in advancing the safety of civil aviation. Through research conducted over more than four decades—from pioneering tests during the 1920s of wing and propeller icing in refrigerated wind tunnels to the building of design data needed to achieve the stability and control characteristics essential for the introduction of passenger jets in the 1950s—NACA was a leading contributor to aviation safety. NASA’s subsequent programs of research and testing in a variety of areas, such as airborne wind shear detection, microwave landing systems, and head-up displays, have helped advance the safety performance of each generation of aircraft and its operating environment. At the same time, NASA has also played a critical role in addressing pressing safety issues that arise from incidents, from its work in the 1960s on means to prevent bird strikes to its collaborations with the FAA in the 1980s to develop a better understanding of the causes of and corrective actions for wind shear.

Continued growth in aviation has generated significant additional challenges to aviation, from community noise and emissions to airline delays and security threats. These challenges have tapped significant R&D resources from NASA, the FAA, other federal government agencies, and the aviation industry. As the demand on research has risen, so too has the demand for results from safety research that can be applied to immediate safety problems and concerns. The 1996 crashes of TWA Flight 800 and Valujet Flight 592 prompted the White House to convene the Commission on Aviation Safety and Security, chaired by Vice President Al Gore (the Gore Commission). This commission was charged with recommending ways to improve aviation safety and security through changes in procedures, regulation, and research and technology. The 1997 report recommended that a principal focus of government and industry safety efforts should be on reducing the rate of commercial airline accidents by a factor of five within a decade. To do so, the report stressed the importance of partnerships, and specifically urged NASA to expand its collaboration with the FAA and the aviation industry to improve airline safety.

In keeping with the Gore Commission’s advice, the FAA and NASA signed an agreement in 1999 formalizing the agencies’ mutual commitment to developing technologies with the greatest potential for reducing the commercial aviation accident rate.⁵ The two agencies agreed to engage in joint research in a number of areas, such as aging aircraft, wake vortex research, wind shear prediction, and aircraft icing detection.⁶ In addition, NASA emphasized its role in bringing about early improvements in commercial aviation safety. It set a target of reducing the airline fatal accident rate by 50 percent (when compared with baseline levels from 1990 to 1996),⁷ as well as a 10-year goal for an 80 percent reduction.

Five years into its realigned safety research program, however, NASA found itself struggling to measure progress in achieving the airline safety targets. The performance metrics of airline accident and fatality rates proved to be problematic for gauging the impact of research over a relatively short time horizon. Hence, by 2005 NASA was intent on reorienting its safety research program to deemphasize the connection to current accident rates. The policy guidance in the 2006 National Policy coincided with NASA’s decision to change course. The National Policy espoused a more forward-looking aeronautics research program, one that would make better use of each federal agency’s core competencies and unique research capabilities for the aviation community generally. It continued to

stress the importance of research partnerships, but emphasized a stronger federal role in conducting longer-term, foundational research consisting of basic research and developing a strong aeronautics knowledge and technology base to overcome challenges to technological progress. NASA, in particular, was charged with maintaining such a foundational research effort. The National Policy also reflected a new federal emphasis on bringing about the Next Generation Air Transportation System (NextGen) as a means of increasing system capacity and reliability while improving safety and security. It therefore stressed the importance of NASA’s engagement in longer-range, fundamental research to address the needs of NextGen.

Commencing in 2005, NASA’s comprehensive restructuring of its aeronautics research programs into the Aeronautics Research Mission Directorate (ARMD) aligned well with the direction of the National Policy. ARMD’s declared focus was on pursuing “long-term, cutting-edge research that expands the boundaries of aeronautical knowledge for the benefit of the broad aeronautics community.”⁸ Consistent with this emphasis, NASA updated its strategic plan in 2006 establishing an aeronautics research goal to “advance knowledge in the fundamental disciplines of aeronautics, and develop technologies for safer aircraft and higher capacity airspace systems.”⁹ The strategic plan called for research that would “identify and develop tools, methods, and technologies for improving overall aircraft safety of new and legacy vehicles operating in the Next Generation Air Transportation System.”¹⁰ Accordingly, ARMD chose a safety research approach that would be more prognostic, aimed at predicting and preventing safety problems rather than reacting to incidents. Particular emphasis was placed on understanding the safety implications of future aircraft operating in NextGen.

During 2005 and 2006, ARMD worked toward aligning its research programs with this new program emphasis. As explained in Chapter 1, the Fundamental Aeronautics Program was specifically charged with conducting cutting-edge research that produces concepts, tools, and technologies that enable the design of vehicles that fly through any atmosphere at any speed. The Airspace Systems Program was charged with addressing the fundamental air traffic management research needs of NextGen and striving to develop revolutionary concepts, capabilities, and technologies to enable significant increases in the capacity, efficiency, and flexibility of the National Airspace System. The Aviation Safety Program was charged with focusing on developing revolutionary tools, methods, and technologies to improve the inherent safety attributes of current and future aircraft that will be operating in the evolving National Airspace System.

Developed with assistance and leadership from NASA, the FAA, and other federal government agencies, the National Plan lays out the research roles and priorities of these agencies in the context of a transformed, NextGen aviation system. With respect to safety, it points to the potential for new and diverse aircraft operations, including new general aviation, advanced rotorcraft, very light jets, and unmanned aircraft,¹¹ that may present new and complex safety challenges that will require research and development of new safety technologies and operating procedures. The National Plan also acknowledges a continued need for research to address existing safety issues but observes that “the current system has reached a state where low accident levels for commercial aviation, and the traditional forensic investigation approach to aviation safety, are yielding fewer insights capable of significantly improving aviation safety.”¹² Consistent with NASA’s revised aviation safety program, the National Plan stresses the importance of research to advance preventative and prognostic techniques.

The National Plan identifies a series of “fundamental research challenges” that include the following three overarching safety goals and seven safety challenges:

• Monitoring and assessing aircraft health at both the material and component level.

• Rapidly and safely incorporating advances in avionics.

• Stabilizing and maneuvering next-generation aircraft in response to safety issues in the NextGen airspace.

• Understanding and predicting systemwide safety concerns of the airspace system and the vehicles envisioned by NextGen.

• Understanding the key parameters of human performance in aviation.

• Ensuring safe operations for the complex mix of vehicles anticipated within the next-generation airspace.

• Enhancing the probability that passengers and crew will survive crash impact and escape safely when accidents do occur.

These challenges are characterized as being top priorities, intended to provide high-level guidance for foundational, advanced aircraft systems, and air transportation management systems R&D through 2020.

Although the National Plan does not explain how these specific safety challenges were identified, it does point to safety-related research needs that have been identified by other sources. Over the past several years, a number of other reports and plans have been issued to advise and guide NASA’s aeronautics research program, both generally and with respect to safety research. In 2003, Congress created the JPDO to guide and coordinate the efforts of NASA, the FAA, the Department of Defense, the aviation industry, and other entities responsible for bringing about NextGen, and in 2006, NASA commissioned the Decadal Survey of Civil Aeronautics from the NRC to provide guidelines for investment in aeronautics research and technology (R&T) development. Other inputs include the safety data-driven analyses of the CAST administered in conjunction with the International Civil Aviation Organization (ICAO).

The purpose of the NRC’s Decadal Survey was to inform a longer-term strategy for NASA’s involvement in civil aeronautics research. It was undertaken by a 15-member steering committee informed by the work of five panels with expertise in specific research disciplines, such as propulsion, materials, control, and autonomous systems. The expert panels focused on identifying and prioritizing individual aeronautics research and technology (R&T) challenges that are particularly suited to NASA aeronautics research. All of the 39 highest-rated challenges pertaining to safety (scoring a “9” for safety and reliability) are listed in Appendix D. Discipline areas having the greatest number of high-priority safety challenges were (1) Intelligent and Autonomous Systems, Operations and Decision Making, Human Integrated Systems, and Networking and Communications (15 challenges) and (2) Dynamics, Navigation, Control, and Avionics (13 challenges).

As a partner in the JPDO, NASA participates in JPDO working groups and assists in the development of its relevant safety-related R&D plans. In September 2008, the JPDO released its Next Generation Air Transportation System Integrated Work Plan.¹³ The Integrated Work Plan (IWP) is viewed by JPDO partners as a tool to build commitments and coordinate efforts and resources to help bring about NextGen. As such, it outlines federal government and industry responsibilities and collaborations, proposing a series of milestones and timelines for actions, including R&D activities. The IWP calls on NASA to conduct R&D in a number of safety-related areas, including advances in analytic tools for safety assurance and risk management, runway incursion prevention and detection systems, and new aircraft materials and designs that achieve long-term air-

worthiness. In particular, the IWP points to the following safety-related areas requiring coordinated research:

• Tools to support the NextGen Aviation Safety Information Analysis and Sharing (ASIAS) capability;

• Methods for verification and validation of complex systems to support NextGen risk assessment and certification decisions; and

• Performance models that capture human variability and error in highly automated NextGen systems to aid in the development of risk-reducing interfaces, procedures, and training.

NASA’s Aviation Safety Program and the Airspace Systems Program are listed in the IWP as having sole or shared primary responsibility for an array of NextGen-related research, many examples of which are shown in Box 2.1.

NASA connects with the aviation industry through CAST, which employs a data-driven strategy to develop and promote government and industry safety initiatives aimed at reducing the risk of commercial aviation fatalities. Composed of industry and government safety experts, CAST analyzes reports and data from aircraft incidents

worldwide. Teams of experts from member organizations are then tasked with developing methods to more fully understand the chain of events leading to incidents and to identify solutions. CAST aided the FAA in the creation of the agency’s ASIAS program. NASA is a member of CAST, and thus managers from the Aviation Safety Program participate in CAST’s Joint Implementation Measurement Data Analysis Team (JIMDAT), which develops a master safety plan, measures effectiveness, and identifies future areas of study, all key inputs to the CAST Safety Plan. The director of NASA’s Aviation Safety Program is also a member of the Executive Board of ASIAS and thus approves ASIAS-directed safety analyses. CAST has recently expanded its focus to include identifying new and emerging safety risks to enable the implementation of proactive measures before new types of accidents occur. NASA is also participating in this initiative, referred to as “FAST”—Future Aviation Safety Team.

Aware of these many external sources of input, the committee asked NASA to describe how it uses this information, and any other sources, to establish aviation-safety research objectives and prioritize them for the purpose of programming and resourcing research. Figure 2.1 depicts the process as described schematically.

The committee was told that a major function of the Aviation Safety Program office is to review its existing research portfolio and any proposed research for consistency with NASA’s mission, the fundamental safety challenges identified in the National Plan, and identifiable safety research needs. To identify safety research needs,

FIGURE 2.1 NASA depiction of its process for analyzing its safety research portfolio in relation to research needs. SOURCE: Amy Pritchett, Director, NASA Aviation Safety Program, “Safety-Related Research in NASA’s Aeronautics Research Mission Directorate: Overview,” presentation to the committee, September 3, 2009.

the program leaders stated that they consult a spectrum of stakeholders from the aviation community, including the FAA, the JPDO, the National Transportation Safety Board (NTSB), CAST, other federal government agencies, and the aviation industry. The program office also reviews the recommendations in the Decadal Survey of Civil Aeronautics, other independent assessments of research needs by the NRC and others, and advice from the NASA Advisory Council.

The process of prioritization appears to be one in which important safety concerns in need of research are identified through the various external inputs described above and then assessed in light of NASA’s existing safety research projects and capabilities (assets, workforce, and funding). Priority is given to those concerns that align well with the agency’s existing research projects and capabilities. The objectives of these research projects and the utilization of existing research capabilities are then adjusted to better align with these emphasized safety concerns.

Based on this process, the Aviation Safety Program leadership stated that its safety research projects align with the following six overarching safety research concerns (listed in no particular order):

• New Operations,

• Flight In or Around Hazardous Conditions,

• Loss of Control,

• Durable Aircraft Structures and Systems,

• On-Board System Failures and Faults, and

• Analyzing Complex Systems for Safety.

NASA’s safety research projects that align with each of these concerns, as well as their objectives, content, and resources, are examined in Chapter 3. What was not made clear to the committee, however, is if NASA undertakes objective evaluations of safety research priorities irrespective of its current research programs and capabilities. In general, NASA’s safety research prioritization process appears to be heavily influenced by the availability of existing expertise and resources within the agency. Not having been provided with a more objective needs assessment, the committee is hindered in its ability to judge whether the six concerns are indeed the most appropriate priorities for federal investment going forward. The committee nevertheless decided to review each of these research concerns (as well as the safety research in the Fundamental Aeronautics Program and the Airspace Systems Program) with respect to each of the main aspects of the study charge, despite the fact that the committee could not weigh in on their overall appropriateness.