Improving the Continued Airworthiness of Civil Aircraft: A Strategy for the FAA's Aircraft Certification Service (1998)

A number of barriers, both internal and external, will make it difficult to implement the recommended safety management process. The FAA has accepted the challenge of making aggressive improvements in aviation safety over the next 10 years. This goal cannot be achieved unless the FAA's efforts to improve the safety management process include dedicated efforts to overcome these barriers.

Major Recommendation 5. In order for AIR to contribute as much as possible to improvements in aviation safety, the FAA—in partnership with industry, Congress, the Department of Transportation, and other involved parties—must take aggressive action to overcome barriers associated with the following:

• external pressures and influences on the FAA

• coordination and communications within the FAA

• legal issues

• the rulemaking process

• the economic impact of proposed changes to the safety management process

Highly publicized accidents are often caused by factors not associated with the greatest aviation hazards. Unfortunately, time and patience are required to determine the causes of any accident and to understand how to prevent a reoccurrence. Nevertheless, the FAA typically comes under considerable political pressures to do something immediately, even if it is based more on speculation than on facts. These pressures have sometimes caused the FAA to take corrective action that is otherwise unjustified.

Attention should be focused on the highest priority problems, as determined by accident and incident data—not by premature or politicized conclusions about the cause of the most recent major accident. The committee understands that major airline accidents naturally attract a great deal of attention from the media and others and agrees that these accidents deserve intense investigation to identify cause factors and appropriate corrective action. These investigations, however, should be structured in a way that does not interfere with other important safety projects. Implementing the recommended safety management process would help by reducing the number of accidents.

The time pressures typically associated with a "publicly investigated" accident may limit the effective use of risk analysis tools to identify cause factors and the most appropriate solutions. Public pressure may also result in action that is not supported by technical data and may be less effective in the long run than taking the time to develop a more effective response. For example, the secretary of transportation may dictate or industry may request an immediate AD to resolve the controversy associated with a particular accident and to deal with the bad publicity. In one case, political pressure following an in-flight explosion prompted the FAA soon afterwards to mandate aggressive new security measures because the accident was initially assumed to be a terrorist act, which was not the case. As a result, airline costs were increased unnecessarily and levels of service were reduced with no measurable improvement in safety. The crisis-like atmosphere and the public search to find someone to blame for an accident also tends to create an atmosphere of mistrust and animosity that contributes to public questions about aviation safety and whether industry and government are acting in good faith, even though aviation is in fact the safest mode of transportation in the United States.

The NTSB has the sole responsibility for determining the probable cause of an accident. The NTSB may make safety recommendations during the course of an investigation that it later concludes are unrelated to the probable cause. The FAA has a statutory responsibility to participate in accident investigations and to order corrective action it determines is necessary to prevent a reoccurrence. During the long interval between the occurrence of an accident and the time the NTSB makes a formal finding of probable cause, conflicting

views often develop among the NTSB, FAA, and other parties on what corrective action should be taken. These conflicts can be healthy when they test opinions and ensure that the most effective preventative measures are implemented. However, sometimes one or more parties try to shape public opinion in its favor. This can create additional pressures that are counterproductive and contrary to the public interest.

In response to NTSB recommendations that require regulatory action, the FAA is required by statute and executive order to complete cost-benefit analyses. These analyses compare the financial costs of implementing the recommended corrective action with the monetary value of expected benefits. The Department of Transportation specifies the monetary value ($2.7 million) of each life that would be saved by the recommended action. Until recently, if the FAA could not document that benefits exceeded costs, it was prohibited from adopting the recommendation. Even now, a recommendation is rarely implemented if its benefits cannot be shown to exceed its costs. The NTSB has no similar constraints on making safety recommendations.

For major regulatory actions the Office of the Secretary of Transportation and/or OMB verify the FAA's cost-benefit analyses. However, if the FAA determines that recommended actions will not be cost effective and decides, or is directed, not to implement an NTSB recommendation, the FAA may be harshly and publicly criticized for failing to take action. This criticism rarely seems to recognize the different roles of the NTSB and the FAA or the statutory requirements that the FAA must satisfy to take regulatory action (see Chapter 2).

The committee recognizes and endorses the NTSB's status as an independent accident investigatory agency. However, greater harmony between the FAA and NTSB would likely improve the ability of both agencies to address the most significant risks to aviation safety and increase the confidence of Congress and the public in the safety of the air transportation system.

Finding 6-1. Following some highly publicized accidents, there is a technically unjustified loss of public confidence, which leads to political pressure and a counterproductive atmosphere of crisis management in the FAA that interferes with ongoing efforts by government and industry to improve aviation safety.

One approach for addressing this issue would be for the Department of Transportation, FAA, and NTSB to develop a joint process that reduces interagency disagreements about important safety recommendations, while protecting the NTSB's role as an independent agency and the FAA's ability to comply with the requirements of the rulemaking process. This process could include the following elements:

• establishment of a senior interagency communications or safety management board that would coordinate the government's public comments on aviation disasters

• improved communications between the FAA and NTSB, perhaps through combined analysis meetings and short-term personnel exchanges

• establishment of a combined technical team to assess how NTSB recommendations should be disposed of in the context of FAA rulemaking procedures and requirements

Recommendation 6-1. As a first step towards reducing the negative impact of external pressure on the safety management process, the FAA should work with other responsible agencies to educate the public more fully about ongoing efforts to improve aviation safety. Fully addressing this issue is likely to require major organizational changes, such as the establishment of a senior interagency communications or safety management board, that were beyond the scope of this study.

Many of the individual organizational elements within the FAA enjoy considerable autonomy over their assigned areas of responsibility but lack an effective means of communicating with each other and resolving differences. As demonstrated by the long time it typically takes to make regulatory changes, the lack of communication sometimes prevents the FAA from taking timely action to improve overall safety or operational efficiency. More effective communications—within AIR and between AIR and other FAA offices, such as the Flight Standards Service—would considerably improve the aircraft certification safety management process. Consider the following two examples.

This report recommends that the FAA eliminate internal efforts to collect and store data for aircraft manufactured by companies that agree to assume this function. Currently, data is collected by many different offices within AIR and other parts of the FAA. To be most effective, defining how the FAA can increase its reliance on external databases should involve each office that currently collects and maintains data. This would ensure that the new approach will meet FAA data requirements and avoid inappropriate duplication or gaps in data collection and storage. Such an approach, however, would require effective communications and strong leadership so that efforts to obtain the support of involved parties would not prevent timely action.

The relationship between AIR and the Flight Standards Service is also important. AIR has primary responsibility for developing and administering aircraft airworthiness standards and issuing related ADs. The Flight Standards Service has primary responsibility for approving operational procedures and minimum maintenance requirements and for checking that operators comply with airworthiness standards. Each organization has knowledge, expertise, sources of information, and channels of communication with outside organizations that are more fully developed in some areas than

others. Close coordination between AIR and the Flight Standards Service helps ensure that solutions are based on the best information available and that corrective action is disseminated as effectively as possible.

Recommendation 6-2. The FAA should develop a process to facilitate communications and improve coordination among offices within AIR and between AIR and the Flight Standards Service. For example, the Associate Administration for Regulation and Certification could establish a central coordinating office to facilitate the exchange of continued airworthiness information within the FAA and the dissemination of complete and consistent information to industry.

Although the committee was not constituted to address legal issues in detail, it identified three legal issues that could delay full and effective implementation of the recommended safety management process. These issues are associated with the potential for public disclosure of sensitive information under the Freedom of Information Act, the possibility of regulatory enforcement against individuals or companies who voluntarily disclose unfavorable safety data, and increased exposure to legal liability arising from the litigation discovery process in an environment where more data is collected, stored, and shared. The recommended safety management process advocates the voluntary participation of airlines, manufacturers, and other stakeholders during the long period it would take to develop new regulations mandating compliance. Thus, resolving industry's widely held concerns about legal issues is essential. Even a mandatory system would rely heavily on the good faith of all participants, and good faith is not likely to last very long if pilots, operators, and manufacturers are punished for voluntarily drawing attention to safety problems that would otherwise go unnoticed.

Under the recommended safety management process, the FAA would have access to a large amount of safety data generated and collected by industry. This access would enable the FAA to oversee the recommended safety management process. However, the safety data, which are likely to be considered sensitive and proprietary by manufacturers and operators, could be subject to disclosure outside the government under the Freedom of Information Act. Safety data can be disclosed to the FAA under restrictive legends and/or confidentiality agreements, but there is no guarantee that the data would be uniformly protected because disclosure determinations under the Freedom of Information Act are subject to judicial discretion and conflicting rulings by various courts. Moreover, once data have been shared with competitors, they can no longer be considered confidential commercial or financial information. The Federal Aviation Reauthorization Act of 1996 addressed this concern by changing Title 49 of the United States Code. This change directs the FAA not to disclose voluntarily-provided safety-or security-related information "notwithstanding any other provision of law" if the FAA administrator finds that

The FAA is in the process of changing the FARs to implement this legislation.

The FAA's current policy is to not use FOQA (flight operations quality assurance) data provided by industry voluntarily as a basis for enforcement action, and that policy is being codified by a change in the FARs. This change is not likely to apply to enforcement action associated with other types of information that may be voluntarily provided to the FAA.

Legal discovery is a more difficult matter. Industry may be reluctant to increase the amount of safety data it generates and disseminates voluntarily because of concerns that the information will be subject to discovery in civil lawsuits. Moreover, increasing the volume of information could lead to increased potential liability, particularly in areas where liability may not currently exist (i.e., where dissemination of safety information notifies a manufacturer or operator of potential safety problems that were unknown to it). Industry's concerns in this regard may be addressed in part if identifying information is redacted from performance and safety data prior to dissemination. The level of detail retained in the shared data would need to be weighed against the proprietary and privacy rights of the organizations providing the data and the need for a database detailed enough to support a robust risk analysis and risk management process. Even then, the unredacted performance data would still potentially be subject to civil discovery.

The possibility of discovery is a disincentive for manufacturers and operators to participate in the safety management process. The committee believes that the public interest would be served and safety would be enhanced by legislative action to address this disincentive to the voluntary sharing of safety data among operators and manufacturers.

Section 701(e) of the Federal Aviation Act of 1958 and section 304(c) of the Independent Safety Board Act of 1974 specify that NTSB accident reports are not admissible in civil actions. The Congress should consider legislation that would also limit the use of manufacturers' and operators' safety analyses in civil suits against the company that supplied the information.

Recommendation 6-3. The FAA should initiate regulatory action, legislative action (through the Congress), and/or letters of understanding with manufacturers, operators, pilot organizations, and others to serve the public interest and improve safety by encouraging the voluntary sharing of safety data, including data generated by industry safety analyses. This may involve limiting enforcement action based on voluntarily shared data and protecting such data from release to other parties.

The FAA's rulemaking process, which is described in Chapter 2, quite often takes 5 to 10 years to revise airworthiness standards. Rulemaking projects are sometimes completed much more quickly, especially in the case of highly publicized or emergency safety actions. However, many worthwhile, safety-related rulemaking projects languish for years. The barriers to quick action are many and have a considerable history. For example, the current rulemaking processes is complex and not enough staff hours are dedicated to rulemaking in AIR and other offices, such as the Office of Rulemaking, Office of Policy and Plans, and Office of the Chief Counsel. Significant delays also have occurred during reviews of NPRMs (notices of proposed rulemaking) and final rules by the Office of the Secretary of Transportation. Tremendous delays are caused by differing, strongly held points of view within the FAA, other federal agencies, and/ or industry. In such cases, deferring action or referring a matter back to another office for reconsideration may be the easiest—but least productive—course of action.

The FAA is certainly not the only federal agency that has difficulty making timely changes to federal regulations, but delays in the federal aviation rulemaking process are a significant safety issue. In fact, some safety improvements are structured to circumvent the rulemaking process because of the delays it would otherwise take to implement them in regulatory form. The FAA may issue advisory circulars, which provide nonmandatory guidance, or rely on draft documents to provide interim guidance during the years it takes to issue final policies or regulations. In other cases, industry may delay safety improvements until the FAA decides if the proposed action is a sufficient response to a particular concern. Industry is often reluctant to make safety improvements, such as expensive aircraft modifications, without assurance from the FAA that new rules will not require undoing or redoing the modifications. Individual operators who decide to implement safety enhancements voluntarily may place themselves at a competitive disadvantage. The public is not well served by a process that pressures operators to delay implementation of safety enhancements.

The Federal Aviation Reauthorization Act of 1996 addressed rulemaking delays by requiring the FAA to issue a final rule within 16 months after the close of public comments on an NPRM. However, it quite often takes several years for an NPRM to be generated and approved by the ARAC (Aviation Rulemaking Advisory Committee); to be revised and approved by cognizant officials in the FAA, Department of Transportation, and OMB; and to be issued for public comment. The 1996 act does not limit how long this part of the rulemaking process may take. Neither does the act allocate additional resources to rulemaking or waive any regulatory or legislative constraints on the rulemaking process. The FAA has not yet determined how to meet the timeliness requirements of the 1996 act for the large number of rulemaking projects that are typically under way at any one time, nor has it determined how to significantly accelerate the preparation and issuance of NPRMs. Reassigning the duties of current FAA personnel could reduce time delays, and establishing ADOs (approved design organizations) in accordance with Major Recommendation 3 could help by reducing the number of personnel required to support certification activities.

The FAA established the ARAC and its industry working groups to facilitate the rulemaking process. Each of the numerous working groups is assisted by FAA technical experts, cost-benefit analysts, and legal representatives. The ARAC was expected to develop all of the documents necessary to implement its rulemaking recommendations, including NPRMs, cost-benefit analyses, and advisory circulars describing how to implement new or modified regulations. When the ARAC was chartered in 1991, expected benefits included the following:

• taking advantage of industry's domestic and international technical expertise and experience to prepare better rules

• resolving controversies in an open forum prior to the formal rulemaking process, thereby shortening the time required for the disposition of comments elicited by NPRMs

• broadening participation in rulemaking by the public, industry, and the aviation authorities of other countries (thereby facilitating the international harmonization of aviation safety regulations)

The ability of the ARAC to shorten the rulemaking process has been limited for the following reasons:

• The ARAC itself has become large and unwieldy, with representatives of about 70 organizations. The ARAC

also has about 50 working groups (each working on multiple subjects) with a total of about 400 participants. Reaching consensus is time consuming, often taking years, because of the large number of participants and because of the turnover of participants resulting from the protracted duration of individual projects.

• ARAC recommendations are subject to the same lengthy rulemaking process as proposed rules developed without the assistance of the ARAC. The net result: even after the ARAC has invested years reaching consensus, it may take several more years for the FAA to act on a proposal.

• Proposals for many worthwhile and noncontroversial rule changes are backed up within the FAA behind higher priority rule changes that have been delayed because of political sensitivities. As of January 1998, more than 300 rules and rule proposals were awaiting action by the FAA.

• Although the FAA participates in the ARAC process as an observer, industry perceives that many FAA participants either drive committee deliberations toward predetermined positions, which contributes to delays in reaching consensus, or do not accurately portray FAA technical and policy concerns with proposed actions.

• Rulemaking documentation produced by the ARAC often requires extensive reworking after it is submitted to the FAA, which increases delays. Some industry representatives believe this results from having FAA participants with limited expertise and authority who cannot effectively represent FAA positions. Conversely, some FAA personnel believe the problem is the lack of expertise of industry participants. To some extent, both of these views are correct. Doubts about the ARAC's timeliness and effectiveness reduce the incentive for both the FAA and industry to assign more skilled and experienced personnel to ARAC activities.

Recommendation 6-4. Efforts to reform the ARAC should (1) establish more timely and effective processes and (2) encourage the assignment of industry and FAA personnel who have the expertise to develop well written NPRMs and final rules and the influence necessary for building broad support for documents approved by the ARAC.

In August 1996, the FAA's Office of Rulemaking initiated the Rulemaking Process Reengineering Project, which included participation by other FAA offices concerned with the efficiency of the FAA rulemaking process. The FAA administrator reviewed the results of this project and approved implementation of its recommendations starting in early 1998. These recommendations are intended to accomplish the following:

• Improve FAA management of the rulemaking process, including management of ARAC activities.

• Improve the quality of the rulemaking products at all stages.

• Facilitate early resolution of differences among major stakeholders.

This project was limited to a review of internal rulemaking procedures. The committee did not have an opportunity to assess the project's recommendations or the extent to which they are likely to expedite the rulemaking process.

Finding 6-2. It quite often takes 5 to 10 years to issue new regulations or modify existing regulations. This is an important safety issue because it constrains the ability of the rulemaking process to improve aviation safety. The FAA is in the process of reforming its internal procedures, including ARAC procedures. This is a positive first step, but much more needs to be done in this area, and time is of the essence.

Recommendation 6-5. The FAA should make the rule-making process substantially more responsive by convincing the Department of Transportation, other executive branch agencies, and Congress to modify legislation, directives, and regulations to allow major changes in the current process.

Over the long term, more comprehensive data collection and analysis systems are expected to reduce costs and improve safety. The air transport industry is highly competitive, however, and this natural competitiveness could present a barrier to the voluntary sharing of data required to implement the recommended safety management process. For example, the cost of establishing the recommended database management and risk analysis systems could be perceived as a barrier. The costs and associated benefits must therefore be thoroughly evaluated by the FAA in cooperation with industry, and the FAA should provide financial support for pilot programs to validate costs and demonstrate the effectiveness and practicality of implementing procedures. This approach. would be similar to the one that the FAA is using to foster the spread of FOQA systems among U.S. operators, in which the FAA funds pilot projects by several U.S. airlines.

A similar problem will occur as the safety management process becomes more proactive, identifying safety actions that should be taken to eliminate risks before they cause an accident. Manufacturers and operators bear the cost of making safety improvements, and they will not support improvements unless the identified risks are credible and the corrective action seems reasonable in terms of effectiveness, direct costs, and indirect costs, including disruptions in operations and damage to their corporate reputations. It is imperative that any new risk management system have enough credibility to justify making increased financial demands on industry, especially if the financial demands could reduce

competitiveness. This process may be simplified if enhanced data collection, database management, and analysis systems reduce operating costs. For example, FAA-funded pilot projects have shown that FOQA systems can reduce airline operating costs more than enough to pay for themselves.

Recommendation 6-6. The FAA should work with industry to develop confidence in the cost-benefit analyses used to justify changes in the safety management process. The FAA should also subsidize pilot projects by operators and manufacturers to validate the cost effectiveness of new systems for data collection, database management, and analysis.