Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 177
Flight to the Future: Human Factors in Air Traffic Control 9 Human Factors in Airway Facilities The human factors aspects of Airway Facilities operations, equipment, job classifications, selection, training, and performance appraisal are critical to the impacts of current automation. In this chapter we discuss the effects of increased automation on Airway Facilities operations and staffing and the state of research in the human factors of Airway Facilities. This chapter builds on the overview of Airway Facilities presented in Chapter 4. EFFECTS OF INCREASED AUTOMATION It is noteworthy and perhaps paradoxical that, when new components or systems are introduced, the impact of automation is often experienced more by Airway Facilities than by Air Traffic Services. The new components or systems occasionally include increased automation of air traffic control functions; often they represent modernization of aging equipment without significant change to the human-machine interface for air traffic controllers. In either case, the new systems increasingly include automation of such functions as diagnostics, fault localization, status and performance monitoring, and maintenance logging. These automation enhancements are usually transparent to the air traffic controllers, but they can impose on Airway Facilities specialists the requirement to learn new and often complex functional and human-machine characteristics of the modernized equipment. It is difficult to make generalizations about current components and systems; about the procedures and activities associated with their operation, monitoring,
OCR for page 178
Flight to the Future: Human Factors in Air Traffic Control and control; and about the associated personnel selection, staffing, training, and performance appraisal procedures because: (1) there is variation across FAA regions, sectors, and facilities with respect to equipment, systems, and personnel considerations and (2) the current process of modernization involves the piecemeal introduction of new technologies, with associated changes to operations and personnel activities, in a manner that places the Airway Facilities domain in a state of flux. As new technology tends toward more software-intensive and automated functions, network linkages, space-based systems, and highly reliable distributed architectures—introduced at different times in different places—several changes are occurring within Airway Facilities (Schroeder and Deloney, 1983; Reynolds and Prabhu, 1993; Federal Aviation Administration, 1991c, 1993a, 1995b, 1995c). There is a continuing trend toward increased automation of such functions as data acquisition and storage, diagnostics and fault localization for modularized equipment, reconfiguration through the use of redundant software as well as hardware elements, and maintenance logging. At the same time, automation support for such higher-level cognitive functions as system-level diagnostics, trend analysis, decision making, and problem solving is reserved for longer-term development. Maintenance philosophy is turning from an emphasis on corrective and regularly scheduled preventive maintenance to an emphasis on performance-based maintenance that takes advantage of automated trend analyses to identify the most efficient scheduling for maintenance to prevent failures. Maintenance philosophy is also turning away from concentration on on-site diagnosis and repair of elements of equipment toward more centralized and consolidated operational control centers (OCCs) that monitor and control equipment and systems at unmanned facilities, accompanied by automated localization of problems to line replaceable units that are replaced and sent to contractors for repair. The focus on "systems within one's jurisdiction" is being replaced by a focus on sharing of information, resources, and responsibilities across jurisdictions. Airway Facilities job classifications have traditionally stressed specialized knowledge of hardware for specific equipment or systems—knowledge that is still required to keep the current system operational. However, new job classifications are placing much more emphasis on knowledge of and responsibility for monitoring and controlling interacting systems, on management of software-intensive, distributed networked resources, and on application of systems engineering methods to provide system services to users. Selection procedures, which previously encouraged the hiring of military personnel with electronics backgrounds, are placing more emphasis on the hiring of personnel with skills and abilities related to systems engineering, computer science, and automation. Training programs have traditionally involved strings of courses that develop expertise with single items of equipment or single, independent subsystems; there is increasing demand for programs that develop expertise in diagnosing and responding to system-wide difficulties, including understanding of the interactions between
OCR for page 179
Flight to the Future: Human Factors in Air Traffic Control systems that cooperate to provide national airspace system services. There is a corresponding change in the way the performance of Airway Facilities specialists is appraised, away from ''fix the box" tests and observations toward evaluation of specialists' abilities to diagnose and respond to system problems. In this chapter the panel makes some generalizations with respect to such moving targets. In discussing components of the national airspace system, equipment supporting Airway Facilities activities, and its operations, our approach is to suggest sets of items that may have a bearing on automation issues, rather than attempt to distinguish the many site-specific variations. One generalization that has received recent attention in the national media is that the equipment and systems that support air traffic control represent obsolete technology. Triggered by the occurrence of major equipment and system outages at busy air traffic control facilities, media reports have reflected concern that the current components collectively represent a museum of electronic (including computer) equipment whose increasing unreliability may be approaching the point of overwhelming the efforts of Airway Facilities specialists to maintain its operation and of air traffic controllers to cope with its degradations. A highly significant but less broadcasted fact is that the Airway Facilities technical workforce has aged along with its equipment. The FAA estimates that, within 10 years, up to 35 percent of this technical workforce will be eligible for voluntary retirement (Federal Aviation Administration, 1993b). This has raised two questions: (1) If the experienced technicians, who are now relied on to maintain the aging equipment, retire before the equipment is modernized, who will keep the system operational? and (2) If efforts to replace the aged equipment with modernized equipment succeed before the experienced technicians retire, who will be trained to ensure the proper operation of the new equipment? Given the many changes taking place, the near future is a critical period for Airway Facilities, during which the decisions made with respect to automation will be extremely significant. A significant contributor to the success of the FAA's modernization efforts will be the extent to which the application of automation reflects appreciation for the human factors issues associated with each of the following questions, as well as for the fact that answers to each question must properly interact: To what extent do the systems for which Airway Facilities is responsible perform automated functions to maintain their operation and to recover from degradation or failure of their components? What are the automation and human-computer interface characteristics of the monitoring and control tools provided to Airway Facilities? What are the job classifications, descriptions, and qualifications for Airway Facilities technicians? What critical operations do technicians perform?
OCR for page 180
Flight to the Future: Human Factors in Air Traffic Control How do Airway Facilities and Air Traffic cooperate as a team to respond to problems? What are the selection procedures for and the characteristics of technicians? How are technicians trained? How is the job performance of technicians appraised? What roles do Airway Facilities representatives play during the equipment acquisition process? What roles do human factors representatives play with respect to the design, development, and test of equipment and systems used by Airway Facilities personnel? What are the effects of the agency's organizational structure and culture on the job satisfaction of Airway Facilities personnel? Automation at the Maintenance Control Centers As explained in Chapter 4, each Airway Facilities sector is supported by a centralized monitoring and control workstation suite, which functions as the command center for the technical support of en route centers and terminal facilities. Automation and computer assistance (with respect to both information display and control) are applied to different levels in the different systems and equipment monitored and controlled through the maintenance control center (MCC). In general, automation and computer assistance are applied more often to support such sensing, calculation, data searching, and control actuation functions as information retrieval, alarm reporting, remote control, and data recording. Automation is rarely applied to perform or to support such higher-level cognitive functions as trend analysis, failure anticipation, system-level diagnostics and problem determination, or certification. In fact, FAA Order 6000.30B, which establishes a long-term policy for national airspace system maintenance, recommends that automation be applied to repetitive maintenance tasks and that the Airway Facilities specialist be left "free to accomplish higher level, decision-oriented work" (Federal Aviation Administration, 1991c:5). FAA Order 6000.39, which defines the MCC operations concept, summarizes the current philosophy of automation (Federal Aviation Administration, 1991a:5): The MCC will implement automation to the degree that tasks can and should be automated. Advances in expert systems and artificial intelligence will be applied where possible to automate tasks requiring a small degree of human intervention…. The following functions shall be automated: … (1) Routine functions requiring little or no human intervention, such as diagnostic report generation … (2) Data gathering not requiring narrative or human interpretation. (3) Administrative paper documentation. Given the fact that modernization is accomplished through many different
OCR for page 181
Flight to the Future: Human Factors in Air Traffic Control programs within the FAA and involves many different vendors of equipment and systems, and given the fact that the national airspace system is the focus of rapidly advancing technologies, Airway Facilities specialists can find themselves faced with a variety of new technologies, provided by different vendors, with varying levels of automation and different human-machine interface design strategies, at the same time that the procedures and the human-machine interface for air traffic controllers experience more controlled growth and change. As a rule, air traffic controllers are provided with integrated workstations whose display/control logic and formats are carefully guarded and monitored during the development of new supporting systems. In contrast, the technicians who monitor and control the supporting equipment are typically provided with new monitoring and control devices that are tacked onto the array of such devices for other equipment in a loosely arranged combination that lacks integration (Theisen et al., 1987). The Airway Facilities community has specified standardized protocols and data acquisition and processing requirements to ensure that new components and systems will provide data to and accept control commands from the centralized monitoring and control workstations (Federal Aviation Administration, 1994c). However, these and other (Federal Aviation Administration, 1991a) recommendations address only the lower-level automation tasks mentioned above. They do not address the proper allocation of higher-level tasks between human and machine, the integration of automation functions across disparate systems, or the integration of the associated human-machine interfaces at the MCC. Without an adequate attempt to understand, clarify, and standardize the automation requirements and associated human-computer interface for its workstations, Airway Facilities cannot play an effective role in the systems acquisition process that determines to a large extent whether automation of various systems by various vendors under various program management personnel will contribute to or detract from its ability to fulfill its obligations. Supervisory Control and Automation Investigation of supervisory control and automation for air traffic control should not be limited to examination of the air traffic controllers' workstations. In the process of monitoring and controlling air traffic patterns and activities, air traffic controllers do monitor the apparent quality of the data appearing on their workstations and the performance of their display and control devices. Air traffic controllers will question, for example, the quality of radar-provided data, and they do have limited control over the selection of radar parameters for display. However, it is the responsibility of Airway Facilities to monitor and control all equipment that ultimately supports the controllers, to inform the controllers of the status and performance of equipment and systems on which their tasks depend (including the controllers' workstations), to reconfigure and maintain degraded or failed equipment in a manner that minimizes interference with air traffic
OCR for page 182
Flight to the Future: Human Factors in Air Traffic Control control tasks, and to respond to requests for service from controllers. Therefore, the air traffic controllers are not the only supervisory controllers of their equipment; the air traffic supervisory control tasks must be viewed as cooperative efforts of both controllers and technicians. On that account, many issues pertaining to the automation of air traffic control functions, discussed in Chapter 12 of this report, require examination of the activities of both Air Traffic Services and Airway Facilities that contribute to the performance, monitoring, and control of those functions. As equipment and systems evolve, these roles and responsibilities may change with respect to responding to degradation or failure of the software and hardware that support the automated functions, and with respect to performing supervisory control tasks for air traffic control when monitoring and control of automation equipment is included. A word of caution is offered concerning the general use of the term automation. Airway Facilities has been increasingly confronted with new equipment and systems labeled "automated"; this trend helped prompt the creation of the job classification "automation specialist" electronics technician; a similar job classification once existed within the Air Traffic staffing organization (this job classification recently became the responsibility of Airway Facilities). The activities and expertise of automation specialists have largely addressed computer hardware diagnosis and maintenance, as well as computer program analysis and development tasks. Automation was generally understood to be associated with computers, and particularly with computer software, as distinguished from the activities associated with maintaining the other electronic hardware components of radar, communications, and navigation systems. Although this distinction did recognize the technological trend toward software-intensive and software-modifiable computer-based systems, it masked two aspects of the difference between "automation'' and "modernization." First, many modernization efforts within the national airspace system involve application of computers without significant changes to the allocation of functions between humans and machines; automation is not a necessary corollary of computerization. Second, automation is not a discrete attribute of a new system; automation may be applied in degrees or in levels. New systems often do introduce changes in the level of automation for some tasks. However, these changes generally apply to such tasks as data gathering, calculation, rule-based determination of the status of components, comparison of performance against thresholds, and automated switching to redundant backup components when primary components fail. Computerization has not generally introduced to Airway Facilities tasks the automation of such problem-solving functions as diagnosing faults from patterns of failures, predicting faults from trends in data, reconfiguring systems (as opposed to single components) in response to system failures, or certifying systems and services. Masking of the distinctions between automation and modernization and between automation and computerization has significant impact on the tasks expected of automation specialists, and on the selection, training, and performance
OCR for page 183
Flight to the Future: Human Factors in Air Traffic Control appraisal procedures associated with this job classification. The FAA has implicitly begun to recognize these distinctions with the introduction of the GS-2101 job classification, the airway transportation specialist. The position description of this job classification helps to distinguish among automation, computerization, and modernization. The GS-2101 incumbent must understand and will rely on increasingly automated computer functions and must also possess knowledge and skills representative of systems engineering: how components and systems interact to produce services. Almost all former automation specialists have been reclassified as GS-2101s, along with most other electronics specialists who have mastered interacting systems. On this account, it should be made explicit that, within Airway Facilities, "working with automation" is no longer recognized as a special job classification; it is rather a set of tasks that virtually all Airway Facilities specialists perform while working with systems. In this connection, it would be useful if new systems introduced to the national airspace system included precise descriptions of their automated functions rather than the ubiquitous "automation" label. The distinction required now is not whether systems are or are not automated, but which functions of systems are or are not automated. OPERATIONS General Responsibilities The duties most germane to the current concern with the implications of automation for air traffic control are monitoring and control, diagnosis of systems (as opposed to equipment components), certification, and restoration of services. The general applications of automation to monitoring and control have been mentioned in Chapter 4 in connection with maintenance control centers. Automation has been applied to system diagnostics in connection with both the MCC capabilities, discussed above, and the processes of certification and restoration, discussed below. Certification The increased reliability of computer-based systems and the automation support for diagnostics that are often embedded in such systems currently suggest the following trends in certification: extension of the acceptable certification intervals; increasing reliance on the results of built-in diagnostics that can support certification while the equipment remains in operation; more performance of remote certification, replacing the need to examine the equipment directly; and more automated maintenance logging and equipment performance recording. However, these trends and the application of automation to the certification process must be considered in the light of the following current formal procedures
OCR for page 184
Flight to the Future: Human Factors in Air Traffic Control for performing certification, defined in FAA Order 6000.15B (Federal Aviation Administration, 1991b): in addition to the procedures recommended in technical handbooks, instructional books, and other technical documentation that accompanies the delivery of equipment and systems, the certifier should use methods that include: direct measurement of certification parameters; monitoring status indicators; analyzing technical performance; performing a comparative analysis of flight inspection data with previous results; visual and aural observations of data, extraneous noises, excessive heat, and questionable odors; reports by pilots and controllers; and diagnostic testing. It is important to note that the certification process is not governed by standard algorithmic procedures. The FAA emphasizes that the choice of methods used for certification determination is left to the professional judgment of the certifying technician. The automation applied to the certification process should support the judgment strategy of the certifier. Since the FAA orders governing certification suggest that certifiers are free to—and must—develop their independent judgments, the question of automating certification should be associated with attempts to analyze the entire set of certification tasks, including the judgment process. As with the general tasks of monitoring and control, automation is currently applied to lower-level certification tasks of data acquisition, data calculation, and simple rule-based diagnostics, but not to higher-level judgment tasks. A significant practical consideration with respect to the automation of certification functions is: How does the automation of certification affect legal liability? If automation is relied on for certification and it errs, is it appropriate (legally) to blame the machine or to blame the certifier whose judgment accepted the machine's error? With respect to the issue of user trust in automated systems, it is important to emphasize that, within the FAA, the certification process represents the formalization and operationalization of trust. When an Airway Facilities specialist certifies a system, that specialist formally and legally expresses the FAA's conclusion that the system is trustworthy. When the specialist ceases to trust a system, he or she formally decertifies the system. Therefore, when a certified system fails, the issue of trust extends through multiple orders: the controllers may question not only their trust in the system and its equipment, but also their trust in the individual who certified the system. This introduces the possibility of mistrust in the qualifications of the certifier (and therefore in the process by which the certifier was "certified to certify") and in the process of equipment/system certification, which ultimately and formally relies on the professional judgment of the certifier. One response to these concerns has been the suggestion that the certification process should be as automated as possible—in which case the question arises: Who will certify that certifier?
OCR for page 185
Flight to the Future: Human Factors in Air Traffic Control Restoration to Service: An Example of Teamwork The contributions of both automated equipment and its human users to the cause, complication, or resolution of major outages should be considered. Human error, particularly within the staff who control the automation equipment, can cause or contribute to outages. One option frequently considered by Airway Facilities specialists when complex systems demonstrate performance decrements is to do nothing, since experience has shown that frequently performance decrements are transient, and complex systems sometimes salvage themselves. One general rule followed by experienced specialists is: analyze before you act. This suggests that important features of automation are the extent to which it contributes to system self-stabilization, the extent to which it supports system analysis, and the extent to which it discourages human error. Troublesome system problems are not restricted to the catastrophic. Small, infrequent problems can sum to produce an unstable system. These problems can be produced by software bugs, errors in data transmission or storage, timing errors, and subtle design deficiencies not detected in formal acquisition tests. Johannssen (1992) estimated that between 1987 and 1992 there were approximately 4,000 reported software problems in the national airspace system, of which about 1,600 were not resolved by 1992. The FAA specifies a procedure for filing, maintaining, and resolving program technical reports in response to such problems. A significant question regarding the application of automation is: Will Airway Facilities specialists be able to effectively restore equipment and systems to service when (1) the equipment or systems that have failed contain automation on which air traffic controllers rely heavily to perform their duties and when (2) Airway Facilities itself relies on automation to perform the restoration, but the automation has failed or is difficult to work with? Improper design or application of automation to both Air Traffic Services and Airway Facilities can produce a double indemnity situation that complicates extremely any problems relating to failure of the automation supporting air traffic tasks that are the focus of this report. Airway Facilities has always shared with controllers the responsibility for and the philosophy of maintaining the safe and efficient flow of air traffic; it is open to question whether the Airway Facilities roles within the team will actually increase with increased automation, or whether increased automation will require that the controller roles expand into supervisory control functions currently performed by Airway Facilities. The function of maintaining automation software, which traditionally resided within the Air Traffic Services organization, has recently been transferred to the Air Facilities organization. This has helped to eliminate the traditional difficulties associated with assignment of responsibility for hardware and for software to different organizations, especially when diagnosis of problems is at issue. The new Airway Facilities job classification emphasizes
OCR for page 186
Flight to the Future: Human Factors in Air Traffic Control that the GS-2101 specialist must be able to understand and work with new automated technologies from a systems engineering perspective. The responsibility for restoration also highlights the need for teamwork within the Airway Facilities sector itself. Despite the recent reclassification of most electronics specialists to the GS-2101 classification, Airway Facilities remains staffed with technicians whose knowledge base represents depth in specific systems and their associated items of equipment (e.g., radars, communications, navigational aids, computers). The workforce does include operations managers whose understanding spans the systems, but they typically rely on the in-depth knowledge of system specialists. The FAA recognizes that, since facilities may not be staffed with representatives of all disciplines on a 24-hour basis, a frequent requirement during restoration will be to call back needed off-duty specialists. The GS-2101 classification seems to rely heavily on the assumption that the anticipated systems will contain embedded automation that will relieve these system-level specialists of the requirement to understand their in-depth functioning. If that assumption is incorrect, the utility of the GS-2101 classification with respect to the callback problem must be questioned. In addition, since Airway Facilities personnel currently tend to become individual specialists in particular areas (e.g., radar, communications, navigation, or equipment or subsystems within these areas), they informally rely on one another's expertise to solve problems. A move toward more breadth of responsibility may affect the dynamics of Airway Facilities teamwork. Teamwork is the focus of recent study by the FAA. The FAA Civil Aeromedical Institute is conducting research in the following areas: knowledge and skills that predict successful membership in and leadership of self-managed teams; tools to assess the progress of work teams; organizational culture factors that inhibit or facilitate acceptance of new technology by the Airway Facilities workforce; and methods for introducing new technology (e.g., quality circles, town hall meetings, goal setting, teaming). Workload In Chapter 6 we discussed issues relating to workload in general and to air traffic control in particular. Airway Facilities is also subject to the problem of sudden workload transition from low troughs to high peaks. Scheduling of preventive maintenance and certification tasks is currently a commonly applied method to average workload. Airway Facilities also schedules tasks that affect controller operations in coordination with Air Traffic Services, taking into consideration controllers' workload. The most significant workload challenge for Airway Facilities personnel occurs when multiple critical elements fail, creating or threatening service outage. Under these situations they are faced with the complex task of rapidly diagnosing the cause from the pattern of failures, simultaneously
OCR for page 187
Flight to the Future: Human Factors in Air Traffic Control assessing the progress of the diagnosis, logistics support factors, and the utility of applying alternative solutions to maintain or restore service. Acquisition of Airway Facilities Automated Systems Chapter 11 of this report describes the FAA's procedures for development and acquisition of new systems and makes recommendations pertaining to the consideration of human factors issues and the appropriate involvement of human factors professionals during all phases of system development, acquisition, implementation, and test and evaluation. Those recommendations apply with equal force to Airway Facilities equipment. The application of human factors to the process of designing and developing the human-computer interaction for the air traffic controllers' workstation during acquisition of automated systems benefits from two facts: these workstations are highly visible concerns during the acquisition process, and it is widely appreciated that the introduction of new systems must fundamentally preserve the current integration of these workstations. The situation with respect to Airway Facilities has been quite different. As mentioned previously, the maintenance control center (MCC) is a concatenation of disparate workstations without an integrated human-computer interface. Simply expanding the MCC to assimilate additional workstations for new systems will foster idiosyncratic human-computer interface designs that may exhibit internally consistent application of human factors principles but fail to integrate with other MCC designs. The FAA has recently required that all new systems provide data in standard formats to the remote monitoring system that feeds the MCC, but this still allows for idiosyncratic design of the MCC human-computer interface and of automation for each new system. There is a significant need for the specification of an MCC human-computer interface into which all new designs must fit well, and a corresponding need for an overall MCC automation strategy against which proposed automation designs can be evaluated. The FAA has recently produced the Human Factors Design Guide (HFDG) for Acquisition of Commercial-off-the-Shelf (COTS) Subsystems, Nondevelopmental Items (NDI), and Developmental Systems (Wagner et al., 1996). This design guide is intended to overcome the limitations associated with using commercial and military human factors design standards within the FAA environment. The current version of the design guide focuses on ground systems and equipment managed and maintained by Airway Facilities. The FAA plans to expand the design guide to address air traffic control operations, aircraft maintenance, aircraft and airborne equipment certification, and regulatory certification for aviation personnel. The design guide is not intended to be a substitute for in-depth professional practice. The design guide should be maintained and updated in accordance with results from a systematic, continuing program of human factors research. Efforts,
OCR for page 188
Flight to the Future: Human Factors in Air Traffic Control including the use of such guidelines, to standardize the application of new technologies should include attention to issues pertaining to the application of automation to support Airway Facilities tasks—especially monitoring, certification, and restoration tasks—as well as issues pertaining to the human-computer interface characteristics of associated equipment, including maintenance control centers, off-line diagnostic tools, maintenance logging tools, and software development tools. STAFFING Selection and Demographics The GS-2101 job classification, which now covers the majority of Airway Facilities electronics technicians, is likely to require change in the population from which hirees are selected. There is no known FAA documentation of the strategy for identifying this population or for determining the precise relationship between selection criteria, performance during training, and on-the-job performance for the GS-2101 specialists or for any other electronics specialists. The same concern for identifying relationships between automation factors, selection criteria, and the GS-2101 knowledge and skill requirements will apply if the FAA decides to pursue the notion of developing a GS-2101 selection test. As discussed in Chapter 4, demographic data suggest that the Airway Facilities workforce will see, within 10 years, a simultaneous retirement of significant percentages of its experienced technicians and the equipment on which they have developed their experience. This suggests that the introduction of the GS-2101 job classification is quite timely—fostering the hiring and training of new types of people for new types of equipment—but it also adds to the urgency of validating the GS-2101 hiring and training devices and procedures. Training It is noteworthy that the training track specified for automation specialists not only includes instruction in general hardware and software aspects of computers and in computer programming but also emphasizes knowledge of specific computer-based systems such as the HOST computer, display channel processors, and MCC operations. There is currently no training track that specifically addresses the position descriptions of the GS-2101; therefore, GS-2101 trainees currently receive tailored instruction selected from among the pool of instructional sources that were developed to train the specialists in radar, navigation, communications, and automation (computer systems). The GS-2101 position descriptions and qualification standards emphasize knowledge and skills characteristic of systems engineers, with a focus on how systems interact to produce services. In contrast, existing training materials have been developed to effectively
OCR for page 189
Flight to the Future: Human Factors in Air Traffic Control train technicians in the operation and maintenance of specific systems and equipment. There is a clear need for the development of a training program and associated course content for the GS-2101 job classification; the current courses of instruction do not address the systems engineering requirements of the GS-2101 qualification standard. An associated concern looks in the opposite direction. The rationale for the GS-2101 job classification relies partly on the expectations that new systems are likely to automate current equipment- and subsystem-level monitoring, diagnostic, and reconfiguration functions. It also relies on the expectation that these systems will be modularized to permit automatically failed components to be removed, replaced with equivalent modules (pull-and-replace maintenance), and returned to the manufacturer for repair. The concern is that these assumptions may lead to the conclusion that the GS-2101 can focus on system- and service-level activities, relying on automation to monitor and control lower-level functions—and that, consequently, training for these lower-level functions can be eliminated. Such "dumbing down" of training would be suspect in the light of two questions: What will the GS-2101 do when the automation fails, and how will the GS-2101 maintain proficiency in these automated tasks? There is currently no discernible consistent philosophy within Airway Facilities that governs the maintenance of skills and proficiency during the use of automated systems. It is also noteworthy that despite the heavy reliance on teamwork during its operations (i.e., reliance within Airway Facilities on the cooperation of domain experts and general reliance on cooperative work between Airway Facilities and Air Traffic Services), and despite the emphasis in performance appraisal on performance within the team context, training for Airway Facilities team operations is not in evidence in the FAA Catalogue of Training Courses. Performance Appraisal The development of new GS-2101 course work will require associated development of new examinations. The GS-2101 position descriptions will have to form the basis for the development of any associated personnel certification examinations and performance ratings. In each of these enterprises, there will be the challenge of addressing the question: How can the performance of the technician be distinguished from the performance of the machine when tasks are automated? This question currently applies especially to such tasks as certification, monitoring, and control. Detailed analyses must be performed for each system to determine how to characterize the GS-2101 supervisory control tasks to permit effective performance appraisal.
OCR for page 190
Flight to the Future: Human Factors in Air Traffic Control Job Satisfaction In Chapter 8 we discuss the interaction between organizational culture (which includes the perceptions and attitudes of employees) and formal organizational context factors (formal structure, policies, and procedures), as well as their combined influence on job performance, noting the importance of job satisfaction as a key element of organizational culture. Chapter 8 also describes the FAA's employee attitude survey (EAS), which includes indicators of job satisfaction, and reviews their results for air traffic control employees. Key results of the 1995 EAS for Airway Facilities employees are summarized here; these results, and related conclusions, are very similar to those reported for air traffic control employees. The reader is encouraged to refer to the discussion of the EAS in Chapter 8 for more detailed considerations relating to interpretation of the EAS results and related conclusions. Sixty-eight percent of Airway Facilities employees reported that they were either highly or very highly satisfied with their jobs. The 1995 survey showed a 22 percent increase, since 1988, in the percentage of Airway Facilities respondents reporting that they were either highly or very highly satisfied overall; however, reported overall job satisfaction dropped 6 percent from the previous (1992) survey. For reasons detailed in Chapter 8, such findings do not indicate conclusively a net increase in satisfaction with all of the key facets that contribute to overall job satisfaction. In 1995 Airway Facilities employees were especially satisfied with pay, benefits, and the nature of the work itself, but less satisfied with factors contributing to the job environment, including working conditions, the supervisor, the organization, and opportunities to develop potential. These results, combined with a steady decline in reported confidence that FAA management would utilize the EAS results to improve working conditions and morale, suggest that, like air traffic controllers, Airway Facilities employees perceive a need for changes in management style and/or structure. Some suggestions for those changes emerge from examination of the results relating to specific organizational context variables. EAS results indicate that Airway Facilities employees are highly satisfied with their understanding of how their jobs contribute to the FAA mission. However, they are only moderately satisfied with the adequacy of management's communication of policies and with their opportunities to express concerns openly and with impunity. These results occur against a backdrop of agency-wide EAS results indicating that employees are only moderately satisfied with their involvement in decision making, and that many employees do not feel free to discuss problems with their supervisors. Chapter 8 details conclusions related to similar EAS results for air traffic controllers. These conclusions, which apply as well to the results for Airway Facilities personnel, include, in brief: (1) Management should recognize possible discrepancies between the intent of their policy communications and the interpretations drawn by employees, especially with respect to policies that address safety
OCR for page 191
Flight to the Future: Human Factors in Air Traffic Control and efficiency. (2) Management should involve employees in policy development and encourage open expression of concerns. (3) Employee empowerment is a means of aligning responsibility with authority, and empowerment must be internalized within the culture. (4) Employees' perception that communication is inhibited should be addressed as a means to develop improvements in other reported areas of dissatisfaction. In the discussion of training and personnel development above, it was reported that the FAA has established a tailored program of training and retraining, as required, for Airway Facilities specialists. EAS results, however, indicate that Airway Facilities employees are only moderately satisfied that they receive the training needed to perform effectively, and only about 40 percent of Airway Facilities employees reported that they are satisfied with opportunities to develop their potential. The process of performance assessment for Airway Facilities specialists is also discussed above. The EAS results indicate that Airway Facilities employees experience generally low to moderate satisfaction with the extent, timing, and appropriateness of recognition and rewards for exceptional performance. The ongoing process of revising the appraisal and training processes for Airway Facilities specialists should include careful consideration, with continual feedback from Airway Facilities specialists, of how performance and potential should be trained and appraised and how rewards and penalties should be tied to the results of appraisals. Airway Facilities employees reported low to moderate satisfaction with the impact of new technologies on their jobs—including both appropriateness and timeliness of the introduction of new technology. This lack of satisfaction forms part of the cultural backdrop against which user involvement (or noninvolvement) transpires during the acquisition of new systems, as discussed in Chapter 11. HUMAN FACTORS RESEARCH Until very recently there has been an extreme paucity of human factors research pertaining to Airway Facilities at either of the FAA's major human factors research organizations, the Human Factors Research Laboratory at the Civil Aeromedical Institute (CAMI) and the Research and Development Human Factors Laboratory at the FAA Technical Center in New Jersey (Collins and Wayda, 1994; Stein and Buckley, 1994; Human Factors at the FAA Technical Center: Bibliography 1958–1994). A notable exception is Blanchard and Vardaman's (1994) development of an outage assessment inventory to study a broad range of factors relating to equipment and system outages. Blanchard and Vardaman's study concluded that adequate understanding of the factors contributing to outages must include attention to the following variables: system and equipment design factors (reliability, accessibility, level of automation and built-in testing, and degree of automated switching to redundant components); human behavioral processes (information
OCR for page 192
Flight to the Future: Human Factors in Air Traffic Control gathering and interpretation, knowledge base, problem-solving and decision-making strategies, and planning); personnel factors (training and experience level, skills, availability and assignment of personnel, staffing levels, shift scheduling, and management); logistics factors (maintenance philosophy, parts availability, job training aids, and availability and operability of test support equipment); and physical environment factors (travel time to equipment and physical characteristics of the maintenance environment). Blanchard and Vardaman suggested that these factors should be studied in relation to each of the following tasks required for restoring failed equipment: detection of the outage, scheduling of the maintenance, assignment of the maintenance technician, traveling to the equipment, preparing maintenance tools, diagnosing the faults, repairing or replacing components, verifying and certifying the equipment or system, and logging the maintenance report. Blanchard and Vardaman's conclusions represent one promising framework for investigating, within the context of a standard task sequence, variables that may interact with automation to mediate the effectiveness of automation applied to Airway Facilities. Very recently the FAA has developed plans for, and initiated on some fronts, human factors research pertaining to anticipated changes in Airway Facilities job tasks, workstations, skill requirements, demographics, selection procedures, training needs, and organizational structure and culture. The FAA is developing plans for human factors research on automation in Airway Facilities. The FAA's Plan for Research, Engineering, and Development (1995b) proposes that a plan be developed to conduct and apply research in the following areas: task analyses to provide the necessary data for developing knowledge, skills, abilities, position descriptions, and training criteria for current and future positions that work with automated systems; guidelines for the development of human-computer interfaces for Airway Facilities workstations; assessment of factors that affect human performance of Airway Facilities activities; development and validation of selection criteria for Airway Facilities; criteria for effectively using intelligent systems in Airway Facilities maintenance; analyses of organizational effectiveness for work with current and future systems; and analysis of the workload associated with Airway Facilities tasks. The plan proposes extensive use of rapid prototyping and simulation. It suggests a consistent long-term evaluation of proposed new technologies that Airway Facilities must monitor and control and that can be applied to assist its operations. The FAA's National Plan for Civil Aviation Human Factors (1995e) proposes the following avenues of research related to both Air Traffic Services and Airway Facilities: develop concepts and guidelines for applying human factors to the design of human-machine interfaces for automated systems; identify the workload and performance implications of applying automation; investigate transitions between low and high workload; analyze new classes of error that result from new technology and procedures; examine methods to articulate and coordinate
OCR for page 193
Flight to the Future: Human Factors in Air Traffic Control a human-centered automation philosophy; investigate how overautomation and lack of appropriate feedback to the operator can create performance problems; identify the conditions that may lead to overreliance or underreliance on automation; resolve issues related to the degradation of basic skills with associated performance implications should the automation fail; study situation awareness during the use of automation; and identify selection, training, and performance requirements associated with new systems. Recently initiated research at the FAA supports, in part, the research agenda outlined in the National Plan for Civil Aviation Human Factors . It focuses on the impacts of anticipated technology, including automation, and includes: human-centered automation studies of human factors considerations in advanced operations control centers (OCCs); development and validation of selection, training, and assessment methods for Airway Facilities; and study of the organizational impact of new technologies on Airway Facilities performance (Federal Aviation Administration, 1996). The FAA Technical Center is applying the resources of its Research and Development Human Factors Laboratory to study human factors considerations in advanced OCCs. Using its OCC test bed, rapid prototyping, and operational scenarios the FAA Technical Center is studying the operational suitability of the human-computer interface characteristics of proposed OCC concepts and designs, including those that involve the introduction of intelligent systems. The laboratory is also applying virtual reality technology to the visualization and analysis of candidate layouts for advanced OCCs. As the OCC concepts develop, the FAA's plan is to apply the resources of the laboratory to human factors evaluations of conceptual designs and prototype systems, translate research findings into human-computer interface requirements and specifications for modifications to existing systems and for future systems, and perform human factors acceptance testing of OCCs. Related efforts include the development of job task analyses, the identification of knowledge, skills, and abilities, the construction of human performance and workload models, and the definition of team structures—all of which characterize anticipated changes to Airway Facilities personnel and activities. The FAA's plan is to maintain feedback between these efforts and the Technical Center's efforts to develop and evaluate OCC workstations, so that design requirements will reflect consideration of personnel, team, and procedural factors. Toward the development and validation of selection, training, and assessment methods for Airway Facilities, the CAMI human factors researchers are collecting data on job tasks, biodemographics, personality characteristics, and results of post-hire assessments, Academy training examinations, field equipment certification tests, and on-the-job performance evaluations. the immediate goal of this research is to validate methodologies for the post-hire assessment of Airway Facilities specialist knowledge, skills, and abilities (used to decide whether specialists may bypass training courses), as well as other characteristics
OCR for page 194
Flight to the Future: Human Factors in Air Traffic Control relevant to training placement and job performance. The longer-term goal of this research is to develop new or additional tests incorporating innovative assessment methodologies, such as computer-adaptive testing. This research also contributes to the wider, long-term refinement of the Airway Facilities systems model for assessment, recruitment, and training (Airway Facilities SMART) program, whose goal is the development of improved assessment and placement tests, instructional technology, and recruitment strategies appropriate to the anticipated OCC environment. CAMI researchers are performing studies to identify knowledge and skills that predict successful membership in and leadership of self-managed teams and are developing tools to assess the progress of Airway Facilities work teams. CAMI human factors researchers are also developing and validating an organizational culture survey that will be used to study the organizational impact of new technologies on Airway Facilities performance. The FAA plans to include in this study examination of various methods for introducing new technology, including quality circles, advanced training, management town hall meetings, goal setting, and teaming arrangements. Dependent variables for the study include direct measures of performance (e.g., time to identify defective equipment, time to begin repairs and to restore equipment, and number of equipment failures per period) as well as culture variables such as morale and attitude. The goals of the study are to assess attitudes toward new technology; identify relationships between culture factors, organizational structure, and performance; and evaluate methods for the effective introduction of new technology into the Airway Facilities work environment. The general plans for human factors research pertaining to Airway Facilities, described above, suggest the need to develop knowledge based on empirical findings, and those plans recommend the application of that knowledge to the design, development, and evaluation of new systems. The high level of detail at which the plans are discussed, the general issues planned for consideration, and the instances of ongoing research confirm that: (1) attention to the human factors of Airway Facilities is at its inception, and (2) the FAA has recognized the value of investigating several of the Airway Facilities human factors issues identified in this report. Although ongoing human factors research is consistent with several areas of concern identified in this report, the scope of Airway Facilities human factors research remains very small by comparison with the FAA's air traffic control research efforts. It is a central theme of this chapter that ongoing and impending changes to Airway Facilities technology and personnel are dramatic and require significant human factors attention. Ongoing research should continue and expand according to the plans stated in the Human Factors Research Project Initiatives (Federal Aviation Administration, 1996). Additional research should be conducted across the spectrum of research areas defined in this report and in the National Plan for Civil Aviation Human Factors (Federal Aviation Administration, 1995e).
OCR for page 195
Flight to the Future: Human Factors in Air Traffic Control CONCLUSIONS The impact of automation when new equipment or systems are introduced is often experienced more by Airway Facilities specialists than by air traffic controllers. Even at centralized maintenance control centers. Airway Facilities specialists can find themselves faced with a variety of new technologies, provided by different vendors, with varying levels of automation and inconsistent human-machine interface design strategies. The current process of modernization involves the piecemeal introduction of new technologies into the national airspace system, with associated requirements for changes to operations and personnel activities, in a manner that has placed Airway Facilities in a state of flux. The workforce has been aging with the equipment it maintains, and the FAA is faced with the prospect of hiring ''modernized" Airway Facilities specialists in conjunction with modernized equipment and systems. The near-term challenge will be to maintain operation of existing systems while phasing in new ones whose complexity and increased automation are likely to demand new job skills. Automation has not been applied on a large scale to support decision-making and problem-solving functions such as system-level diagnosis of faults from patterns of failures; trend analysis of system performance; prediction of failures; certification of equipment, systems, and services; and the development of restoration strategies—which are critical functions performed by Airway Facilities in support of air traffic control. Changes in the definition of Airway Facilities job responsibilities and associated qualification standards, combined with a tendency to label new computer-based systems as "automated," suggest that the FAA has not established clear and distinctive definitions for the terms automation, modernization, and computerization. Masking of the distinctions between these terms has been one factor detracting from detailed analysis of new systems with respect to: the precise allocation of functions between human and machine that each new system introduces; the resulting impact on the performance of job tasks; associated requirements with respect to selection, assignment, training, maintenance of proficiency, and performance appraisal of Airway Facilities specialists; appropriate criteria and methods for the evaluation and testing of the systems; human-computer interface requirements; and avenues of research required to support effective system design. The term automated should not be used as a label for systems that are not fully automated. Definition of new systems should include clear identification of which tasks are automated, which tasks are performed by the human, and which human-performed tasks involve the use of automated support. The description of new systems should not focus on whether they contain automation, but, rather, on which specific functions they automate. Detailed human factors guidelines (such as the Human Factors Design Guide for Acquisition of Commercial-off-the-Shelf Subsystems, Nondevelopmental Items, and Developmental Systems) should be maintained and updated in accordance
OCR for page 196
Flight to the Future: Human Factors in Air Traffic Control with results from a systematic, continuing program of human factors research. Efforts, including the use of such guidelines, to standardize the application of new technologies should include attention to issues pertaining to the application of automation to support Airway Facilities tasks—especially monitoring, certification, and restoration tasks—as well as issues pertaining to the human-computer interface characteristics of associated equipment, including maintenance control centers, off-line diagnostic tools, maintenance logging tools, and software development tools. There is no clear strategy evident for the application of automation to the systems management tasks of GS-2101 specialists. The GS-2101 job classification will require revision to the procedures for selection, assignment, training, and performance appraisal of Airway Facilities specialists. Specific requirements for work with automation should be defined for the GS-2101 job classification. Team training should be developed to support both Airway Facilities team activities and cooperative Airway Facilities and Air Traffic teamwork. The Employee Attitude Survey (EAS) results indicate that, whereas Airway Facilities personnel are highly satisfied with the nature of their work, they are only moderately satisfied with their working conditions, opportunities for training and for developing potential, processes for appraising and rewarding performance, opportunities to communicate concerns with impunity, and the ways in which new technologies have been selected and applied. Management should address these perceptions by facilitating communication and by effectively involving Airway Facilities specialists in the development of new equipment and procedures.
Representative terms from entire chapter: