National Academies Press: OpenBook

Tactical Display for Soldiers: Human Factors Considerations (1997)

Chapter: 7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION

« Previous: 6 STRESS AND COGNITIVE WORKLOAD
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

7
Research, Development, Testing, and Evaluation

The Army has a well-defined developmental and operational testing program for all new major and minor systems. The complex Land Warrior System, with its range of evolving technologies embedded in the helmet-mounted display, requires even more advanced test programs, supported by the thorough identification of critical issues and criteria and driven by systematically developed measures of effectiveness and measures of performance. In order to achieve a smooth testing process for this complex program, specific performance criteria-which are not presently available-must be identified to allow for unambiguous subsystem evaluation prior to testing the complete, integrated Land Warrior System.

With regard to the design effort for the helmet-mounted display, the lack of user-identified performance priorities makes the development of an effective test and evaluation program more difficult. Key issues in building an effective research program and an effective test and evaluation plan include identification of the technical areas requiring additional research and performance requirements that need discrete, objective evaluation. Historically, human factors research in the Army's programs has been underfunded and understaffed; it has not been a primary driver of revisions to proposed requirements; and it has not played a central role in the planning of creative testing programs. This situation needs to be changed.

In this chapter, we highlight the research that the panel judges to be essential and present a concept for research and testing that effectively incorporates critical human factors considerations for the Land Warrior System's helmet-mounted display.

In order to properly address these issues, a three-tier, highly integrated re-

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

search, testing, and evaluation strategy is proposed. This approach represents a shift from current practice in that it includes an intermediate, semi-controlled set of research and testing experiments between laboratory and bench testing and operational field operations. Equally important, it incorporates the active involvement of users in every stage in the development sequence.

First, do the technologies and capabilities embedded in the helmet-mounted display provide a significant improvement in the soldier's local situation awareness under the expected environmental and operational conditions? Second, does the helmet-mounted display significantly improve the visual perception of a soldier and his ability to perform operational tasks under tactical operational conditions? Third, how does the cognitive workload created by the helmet-mounted display affect task performance by soldiers under tactical and operational conditions? And forth, can current infantry soldiers, with existing skill qualifications, perform their expected operational tasks more effectively with the capabilities provided by the helmet-mounted display? These issues are elaborated in a later section of this chapter.

ADOPTING A NEW STRATEGY

Until the advent of the electronic age, the systems a soldier was required to use were relatively simple mechanical devices, and field testing had only to demonstrate that the equipment would continue to function in mud, rain, and ice. As systems have become more complex, requiring sophisticated training, and electronics-intensive with information transfer aspects, the gap between laboratory or bench testing and operational testing has become much larger. As a result, it is now possible for a system to meet all its engineering specifications in laboratory tests but turn out to be unusable in the field. Furthermore, the reasons for failure in the field may be obscure to the engineering design team, leading to "chase the tumbleweed" redesign efforts. The problem is most likely to be related to the combination of variations in field conditions and variations in the attributes of human users. The failure to recognize and deal with these variables and their interaction can result in incorrect definition or incomplete specifications that lead to limited prospects for success.

We propose a methodology that brings the user into the testing and evaluation process earlier, through controlled testing that combines some varied environmental and personnel conditions from operational testing with the structured data collection and controlled conditions characteristic of laboratory testing. It is proposed that testing and evaluation for the helmet-mounted display be developed in three tiers as follows:

  1. Controlled laboratory or bench testing of technical performance of the system-both with and without human users.
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
  1. Controlled field experiments with a variety of users, from experienced to new entry, and with system experts from the design team involved.
  2. Operational test and evaluation exercises employing soldiers from the target population in virtual-type simulations and live simulations in a realistic operational environment.

The insertion of the mid-level tier of trials allows for the interaction of potential users and the design team in conditions that combine structured data collection with variability in environmental conditions (e.g., day, dusk, night for visual factors; camouflage for terrain variations) and individual variation in users (e.g., effects of regional accents on the performance of a voice recognition system for acoustics).

We list below some guidelines, considerations, reservations, and underlying problems in human performance measurement from the Guide to Human Performance Measurement (American National Standards Institute) that should be kept in mind in designing the testing and evaluation program and user trials:

  • Lack of a general theory to guide performance measurement. At present, only a few basic relationships have been discovered-for example, the inverse relationship between speed of task performance and accuracy, or performance quality.
  • Measurement control and realism. An inverse relationship exists between the control of measurement conditions and operational realism.
  • Behavior is multidimensional. Many factors influence human performance, some of great weight, others of little importance. Moreover, these factors probably change their importance over time and with different measurement contexts, and they may have different weights in different individuals. This results in the need to perform more experiments with more variables, subjects, and test trials.
  • Unclear relationship between objective and subjective data. Objective measures are based on observable behavior (i.e., moving a lever, speaking a word). For complex tasks as well as phenomena that are not directly observable, objective measurements may not be possible or may be too costly to develop.
  • Generalizability. Results may not be generalizable to the real world.
  • Measuring cognitive activity. Cognitive activity is inherently more difficult to measure than physical performance because it takes place within the individual. Cognitive activity cannot be observed directly; it requires analysis of the output consequences of the cognition and, even more important, some kind of self-reporting, which may be tainted by its subjectivity. This means that measurement of behavior continues to be an inherently difficult process that is not becoming easier. In these cases, performance must be inferred from system inputs, outputs, and states, as well as from models of the operator's cognitive processes (Vreuls and Obermayer, 1985).
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
  • Performance criteria. There is a lack of objective performance criteria for most tasks. This lack of criteria makes it difficult to (a) assess performance quality and sufficiency and (b) identify the type of performance measurement techniques required to operationally define significant differences in performance (Vreuls and Obermayer, 1985).

The considerations presented above are not intended to reflect an inability to design and evaluate user trials but rather to point out some potential pitfalls. Later in the chapter, we list the test conditions and criteria that may be useful in any testing or evaluation sequence designed to assess the helmet-mounted display and the Land Warrior System.

PLANNING THE STRATEGY

To be effective, developmental testing of the Land Warrior System and the helmet-mounted display must expand in scope and increase the human and operational variables introduced. The introduction of operational considerations should be executed in a controlled and prioritized manner with a deliberate research and testing sequence using controlled field experiments. To be effective and cost effective, subsequent operational testing should be implemented at the small unit level, minimizing larger-scale testing scenarios. The testing should have increased scope, with longer durations or cycles of task performance than have been used in the past. Soldier in-the-loop testing with increased emphasis on the number of operational considerations should be an early objective.

In recent years, the Army has made major strides in developing, refining, and integrating constructive models, virtual simulations, and live simulations in the training arena. However, these advances have not been effectively applied in the area of test and evaluation, especially in the early stages of the development cycle. Rapidly growing modeling and simulation technologies appear to present a significant opportunity for enhancing the integration of human factors considerations into the development of the Land Warrior program. Simulation technology now presents a real possibility of assessing human performance. The U.S. Army Training and Doctrine Command is developing a series of virtual simulations and simulators that may be very useful in assessing human performance. However, there is insufficient attention to the measurement of human performance in these activities at the present time. Although the virtual simulations in their current form do not meet the design consideration needs for the helmet-mounted display per se, it is an example of simulation approaches that should be considered in planning and structuring the research and testing program for the helmet-mounted devices. As described by the user (U.S. Army Infantry School), the Land Warrior System is not being employed. This void must be filled prior to the development of a detailed test and evaluation program. Modeling and simulation can play useful role in specifying the parameters for use of the Land Warrior System use in the field.

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

By simulating a variety of threat environments, ranges of employment options, and different organizational structures, it should be possible to assess detection ranges, friendly and enemy engagement ranges, and the impact of a host of environmental and electronic warfare issues on the battlefield. The ability to quickly and relatively inexpensively vary potential battlefield conditions would allow the investigation of difficult performance specifications.

As suggested above, the design, testing, and evaluation processes of the Land Warrior program and the helmet-mounted display can benefit from expanding the capabilities and application of constructive and virtual simulations. Such approaches could reduce the amount of operational testing, but they will not entirely eliminate the need for live simulations and tactical exercises. Executing experiments and employment exercises with real soldiers and their leaders in the loop has always been a primary part of the Army's test and evaluation process. If, however, live simulations are to have an impact on the design of the helmet-mounted display, a more stringent scientific methodology must be adopted for early operational testing.

The large number of integrated subsystems that are part of the Land Warrior System make it difficult to measure the effects on human performance of a specific subsystem. The ability to measure the speed and to automatically locate and estimate the range of a potential enemy target, coupled with an automatic request for indirect fires, does not measure the soldier's concurrent potential to detect or failure to detect other, more significant threats that might mitigate the sheer speed of engaging one target. In order to properly evaluate the task-loading effects of the helmet-mounted display and the interfaces within the Land Warrior System, the discrete subsystems, sequences of events, and associated human task loads must be tested in a controlled environment. Such an approach will allow measurement of the addressed factors with a sampling of typical soldiers.

User-Centered Design

In the early stages of the research, development, test, and evaluation process, critical design decisions are often made by scientists and engineers who are knowledgeable concerning the technology but less expert in the operational employment of the new system. Without consistent and thorough collaboration with knowledgeable user representatives, the evolving design is driven by the engineer's and technician's view of what is important or what is possible, with less focus on what is most needed by the user. The Land Warrior System and the helmet-mounted display represent a major step forward in the integration of technology. To be effective, the research, development and design process must have more than user inputs; it must have user involvement and commitment. User-centered design is a significant aspect of the three-tier process described earlier.

User-centered design is an outcome-oriented approach to system develop-

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

ment. This phrase means that user needs, capabilities, limitations, and preferences are not only taken into account when design decisions are being made, but also can actually drive such decisions. It also means that the user is the focus of attention throughout the development, and that the developing system is tested and evaluated in a realistic operational environment early in the development process. The objective is to build a high level of user acceptance and confidence in the system. The Land Warrior System will only be effective if infantry soldiers can achieve enthusiastic confidence in their abilities to perform their mission using it.

The Army development process is well defined. The role of the Training and Doctrine Command as the user representative is well known and accepted within the development community. However, the level of knowledgeable and focused user interaction in the design process is, all too often, lacking. Using the philosophy of user-centered design can aid the accelerated development cycle that has been outlined for the Land Warrior System and the helmet-mounted display.

Critical Test Issues

In order to make human factors a vital part of the test and evaluation process for the helmet-mounted display, it is necessary to identify the critical human factors research questions that must be answered during the test and evaluation process. After careful analysis, the panel identified the following test issues in four broad areas as critical to the design of displays for the individual soldier:

  1. Local situation awareness,
  2. Visual perception,
  3. Cognitive workload, and
  4. Soldier qualifications.
Local Situation Awareness

Do the technologies and capabilities embedded in the helmet-mounted display provide a significant improvement in a soldier's local situation awareness under the expected environmental and operational conditions? An operational way to phrase this would be, "Does a dismounted infantry soldier equipped with a helmet-mounted display have a significantly improved ability to detect, identify, and effectively engage enemy targets over a currently equipped soldier?" The discussion on situation awareness in Chapter 3 addressed both global and local situation awareness. From the soldier's point of view, both his survival and his performance enhancement involve local situation awareness. Under what set of environmental or tactical employment conditions does the helmet-mounted display enhance soldier response time? Clearly, on the basis of what we know from current human factors research and technology development, enhanced

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

performance does not result simply by adding a helmet-mounted display capability. The criteria associated with this test issue deal with the employment conditions under which enhanced performance is sought. Performance comparisons of the currently equipped soldier with a Land Warrior-equipped soldier should be integral to the test plan.

Visual Perception

Does the helmet-mounted display significantly improve the visual perception of a soldier and his ability to perform operational tasks under tactical and operational conditions? This report has addressed some of the major visual perception issues that the design of the helmet-mounted display raises. More research is required to define operational parameters for task performance under likely operational conditions to establish acceptable trade-offs to maximize and enhance human visual capabilities.

Cognitive Workload

How does the cognitive workload created by the helmet-mounted display affect task performance by soldiers under tactical and operational conditions? Measuring cognitive workload is the most significant problem area that affects the design of the helmet-mounted display and the one that is most difficult to objectively quantify in testing. The outcomes of workload measurement are dependent on environmental conditions, individual human cognitive capabilities, and a large number of other variables. The research and testing program must first be focused on isolating the contribution of key variables under controlled conditions.

Soldier Qualifications

Can current infantry soldiers, with existing skill qualifications, perform their expected operational tasks more effectively with the helmet-mounted display and its designed capabilities? A basic assumption of the Land Warrior program is that the system will be used by basic infantry soldiers. This implies that no special selection criteria different from the current standards will be needed. Based on current research in human performance, it is unclear whether soldiers who meet the current Army criteria can operate the Land Warrior System and the complex helmet-mounted display and achieve enhanced performance under realistic battlefield conditions. The testing program for the helmet-mounted display should be designed to investigate the boundaries of effective performance for soldiers who meet the current standards and with soldiers from controlled groups who meet higher and more selective entrance requirements.

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Cross-Cutting Issues

A number of topics relate to more than one test issue. One is the effects of stress on performance; another is the organization of information and the methods selected to display that information. Stress may be caused by physical or cognitive workload or by poorly fitting equipment. Such stresses can influence situation awareness and the ability of the soldier to process and use information. Specifically, although there is some knowledge of separate physical and cognitive workload issues, there is almost no research information on their combined effects. As stress increases, attention narrows to a few immediately obvious informational cues. Battle places enormous stress on the soldier, so one should expect such narrowing to occur. Likewise, there are many unresolved questions related to the stress imposed by poor and ill-fitting equipment. As the ergonomic portions of the program continue, developers need to be informed by the stress reactions of users; otherwise, apparent gains may well be wiped out under actual combat operations. What is needed is much more basic research on this issue, tied to applied research questions associated with stressful performance using the helmet-mounted display.

Questions of display design relate to all four test issues. One area in need of research attention is the hierarchical ordering of information from immediate threat to minor operational concerns and evaluating alternative presentation sequences and formats. Another research area concerns the allocation of information to visual versus auditory channels and the applicability of advanced technology, such as three-dimensional audio, to making these allocation decisions. Also, research is needed on the way in which graphic displays are structured and how these displays are formatted into standard iconic symbols for action. As discussed earlier, it is possible that new information-processing and display capabilities could be used to reduce stress by providing a global help function (e.g., location of nearest friendly force) at all times. A critical area of software development is the provision of this information in a secure manner. One could even imagine that an adaptive interface system could be used to on-load the soldier during periods of boredom and off-load the soldier during periods of high workload.

IMPLEMENTING THE STRATEGY

The panel proposes a three-tier concept of research and testing. The ideas of overlaying developmental and operational testing and expanding the use of simulations are not new to the Army, but embedding a level of controlled field testing with real users is. The Army test community has long recognized the potential of combining developmental and operational testing and evaluation as a cost- and time-saving approach. However, the relative difficulty of assessing technical performance to meet specific technical contract specifications, versus evaluating

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

system performance in typical user's hands under operational conditions, has never been adequately addressed. The latter requires integration of operational performance criteria and user personnel into technical and developmental tests as well as controlled field tests. In doing so, the experimental design features used in behavioral research should be incorporated, including the procedures associated with sampling human subjects that specify the sample characteristics and size requirements needed to draw robust conclusions.

Such testing must be focused on critical performance criteria that can be specified in operational terms. This has been a stated objective within the Army's acquisition guidelines and the test and evaluation community for a number of years. The Land Warrior System Specification (Specification A3246133G) dated February 22, 1995, documents the requirements for the system and includes many operational criteria and parameters. However, the specifications are not at a level of operational specificity to provide insight into the human performance issues associated with the helmet-mounted display. Greater focus is needed on the types of human performance expected under realistic combat conditions. Table 7-1 highlights existing requirements for helmet-mounted display subsystems and the critical human-factors related test issues that must be resolved to permit the necessary specification of design requirements that address human operational performance criteria. In the panel's view, these issues, which relate directly to the broader test issues discussed previously, must drive the helmet-mounted display design and research and testing process. Although cognitive overload is addressed for each component in Table 7-1, it is important to keep in mind that questions of workload cannot be answered in a piecemeal fashion but require the integration of components.

A system's performance requirements should define the environmental conditions under which the system must operate. Human performance is influenced by the single or combined effects of numerous factors that enhance or degrade and/or facilitate or interfere with that performance, depending on the particular situation. These factors, referred to as performance-shaping factors, are aspects of the human-machine system that influence behavior and affect the time or accuracy of the human response (Swain and Guttman, 1980). To be relevant, performance-shaping factors must have the potential for significantly affecting the magnitude of, frequency of, or variability in human performance. Identifying performance-shaping factors and establishing their effect on human performance is an important aspect of the research and testing necessary for designing the helmet-mounted display.

Performance-shaping factors are grouped as operational, equipment, task, personnel, and environmental factors. Operational factors include doctrine and tactics, length of time of use, and system objective. Equipment performance-shaping factors for the helmet-mounted display include the physical parameters, operating characteristics, display layout, and reliability and maintainability. Personnel factors relate to items such as training, experience level of the user, moti-

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

vation, skill level of the user, fatigue, monotony, attitude, and workload. Task factors such as complexity, duration, repetitiveness and environmental factors such as distracting stimuli, vibration, motion, visibility, temperature, and noise must all be addressed in the design of specific test iterations to build a better understanding of the impacts on human performance of use of the helmet-mounted display.

Research and Testing Approaches

Table 7-2 lists the key research issues identified by the panel and proposes a general approach to addressing them. Included are the subsystems of the HMD and the subsystems that interface with the HMD. The test conditions and test criteria listed are representative of the conditions and measures that are most significant. The intent of the table is to list the kinds of conditions that must be addressed to assess human performance differences and the types of standards or criteria that should be considered. As we've said, the intent of the proposed three-tier test approach is (1) to use infantry soldiers in the research during controlled laboratory and bench testing whenever practical as well as under controlled and operational field conditions and (2) to strictly control test variables in an attempt to isolate the human performance differences.

Identifying Measures of Effectiveness

A measure of effectiveness is a measure with a standard or criterion that allows testers or researchers to evaluate performance. A measure of performance is a quantitative indicator without a standard that describes performance but does not evaluate it. The thrust of the proposed research and testing effort for the helmet-mounted display should be directed toward assessing the human performance of soldiers against operational measures of effectiveness early in the development cycle. The intent is to use controlled experiments in a much more operational way and in smaller structured test scenarios. Although it would be desirable to build simulations that allow the conduct of many iterations using actual soldiers, controlled testing can be initiated now and does not have to be executed on a large scale to obtain significant levels of human performance data.

Tables 7-3 and 7-4 illustrate links among the Land Warrior System's mission tasks, potential measures of effectiveness and the proposed measures of effectiveness. The individual soldier tasks using the Land Warrior System are listed in Table 7-3 for a reconnaissance mission are not an exhaustive set. However, they represent tasks that we would specifically expect the individual soldier equipped with the Land Warrior System to perform in an enhanced manner over a currently equipped infantry soldier.

Table 7-4 presents a similar set of tasks for a squad leader equipped with the Land Warrior System linked to a series of potential measures of effectiveness.

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

TABLE 7-1 Land Warrior/Helmet-Mounted Display Capability and Critical Issues

Subsystem Description/Requirement

Critical Helmet-Mounted Display Test/Research Issue

GPS/computer radio subsystem

  • -  

    Provides position location data

  • -  

    Integrates information via software

  • -  

    Operates under all operational conditions

Does the GPS/computer radio subsystem cognitively overload the IIB Land Warrior soldier? (#3)

Weapon subsystem

  • -  

    Integrates target acquisition, location, and marking

  • -  

    Increases speed of engagement

  • -  

    Operates under all operational conditions

Can Land Warrior soldiers more effectively engage targets with the integrated helmet-mounted display sight systems? (#1)

Computer processor in the computer radio subsystem

  • -  

    Executes software programs and computations

  • -  

    Processes sensor inputs

  • -  

    Operable by IIB soldiers (ASVAB scores)

Do the functions of the computer processor cognitively overload the IIB Land Warrior soldier? (#3) Can IIBs who meet current qualification standards effectively employ the CPS? (#4)

Sensor/display assembly (night sensor display)

  • -  

    Displays clear video images

  • -  

    Operates under all operational conditions

  • -  

    Controls permit tactile identification

  • -  

    Adjustments while wearing gloves

  • -  

    Operable by IIB soldiers (ASVAB scores)

Is local situational awareness of the Land Warrior soldier increased with the night sensor display? (#1) Does night sensor display enhance Land Warrior visual perception? (#2) Can IIBs who meet current qualification standards effectively employ the night sensor display? (#4) Do the functions of the night sensor display cognitively overload the IIB Land Warrior soldier? (#3)

Sensor/display assembly (day sensor display)

  • -  

    Displays clear video images

  • -  

    Operates under all operational conditions

  • -  

    Compatible with laser eye protection optical

  • -  

    Operable by IIB soldiers (ASVAB scores)

Is local situational awareness of the Land Warrior soldier increased with the day sensor display? (#1) Does day sensor display enhance Land Warrior visual perception? (#2) Do the functions of the day sensor display cognitively overload the IIB Land Warrior soldier? (#3)

Remote input pointing device

  • -  

    Operates under all operational conditions

  • -  

    Operable by IIB soldiers (ASVAB scores)

  • -  

    Tactile feedback with gloves

Does the remote input pointing device support the workload of the Land Warrior soldier effectively? (#3) Can IIBs who meet current qualification standards effectively employ the remote input pointing device? (#4)

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

Subsystem Description/Requirement

Critical Helmet-Mounted Display Test/Research Issue

Audio amplifier of computer radio system

  • -  

    Allows mixing of computer and radio outputs

  • -  

    Controls overall audio volume level

  • -  

    Provides audio emphasis control

Is local situational awareness of the Land Warrior soldier increased by use of the audio amplifier? (#1)

Audio headset (microphone/speaker)

  • -  

    Receives/transmits radio and computer generated tones and outputs

  • -  

    Useable in airborne and air assault operations

  • -  

    Noise cancellation microphone

Is local situational awareness of the Land Warrior soldier increased by use of the audio headset? (#1) Do the functions of the audio headset cognitively overload the IIB Land Warrior soldier? (#3)

Ballistic helmet assembly (shell and suspension)

  • -  

    Fits 5-95 percent male with improved stability

  • -  

    Standard PSGT tooling (40 percent improve protection)

Does the ballistic helmet assembly provide the required level of stability to support Land Warrior missions?

Hand-held display and mini keyboard

  • -  

    Operable while wearing gloves

  • -  

    Tactile feedback with gloves and NBC

Do the functions of the night sensor display cognitively overload the IIB Land Warrior soldier? (#3)

The last column lists candidate measures of performance that would be used to quantify performance differences. Both tables are representative of the types of objective measures that need to be constructed for the individual soldier, squad leader, and platoon leader for all tactical infantry Land Warrior missions.

The potential measures of effectiveness and measures of performance illustrated in Tables 7-3 and 7-4 are basic operational measures that could be applied in all three tiers of the proposed research and testing process. The intent is to conduct controlled experiments with test and control groups using representative soldiers under operational conditions starting early in the research and testing cycle. Such an approach would facilitate human performance assessment, and at the same time, minimize the amount of follow on operational testing required to support the Army's acquisition process. The key is establishing a consistent set of agreed upon operational conditions that can be applied in all testing phases.

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

TABLE 7-2 Research and Testing Approaches for the Helmet-Mounted Display

Equipment

Design Research

Test Approach

Test Conditions

Test Criteria

Helmet-mounted display and weapon sight

Distance-offset of sensor/display

Controlled field experiments Operational test and evaluation

In buildings and with obstacles

Time to detect targets

Helmet-mounted display/night vision goggle and TWS

Resolution Field of view Chromaticity

Controlled laboratory/bench tests

Controlled field experiments

Operational test and evaluation

± Levels of visibility, distance, background, contrast, target-nontarget similarity

Time to acquire

Time to identify targets

Navigation time and error rate

Obstacle error rate (in lab and operational field conditions)

Helmet-mounted display/night vision goggle and TWS

Monocular versus biocular versus binocular

Controlled laboratory/bench tests

Controlled field experiments

Operational test and evaluation

± Levels of employment, duration and use, light condition

Time to acquire

Time to identify targets

Navigation error rate

Physiological effects

Obstacle error rate

Time to detect unaided after aided

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

Helmet-mounted display and hand held device

Information to be communicated

Display format

Input format

Controlled field experiments

Operational test and evaluation

± Levels of duration, fatigue, stressors, experience

Time to perform tasks

Task error rate

Level of local situational awareness

Helmet-mounted display/audio modality (monaural)

Information cues in display:

Placement, format, types: attentions, warnings

Controlled field experiments

± Levels of visibility, distance, contrast, sensitivity, noise, task loading

Percent cues detected

Time to detect cues

Time to identify information in environment

Helmet-mounted display/3-D audio displays (biaural)

Resolution needed

Types need:

Way-point navigational

Localized communication

Enhanced hearing

Head tracker

Stereo audio

Controlled laboratory/bench tests

Controlled field experiments

LOS radio acoustically controlled; noise at defined volume/ frequency content; visual displays with control target

Outdoors, ± ambient noise; LOS, masked at min/max ranges; visual targets at given distances; day, dusk, night

Percent cues detected

Time to detect cues

Time to identify information in environment

Percent correct

Time to detect target information

Time to identify target

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

Equipment

Design Research

Test Approach

Test Conditions

Test Criteria

 

 

Operational field testing

Field exercise, noise, fatigue, masking, varied ranges and

Percent correct recognition of standard numbers and letters

Percent correct recognition of standard message format content

Speech recognition (N.B.:

Assessing system recognition of user/command voice input, converting to text)

Vocabulary design

Speaker display versus speaker independent

Controlled laboratory/bench tests

LOS radio acoustically controlled environment; insertion of noise at defined volume/frequency content

Percent correct words on modified rhyme test

 

Discrete versus continuous

 

 

Percent correct of standard numbers and letters

 

 

Controlled field experiments

Outdoors, ± ambient noise; LOS, masked at minimum/maximum ranges; visual targets at controlled distances

Percent correct and time for target information detection and identification

 

 

Operational field testing

Field exercise, noise, fatigue, masking, varied ranges-day, dusk, night.

 

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

 

 

 

 

Percent correct recognition of standard message format contents

Recognition and application of message report contents

Soldier/radio

Intelligibility of speech

Controlled laboratory/bench tests

LOS radio acoustically controlled environment; insertion of noise at defined volume/ frequency content

Percent correct words on modified rhyme test

Percent correct of standard numbers and letters

 

 

Controlled field experiments

Outdoors, ± ambient noise; LOS, masked at minimum/maximum ranges; visual targets at controlled distances

Percent correct and time for target information detection and identification

 

 

Operational test and evaluation

Field exercise, noise, fatigue, masking, varied ranges-day, dusk, night

Percent correct recognition of standard message format contents

Recognition and application of message report contents

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

TABLE 7-3 Soldier Battle Tasks and Potential Measures of Effectiveness (MOE)/Measures of Performance (MOP)

Land Warrior Task

Potential MOE

Potential MOP

Receive squad order

Land Warrior soldiers have better knowledge

Time required to issue order

Percent of soldiers able to state mission and location

Move tactically

Moving Land

Warrior soldiers are less detectable

Time required to execute movement

Number enemy Land Warrior detections

Detection range

Identify ORPs

Land Warrior soldiers correctly identify ORPs

Percent ORPs correctly identified

Observe/listen for enemy

Land Warrior soldiers detect enemy activity more effectively

Percent enemy activity detected

Number of correct detections

Conduct reconnaissance

Land Warrior soldiers are able to conduct accurate reconnaissance in less time

Time to conduct reconnaissance

Ratio of enemy Land Warrior detections to Land Warrior enemy detections

Locate objective area

Land Warrior soldiers locate objects more effectively

Time to locate objectives

Percent correct identification

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

Land Warrior Task

Potential MOE

Potential MOP

Observe enemy activity

Land Warrior soldiers detect enemy activity more effectively

Percent enemy activity detected

Number of correct detections

Record information

Reconnaissance results are more complete

Time required to record results

Percent observations recorded

Report enemy information

Land Warrior reports are more timely and accurate.

Time required to deliver report

Number of reports received

Percent complete reports

CONCLUSION

The complex human performance issues associated with the Land Warrior System require a rigorous program of research integrated with effective user oriented field experiments and operational tests. Applying advanced technologies to enhance the performance of infantry soldiers cannot be achieved without also enhancing individual cognitive processing and individual local situation awareness under expected future battlefield conditions. To successfully achieve the optimistic capabilities envisioned in the Land Warrior program, there are significant elements of developmental risk that must be minimized by further research and controlled testing. The research must be completed in time to help drive the design of the helmet-mounted display.

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

TABLE 7-4 Squad Leader Battle Tasks and Potential Measures of Effectiveness (MOE)/Measures of Performance (MOP) Reconnaissance

Reconnaissance Task

Potential MOE

Potential MOP

Receive platoon order

Land Warrior squad leaders have more order complete mission knowledge

Time to receive platoon

Percent squad leaders able to correctly state 5 Para Order details.

Evaluate move route

Land Warrior squad route effective in route evaluations

Time to complete route evaluation

Percent identification of critical features

Move tactically

Moving Land Warrior squads are less detectable

Time to execute movement

Number enemy Land

Warrior detections

Detection range

Control quad movement

Squad movement is more effective

Time to execute movement

Average squad dispersion

Land navigate

Squad leaders navigate more accurately

Time to move and identify ORPs

Percent correct location reports

Identify ORPs

Squads correctly identify ORPs

Percent ORPs correctly identified

Determine location

Squad locations are accurately known

Percent time squad leader correctly locates elements

Report location

Squad locations are accurately reported

Percent time higher command levels know the correct squad location

Observe/listen for enemy

Squads detect enemy

Percent enemy activity more effectively detected

Number correct detections

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×

Reconnaissance Task

Potential MOE

Potential MOP

Conduct reconnaissance

Squads conduct effective reconnaissances

Time to conduct reconnaissance

Percent enemy detections to Land Warrior detections

Locate objective area

Squads locate objectives effectively

Time to locate objectives

Percent correct identifications

Observe enemy activity

Squads detect enemy more effectively

Percent enemy activity detected

Number correct detections

Record information

Squad reconnaissance results are more complete

Time required to record results

Percent observations recorded

Receive enemy information reports

Squad reports of enemy are accurate and timely

Percent enemy activity reported

Number correct reports

Time required to record results

Percent observations recorded

Report enemy information

Squad reports of enemy are more timely and accurate

Percent enemy activity reported

Number correct reports

Time required to record results

Percent observations recorded

Report situation

Squad reports of enemy are accurate and timely

Time required to deliver report

Number enemy reports received

Percent complete reports

Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 164
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 165
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 166
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 167
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 168
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 169
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 170
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 171
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 172
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 173
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 174
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 175
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 176
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 177
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 178
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 179
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 180
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 181
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 182
Suggested Citation:"7 RESEARCH, DEVELOPMENT, TESTING, AND EVALUATION." National Research Council. 1997. Tactical Display for Soldiers: Human Factors Considerations. Washington, DC: The National Academies Press. doi: 10.17226/5436.
×
Page 183
Next: 8 RECOMMENDATIONS »
Tactical Display for Soldiers: Human Factors Considerations Get This Book
×
Buy Paperback | $67.00 Buy Ebook | $54.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This book examines the human factors issues associated with the development, testing, and implementation of helmet-mounted display technology in the 21st Century Land Warrior System.

Because the framework of analysis is soldier performance with the system in the full range of environments and missions, the book discusses both the military context and the characteristics of the infantry soldiers who will use the system. The major issues covered include the positive and negative effects of such a display on the local and global situation awareness of the individual soldier, an analysis of the visual and psychomotor factors associated with each design feature, design considerations for auditory displays, and physical sources of stress and the implications of the display for affecting the soldier's workload. The book proposes an innovative approach to research and testing based on a three-stage strategy that begins in the laboratory, moves to controlled field studies, and culminates in operational testing.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!