Click for next page ( 24

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 23
for Kieras and Polson (1985) to specify further what production language style underlies these production analyses, and further, whether this style can be argued to be consonant with a the- oretically reasonable mode} of the architecture in which human information processing operates. The chief limitation of the GOMS analysis is that it considers only error-free performance. This is a serious limitation since even skilled users spend at least a quarter of their time making and recovering from errors. In GOMS, goals are very specific to task situations; they are not currently in a general form (Card et al., 1983~. This is not a limitation in principle, however, since GOMS is a deliberate simplification of Newell and Simon's (1972) general problem solver, a mode! that is general enough to describe any goal-directed behavior. Robertson (1983) suggests how error and error recovery could be incorporated into a GOMS-like analysis. Rumelhart and Norman (1982) present a performance analysis of skilled typing that takes the description of errors as a primary concern. The treatment of errors in their analysis raises an im- portant issue. In order to describe the occurrence of some kinds of errors, they were forced to change the assumption of how in- formation is stored in memory. The analysis was fundamentally altered in order to qualitatively predict the typical errors for the task. This raises the question of whether GOMS, in which only error-free behavior was analyzed, embodies a representation that can be generalized to real performance that includes errors. APPLYING WHAT WE KNOW OF THE USER'S KNOWLEDGE TO PRACTICAL PROBLEMS The foregoing discussions have reviewed various representa- tions of the user's knowledge of a system. We have described them in terms of the theoretical representations posited and some of the cognitive processes included in each type of analysis. It seems safe to conclude that while the area of research on users' mental representations is very active, it is not yet well developed (see the Research Recommendations section below). Nevertheless, software human factors is an applied area, and there is continual pressure to apply what we do know in this work to the task of design and training. Applying what we know about mental representations to prac- tical ends raises many questions. For example, if we knew what 23

OCR for page 23
the user knew, how would we use this knowledge in design? Do we build the user interface to reflect a consistent mental model? If so, what does the input and presentation look like? Should we tell the learner what mode} to build? Designing Interfaces If the interface suggests or reflects an appropriate model, then the user could conceivably learn it with less guidance and perform it with fewer errors. The question is: What should the mode! be? One approach to picking a mode! is to design user interfaces to accord with naive user conceptual models (Carroll and Thomas, 1982; Mayer and Bayman, 1981~. Although this approach is simple and straightforward, its general utility is open to question. For example, Wright and Bason (1982) designed two software packages for a casual user population. One package was designed to be maximally consistent with the users' prior knowledge; users were asked how they thought about their data and what they wanted to be able to do with it, and this formed the basis for the user interface. The second package was also designed with input from potential users, but in this case, the designer used this information to determine how the users ought to think about their data and operations on it. The finding was that, in every way, the second package was a better design. In a similar vein, Landauer et al. (1983) replaced the verbs in a word processor's command names (like append, substitute, and delete) with those that secretaries generates! most often when describing to another secretary how to change a marked up manu- script (such as add, change, and omit). Paired associate learning theory would have predicted that these well-learned goal-action pairs from the secretaries' own vocabulary would have been good command names for secretaries learning a new word processor. The goal-action pairs are presumably preexisting paired associates, ones not needing new learning. Learning the word processor with these command names, however, was no better than learning the one with the system developers' names or even one with random names like allege, cypher, and deliberate. Naive users do not necessarily design better systems. A variant of the naive mode! approach is to enter into the design process with a preconceived model, and then to iteratively build a prototype, test it, and refine the design (including the user's 24

OCR for page 23
model) until acceptable usability is attained. This technique is the classic empirical approach (Dreyfus, 1955~; it has been employed in recent designs that use the desktop metaphor in the interface for office systems (Bewley et al., 1983; Morgan et al., 1983), as well as in other application system designs (Gould and Boles, 1983; Dixon et al., 1983~. The theoretical problem with this approach is that in the context of iterative and often radical redesign of a user interface, it ~ difficult to clearly separate the effect of the mode} on usability from that of other aspects of the redesign.4 A second design approach is to reduce the problem of commu- nicating an appropriate conceptual mode! to the user by simpli- fying the system and its interface. DuBoulay et al. (1981) stress this in their characterization of a glass box mode] that consists of only a small number of components and interactions, all oh viously reflected in the feedback that learners get from running the system. Carroll and Carrithers (1984) implemented this- ap- proach by providing new users with only a small but sufficient subset of commands to learn. This small set fits a relatively sim- ple conceptual model. Carroll and Carrithers (1984) called this the "training wheels" approach, borrowing the analogy from learning to ride a bicycle. Once the subset of commands was learned, the user was gradually introduced to more complicated or more rarely used commands. This approach led to faster and more successful learning. An important question raised in this work, however, is how to decide which subset of commands is sufficient to do the task and fits a simple model. Furthermore, it raises the question of how to embellish the initially simplified conceptual mode} so that the change does not disrupt the learning the user has already accomplished. A third design approach focuses on the method that the user learns rather than on the mental model. Moran (1981), Reisner (1981, 1984), and Young (1981) all stress the potential utility of task-oriented knowledge for design. Such knowledge can be represented formally. The suggestion is that these representations can be examined or manipulated prior to actual construction of the user interface to determine the least complex organization for 4 However, Olson et al. (1984) highlight the importance of running prototype tests with two prototypes that differ in only one variable at a time, so that the effects of individual design changes can be measured independently. 25

OCR for page 23
the interface. For example, the designer can calculate values of merit for a system based on the number of rules In a grammar, the number of different terminal symbols, or various other metrics known in computational linguistics.Th~s approach could also make it possible to define precisely concepts like consistency: similar tasks or goad should be associated with similar or identical actions (Moran, 1983~. For example, deleting a sentence ought to have similar actions to deleting a paragraph. Empirical work has shown the importance of such concepts (e.g., Barnard et al., 1981; Black and Sebrechts, 1981; Thomas and Carroll, 1981; but see Landauer et al., 1984, for a caveat). It should be noted that analysis of these relations, like consis- tency, may not go very far toward describing the interface design fully. For example, two interfaces with exactly the same grammat- ical description of a command language may have very different visual layouts. The visual layouts may lead to performance dif- ferences not predicted by a calculated complexity measure that is based only on inconsistencies in the command language. With the exception of Dunsmore (1986), most grammars do not rem resent features of visual layout that are known to be unportant. Dunsmore (1986) predicted and then experimentally verified that a crowded display would be more difficult for users to deal with than an uncrowded one. Furthermore, with the exception of Shneider- man (1982), whose multiparty grammars can be used to describe both a user's action and the system's response, there has been little attempt to integrate models of the various components of a sys- tem. Moreover, optimizing a design with respect to a task-oriented analysis will not necessarily include any of the design considera- tions that would be indicated by optimizing the presentation of a good mental model. User Taming If a system has been built to conform to a consistent mode! or a well-formed set of methods, training may simply involve presenting the user with the mode} or methods. Several researchers have been concerned with developing techniques for providing users with appropriate conceptual models, something that even state-of-the- art instructional materials for software often fad] to do (Bayman and Mayer, 1984; DuBoulay et al., 1981; Halasz and Moran, 1982~. The benefits from presenting a mental model, however, are unclear. 26

OCR for page 23
SchIager and Ogden (1986), for example, incorporated both a method representation and a mental mode! in the training mate- rials for teaching students how to form successful queries in a data base. For those specific query types that fit the mode] or meth- ods presented, both representations speeded learning, regardless of whether the representation was a method representation or a mental model. Errors and difficulty occurred only when queries were different from the method or mode! taught. Mayer (1976, 1981) provided students with a diagrammatic too! which incorporated a variety of concrete metaphors (e.g., input as a ticket window and storage as a file cabinet). Students who were exposed to this too] before studying a training manual were later able to perform better on both programming and recall tasks. Kieras and Bovair (1984) taught people how a simple device worked either by a rote sequence of steps, with a mode! of the system, or with an analogy. The sequence of steps showed them what to do when. One mode! displayed what part was connected to another part beneath the surface, as if a flow diagram were painted on the control panel. The analogy described the control pane} as being part of a mock spaceship, explaining what each control knob did in terms of battle-related actions. The results showed no benefit from either of the models over the rote sequence. On closer inspection, Kieras and Polson (1985) noted that neither of the models gave the user any action-oriented help; the models merely gave a story about what the connections were, not how they worked. Halasz and Moran (1982) taught students how to use a cal- culator using either a step-by-step action sequence to do stan- dard calculations or instructions which included a verbal model of how the internal registers, windows, and stacks worked. They found that performance on standard tasks was identical for the two groups, but that the group who learned the model performed better on novel tasks. Foss et al. (1982) provided a file folder metaphor to students learning to use a text editor. They found that students who were provided with the metaphor learned more in less time. In the same domain, Rumelhart and Norman (1981) used a composite of three metaphors: a secretary metaphor, which was used to explain that commands can be interspersed with text input; a -card file metaphor, which was used to describe the deletion of a 27

OCR for page 23
single numbered line from a file; and a tape recorder metaphor, which was used to convey the need for explicit terminators in files. Although performance was good overall, the fact that there were several metaphors produced cases in which a subject would employ one of the metaphors when another was appropriate. Most of this work has focused on the use of mental models narrowly in training, namely, by telling the student the mode! or by providing simple and explicit advanced organizers (Ausubel, 1960~. In another approach, an explicit mental mode! was pre- scribed; a system's training manual had a diagrarrunatic mode! of control flow for a menu-based system (Galambos et al., 1985~. The resultant benefits were equivocal, however. Even greater integra- tion between mode} and training appears necessary. The feasibility of this approach is exemplified in systems that have mental mode} analyses in their expert systems to interactively diagnose learner problems and to provide tailored support (e.g., Burton, 1981~. No systematic behavioral studies have been carried out, however, to evaluate the effectiveness of this approach. A more theoretical issue in the area of training pertains fun- damentally to the nature of learning and the implications for designing training programs. One view of human learning and memory conceives of learning as an active process of problem solv- ing in which concepts are created by the learner (e.g., Jenkins, 1974; Wittrock, 1974~. This view contrasts with one in which learning is merely the storage of concepts in memory. In the latter view, a learner can be given a conceptual mode! explicitly (by diagram or a verbal explanation). In the active learning view, however, a conceptual model must be invented by the learner after an appropriate series of experiences. Mayer (1980) adopted the active learner view. He asked learn- ers to generate a metaphorical elaboration of programming state- ment types as they were learned. For example, after learning a FOR statement, the student was asked to describe its function using a metaphorical desktop vocabulary. He found that learners who had provided these elaborations were later able to perform better on novel and complex programming problems. Carroll and Mack (1985) suggested that taking a serious active learning view raises the possibility that metaphors are useful not only when they provide familiar descriptions of novel experiences, but also when they provoke thought by failing to accord perfectly with the target of the metaphor comparison. Carroll and Mack 28