Click for next page ( 20

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 19
GOMS can be viewed as a simplified and more parameterized, compiled COG. Moran (1981), however, stresses two contrasts. First, where COG incorporates a limited mental mode! of the system in its semantic level, GOMS incorporates no mental mode] whatsoever. GOMS incorporates only the knowledge required to perform a task. Second, where the focus of COG is the functional description of various levels of user knowledge and the mappings between these levels, the focus of GOMS is the sequencing of operators and the time requirements for each. The bottom line for GOMS is predicting performance times. Kieras and Poison (1983) simulate users' behavior on partic- ular computer systems. They have two representations in their simulations, which with an additional twist can be viewed in much the same spirit as Moran's (1981) view of the relation between COG and GOMS. In the Kieras and Polson (1983) model, a job task representation describes the person's understanding of when and how to carry out tasks (very much like GOMS). The simulated user's behavior is responded to by a simulation of the system, a device representation, which is a GTN of the states and transi- tions between them in the system. Some knowledge of this sys- tem behavior, a mental GTN, can represent what the user knows about the system- a thin, surrogate mental model. The former GOMS-like representation is the user's knowledge that produces performance, while the latter, the mental GTN, could be the user's theory during learning, problem solving, and explaining how the system works. HOW USERS' 1iNOW[EDGE AFFECTS THEIR PERFORMANCE The discussion up to this point has treated what the user knows as a static structure. While we have alluded to its un- derlying role in behavior (learning, problem solving, explanation, skill), we have not focused on these behavioral processes per se. Nevertheless, this aspect is critical both to assessing the empirical content of current analyses and to determining how these analy- ses might be applied to practical problems like the design of user interfaces and training materials. 19

OCR for page 19
Chaos and Misconception m Both Navices and Exerts Learning involves internalizing, constructing, or otherwise at- taining a representation of the system being learned. How does this process proceed and what are its early results? The summary picture is of a halting and often somewhat nonconvergent pro- cess of problem solving and invention (e.g., Bott, 1979; Mack et al., 1983; Rumelhart and Norman, 1981~. Indeed, the models that learners spontaneously form are incomplete, inconsistent, unstable in time, overly simple, and often rife with superstition. A person may develop an understanding that is adequate for simple cases but that does not extend to more complex cases. For example, Mayer and Bayman (1981) found that users of calculators often believed that evaluation only occurs when the equals key is pressed. Scandura et al. (1976) describe a student who concluded that the equals key and plus keys on a calculator had no function because they caused no visible change in the display. Norman (1983) describes learners who superstitiously pressed the clear key on calculators several times, when a single key press would do. People learning to use a simple programmable robot developed wrong analogical models of its behavior that they accepted without -testing unfit the models failed to predict the actions the robot took (Shrager and Klahr, 1983~. Mantel (1982) found that users performing a task in a menu-based retrieval system developed and maintained simplistic sequences of actions that were eventually ineffective in accomplishing their search goals. Chaotic and misconceived conceptual models are not merely an issue of early learning and something that users outgrow. Expe- rienced users hold them as well. For example, Mayer and Bayman (1981) asked students to predict the outcomes of key press se- quences on a calculator. Even though all of the students were experienced in the use of calculators, their predictions varied con- siderably. For example, some predicted that an evaluation occurs immediately after a number key is pressed, some predicted that evaluation occurs immediately after an operation (e.g., plus) key is pressed, and some predicted that an evaluation occurs immedi- ately after equals is pressed. Rosson (1983) found that even experienced users of a text editing system often had rather limited command repertoires, rou- tinely employing nonoptimal methods (such as making repeated 20

OCR for page 19
local changes instead of a single global change). Even in large pow- erful systems, most of the activity involves the use of only a very small portion of the system. In the case of UNIX, for example, 20 of the available 400 commands accounted for about 70 percent of the usage (Kraut et al., 1983~. Like the Mayer and Bayman work (1981), this suggests that even an extensive amount of experience does not necessarily lead the user to a complete, consistent, or even correct conceptual model. There are some things about a system that most users never learn. SkiBed PerfoImance Human performance analyses have been well developed in ve- hicular control (e.g., aircraft, ship, automobile) and target pursuit tasks. Many of these analyses explicitly hypothesize a mental model of the system being operated (e.g., Baron and Levison, 1980; Jagacinski and Miller, 1978; Pew and Baron, 1983; Veld- huyzen and Stassen, 1976~. In these cases, the mental model is used to anticipate the response of a dynamic system and hence to overcome the deleterious effects of time delays either from other humans or hardware. These models have produced good descrip- tions and predictions of human performance. Because these-models deal with spatio-temporal trajectories, their applicability is limited to continuous detection and movement tasks. In contrast, episodic models of movement that incorporate an additional, abstract level of description in terms of discrete situation-action pairs have much in common with goal-action mod- els in human-computer interaction. Discrete representational and data reduction techniques developed for episodic skilled perfor- mance (Jagacinski et al., in press; Miller, 1985) may prove useful in the domain of human-computer interaction. Software user tasks do, however, typically involve a larger set of situation-action pairs than is covered in human performance analyses, and they proba- bly involve more varied categorization and planning by the human operator. Whether they can be generalized to the greater cogni- tive complexity of human-computer interaction tasks is an open question. If we assume that knowledge of simple sequences is in the form of goal-action pairs, then we should be able to apply what we know from traditional verbal learning studies about the retention of paired associates (e.g., Hilgard and Bower, 1975; Postman and 21

OCR for page 19
Stark, 1969) to predict which systems will be easy to learn and what kinds of errors will occur. For example, presumably, those systems that have few paired associates to be learned or those that have distinct, nonconfusable goal-action pairs will be easy to learn and remember. T ~ . ~ , - A~` ~ ~ ,' Knauer et se. t.Y6~', Barnard et al. (1981), and others have explored certain aspects of this issue with muted results. Lan- dauer et al. (1984) cliscuss the difficulties of constructing command names that are natural, that is, those that would have existing goal-action paired associates in memory and ready to transfer eas- ily to a new task. They argue that if one incorporates command names generated by naive users, these names are natural but often are not distinctive enough to allow users to keep from getting them confused among each other. Preexisting paired associates can help transfer, but if they are not distinct paired associates as a set (e.g., A-B may be good until it must be learned along with A-C), the confusion can offset any positive effect from their naturalness. Poison and Kieras (1984, 1985) embody the GOMS model in a production system-based simulation of users' behavior while using software. This is a very concrete representation of what the user knows when performing well-learned tasks and has a number of confirmed behavioral correlates. Their analyses postulated that -the number of productions (the number of rules needed to decom- pose goals into subgoals, to find methods to fit the subgoals, and to execute the sequence of actions in a method) necessary to perform a task is a good predictor of the time it takes to learn a system, that the number of productions that two systems have in common predicted the ease of learning the second after the first, that the number of productions used in constructing the next overt action predicted the delay from one overt action to the next, and that the number of items held temporarily in a working memory predicted the likelihood of errors or delays (Kieras and Bovair, 1985; Kieras and Poison, 1985; Polson and Kieras, 1984, 1985; Poison et al., 1986~. Some of the predictions afforded by this specific analysis have been successfully tested; others are being tested now. Though this approach is to be lauded for its specificity and the accuracy of some of its predictions, its weakness lies in de- termining how one counts the number of productions required for a task. Since production rule formalisms are general program- ming languages, a single function can be programmed in many ways. Consequently, for purposes of replicability, it is important 22