Page 295
EPILOGUE
Institutional Impediments to Effective Training
The role of organizational values, attitudes, and structures in enhancing or impeding individual and team performance was not on the committee's agenda, nor were the committee members chosen for their expertise in that domain. Similarly, neither of the two previous books of the committee dealt with this topic. Yet, after almost a decade of work on issues of performance, we are struck by the key role of the organizational context in which performance occurs.
This epilogue is a product of the individual and collective experiences of committee members, past and present, during more than two dozen site visits. What we have encountered repeatedly during such site visits is most curious: an openness to changes that might improve individual or team performance coupled with institutional and organizational reasons why those changes cannot be implemented. We have gotten this messageto a greater or lesser extentfrom people in a wide range of military, commercial, governmental, and educational settings.
In short, what has become apparent to us is that specifying the techniques and innovations that do and do not have the potential to enhance individual and team performance is only part of the battle. Without an organizational culture that fosters the changes needed to implement those innovations, proposals for change, however credible their source or convincing the evidence, will have little effect. This fact, however, is hardly news to most trainers and other practitioners. The purpose of this epilogue is to take the next step, that is, to specify some of the institutional attitudes and constraints that, in the committee's experience, appear to be the principal organizational impediments to improving human performance.
Page 296
THE PERCEIVED VALUE OF TRAINING
The assessment of techniques and innovations that might enhance training has been a continuing theme during the life of the committee. Based on its analysis of existing research, the committee has suggested certain innovations and argued against certain existing practices. In general, individuals responsible for training in various real-world settings have responded positivelyeven enthusiasticallyto the committee's recommendations. They have frequently argued, however, that it would really not be feasible, given the institutional realities of their particular job setting, to change existing training programs in ways that would implement those recommendations.
One argument is that the resources necessary to make the changes are not available. Training and retraining programs are not usually high among an organization's priorities, which translates into little and often inadequate funds, time, and personnel being assigned to the training mission. In part, the low priority assigned to training is based on financial considerations that are intrinsic to the nature of training: however fruitful training programs might be from a cost-benefit standpoint, the costs are immediate and the benefits are long term. Whenever the short-term bottom line is the primary concern of individuals responsible for management decisions, allocating resources to create or upgrade training or retraining programs will not be an appealing strategy. Such programs not only require expenditures, they also result in the temporary loss of production of the employees being trained or retrained. (''Training is a slice out of your profits," said one company official.) The benefits of such programs may also reflect well not on current management personnel, but, rather, on their successors (which runs counter to the principle, as a member of the Los Angeles Police Department put it, that one should only do enough so that "the bridge falls down when the next mayor is in office").
In addition to such financial considerations, however, training often seems not to be valued in absolute terms. Training programs and the people involved in those programs frequently have less than exalted positions or status in an organization. Such programs (particularly retraining programs) are frequently viewed as a necessaryor even unnecessaryevil. Retraining, refresher, and counseling programs, rather than being viewed as a normal part of improving ongoing job performance in a difficult profession, are viewed as remedial or disciplinary measures. It is little wonder, then, that such programs are frequently viewed by employees as punishment-as a sign that "you screwed up." In times of budgetary crises, such attitudes toward the value of training can add to the reasons that funds for training are among the first to be cut.
During recent years, for example, when being a police officer has become an ever more difficult, dangerous, and complicated job, funds to recruit and train officers have been cut in many communities. Excelling as a
Page 297
police officer requires a variety of motor, procedural, and interpersonal skills, and even, occasionally, that the wisdom of Solomon be executed in a second or two. Yet, in California, the basic course of training for a police officer can satisfy statewide requirements with as few as 560 hours of training. In contrast, cosmetologists in the state of California are required to undergo 1,600 hours of training simply to qualify for the state examination. In the city of Los Angeles in 1991, when no additional funds could be found for training, almost $15 million was found to pay the costs of lawsuits against the police for excessive use of force and unlawful shootings. During 1992 those costs rose to nearly $20 million. In 1993, as a consequence of a number of changes, one of which was a greater emphasis on and resources devoted to training, such costs to the city were dropped to less than $11 million. How much of the $9 million savings should be attributed to improved training, and how much to other factors (such as changes in police procedures and more effective work by the city attorney's office) is difficult to say, but it is worth noting that those savings dwarf the total funds allocated by the city of Los Angeles to training, per se.
The needs that drive training programs and determine their content often have little to do with such fundamental considerations as what skills are most necessary, complex, called on most frequently in the real-world environment, or most likely to be forgotten. Rather, administrative decisions as to how training time and resources are spent are often guided by regulations and fear of lawsuits. At the Nuclear Training Center in Connecticut, for example, the single consideration that is probably most influential in determining the content of training programs is the anticipated nature of upcoming certification testing by examiners from the Nuclear Regulatory Commission.
The way fear of lawsuits can influence the allocation of training resources is illustrated by an example cited by training personnel at the Los Angeles Police Academy. When two officers, out of a total of over 8,000 officers, shot dogs under circumstances where the justification for doing so was questionable, the immediate reaction was to propose that all officers should receive training on when shooting a dog is, or is not, justified. The proposal was eventually scrapped. The point is not that such training is without value, but, rather, that such an administrative reaction was guided by considerations other than a reasoned analysis of the best use of limited time and resources. Another example in the police world that illustrates that anticipated job demands are not the principal guide to training is the following: whereas 50 percent of police calls involve "dispute management," that is, intervening in conflicts and arguments between individuals, less than 1 percent of training time, until recently, had been devoted to dispute-management training.
In general, training programs are often not as effective as they might be because training is not highly valued. The converse, of course, is true as
Page 298
well: training programs are not highly valued because they are seldom as effective as they might be. Thus, a type of "catch 22" impedes progress.
SELECTION VERSUS TRAINING
One reason training programs are not as effective as they might be is a prevailing tendency to attribute differences in performance among individuals not to differences in level of training, experience, or practice, but, rather, to differences in innate ability. For whatever combination of reasons, the role of aptitude is overestimated and the role of practice, experience, and effort is underestimated in performance. (See Ericsson et al. [1993] for a recent example of the type of research findings that suggest that practice, not innate ability, is typically the larger factor in determining performance.)
The belief that the ability to perform well on a given task is a function of whether a person possesses the relevant talent or "gift" has a number of negative effects on organizations and individuals. First of all, it engenders a type of helpless attitude; people hope that they or the others they hire or work with have the "gift," so to speak, and they think there is nothing much to do if they do not. To the extent that an overemphasis on innate ability as a determinant of performance is a societal belief, it can function as a self-fulfilling prophecy: an early bad experience or poor performancein a mathematics course, for examplecan lead a person to think that he or she has no potential in that domain, which then, in turn, influences the path that the person follows. People avoid educational or job contexts that might give them the experience and training to succeed in domains where they have categorized themselves as without talent; conversely, they seek out contexts and roles that exercise talents they think they might have, which then fosters the development of those abilities. Stereotypes as to what innate abilities the members of different racial or ethnic groups tend to have and not have can also function in a self-fulfilling fashion.
At an organizational level, the innate-ability fallacy leads to an emphasis on selection rather than training. Resources are spent on trying to find individuals who possess an innate talent or characteristic of some type rather than on creating programs of training and experience that can improve performance in a given job context. Assessment instruments designed to give self-insight or insight into others are extraordinarily popular in a variety of real-world settings, even though credible evidence is lacking that such instruments actually enhance the selection of careers by individuals or the selection of individuals by organizations. The use of such instruments was examined in the committee's last report. The committee concluded that the widespread use of such instruments was based on considerations such as face validity and personal testimonials, rather than on solid evidence attesting to their effectiveness. For example, in the case of the Myers Briggs Type Indicator (Myers and McCaulley, 1985), which is probably the most
Page 299
popular of such assessment instruments, the committee was unable to find research evidence sufficient to justify its widespread use in career counseling (see Druckman and Bjork, 1991:Ch. 5).
MISUNDERSTOOD ASPECTS OF TRAINING
Errors
A generalization that emerges strongly from this report (see Chapters 3 and 4) and from the committee's last report (particularly Chapters 3 and 4) is that training procedures should introduce desirable difficulties for the learner. Performance during training is an unreliable indicator of the extent to which the learning that is the goal of training has been achieved. Conditions that yield a high rate of correct responses during training can fail to support performance in the posttraining environment; conversely, conditions that appear to slow or impede performance during training can enhance the subsequent real-world performance that is the target of the training.
Training regimens need to introduce the difficulties, unpredictability, and variability expected to be present in the posttraining setting. Manipulations of training that amount to crutches that prop up performance artificially-such as massing practice on a given subtask or keeping the conditions of practice constant and, hence, predictable-not only impede learning, but can also lead to illusions of comprehension or competence. Trainees who perform well under artificially easy training conditions can gain a false confidence in the extent to which critical knowledge and skills have actually been acquired. Introducing certain types of difficulty during training is "desirable," therefore, not only to enhance the learning process but also to educate the learner's subjective experience-that is, to provide real feedback to the learner as to the level of knowledge or skill that has, or has not, been achieved.
The foregoing conclusions suggest that quite dramatic changes are necessary in many existing training programs in a variety of institutions. In the committee's experience, training programs are usually designed to optimize performance during training. In part, that is so because individuals responsible for training act on the reasonable, if fallacious, assumption that there is a one-to-one correspondence between the conditions that enhance performance during training and the conditions that enhance the long-term learning that enhances performance on the job. Errors made during training are generally not viewed as opportunities for learning, but, rather, as evidence of a less-than-optimal training program. Thus, the role of errors and mistakes during training is poorly understood.
More important, however, is that the meaning of errors is misunderstood.
Page 300
The tendency to attribute differences in performance to differences in innate ability means that errors are to be avoided. To the extent that errors and mistakes are not viewed as a necessary aspect of an effective training program, but as evidence of questionable aptitude or abilityby both trainers and traineesthey are to be avoided. Certain mottos that seem common in Army training environmentssuch as "We do it right the first time" or "We don't practice mistakes"seem to reflect such a mistaken view of the role and meaning of errors.
It would be misleading, however, to imply that such attitudes pervade all Army training. A striking counterexample is provided by the National Training Center in California, where units are brought in from around the United States to engage a socalled opposing force (OPFOR) regiment in a series of tank and infantry battles. The OPFOR, a highly trained and practiced regiment stationed at the Training Center, is nearly unbeatable on its home turf1,000 square miles of harsh, uninhabited desert and mountains. The typical visiting unit is defeated decisively in the initial exercises, but it becomes much more competitive as training proceeds. Every misstep is analyzed in a unique after-action review that follows each engagement and permits communication across all levels of command. The basic idea is that there is more to be learned from defeat than from victory and that such learning is better accomplished in simulated battle than in actual combat.
Tests
Like errors, the role of tests as a component of training is commonly misunderstood. As highlighted in the committee's first report (see Druckman and Swets, 1988:Ch. 4), there is abundant evidence that tests are learning events. Information that is recalled and procedures that are carried out become more accessible to learners than they would have been without tests. And tests can increase the effectiveness of subsequent study opportunities, partly by providing feedback to the learner as to the information or procedures that are in need of further study. The importance of testing as a pedagogical device is supported not only by controlled experimentation in laboratory settings, but also by studies of educational environments. (See, e.g., the summary in the New York Times by Fiske [1990], which reports the results of the Harvard Assessment Seminar on those aspects of the college environment that do and do not enrich learning.)
The optimal use of tests as a component of training programs is often impeded by a focus on tests as assessment devices. Ideally, there should be a clear distinction between testing that is embedded within training as a pedagogical tool and testing that is administered at the end of training as an assessment tool. That distinction is typically blurred in actual training programs: as a consequence, trainees are afraid to volunteer answers that might be wrong or
Page 301
to speak up when they are confused or uncertain, and instructors are hesitant to use tests that might induce instructive errors (in part because they themselves may be assessed in terms of the scores of their trainees). Errors or uncertainties committed or admitted by a trainee becomeformally or informallypart of that trainee's record. In certain highly monitored training programs, such as operator training in the nuclear power industry, there are even regulations requiring that certain types of errors and failures made by trainees during the training process must be reported to the appropriate regulatory agency. Management personnel in some such settings are even vulnerable to being charged with "negligent retention" of a given trainee on the basis of that trainee's performance during the training process.
One part of the problem, once again, is the tendency to attribute performance differences across individuals, however localized and temporary those differences may be, to differences in innate ability. As an overall generalization, trainers and trainees alike are too distressed by errors and mistakesand too encouraged by successes and rapid improvement. Examples abound of trainees who appeared to perform perfectly at the end of training but who could not perform adequately months later in the posttraining environment, especially if the posttraining conditions differed from those of the training situation. (The fact that medical students, at the time of graduation, could remember only about 10 percent of the basic-science material they had presumably mastered during the first 2 years of medical school was one of the factors that led Harvard Medical School to move away from the traditional model of medical school education.) And errors during training may preclude rather than portend errors in the posttraining environment. In fact, constructing the conditions of training so as to avoid or minimize errors may simply defer those errors to a time and place where they matter much more.
Measures of Effectiveness
It almost goes without saying that the appropriate measure of a program of training or instruction is the extent to which that program facilitates posttraining performance. That is, the goal of training is to "transfer" that traiing in positive ways to the real-world settings in which the trainee will work. For a variety of reasons, however, measures of posttraining job performance are frequently missing or of questionable validity. And when appropriate measures exist, there may be no feedback loop: that is, there may be no administrative machinery in place that provides information to training personnel as to the actual performance of their trainees months or years after training.
If measures of the long-term consequences of a given training program tend not to be available to the people responsible for training, what do they use to evaluate different methods of training? The answer is that they tend
Page 302
to use one or both of two unreliable measures that have the potential to be very misleading: the performance of trainees during the training process and the evaluation of a given training program by the trainees themselves. As noted above, performance during training is a poor guide to choosing those conditions of training that maximize posttraining performance. Constructing the conditions of training so as to yield the maximum rates of correct performance during training will tend to result in a training program that stresses such undesirable characteristics as massed practice on subtasks, fixing the conditions of practice, and providing solutions and answers rather than providing opportunities for those solutions and answers to be generated by trainees themselves (see Druckman and Bjork, 1991:Ch. 3).
Trainees' ratings of their own happiness or satisfaction with a given training program are an equally unreliable basis for the design of training programs. Such ratings, frequently referred to as "happy sheets" or "smile sheets," are subject to the illusions of comprehension and competence noted above, illusions that may well be fostered by the types of manipulations that enhance performance during training, but fail to support posttraining performance. And the types of desirable difficulties that enhance learning, in part by exercising those processes likely to be demanded in the posttraining environment, are unlikely to be well received by trainees, almost by the very nature of such manipulations.
Most trainers may uncritically assume that trainee happiness and performance during training are appropriate criteria against which to evaluate training. But even trainers who understand that such criteria are faulty still face a problem in attempting to introduce innovations in training of the type the committee has recommendedbecause they themselves may well be evaluated in terms of the performance of their trainees during training or in terms of the happy sheets filled out by trainees. To really optimize training requires that supervisory personnel, not just the individuals who have the day-to-day responsibility for training, understand the practical implications of the committee's conclusions and recommendations on training.
Finally, when well-defined measures of the long-term consequences of training do not exist, training personnel lack a way to demonstrate the product of any special efforts and innovations on their part. It is demoralizing to believe that if you do a good job no one will know. Had Jaime Escalante (of Stand and Deliver fame) been teaching a standard honors course in calculus at Garfield High School in east Los Angeles, rather than an advanced-placement course, he might well have labored in vainor possibly, been firedbecause only students in the advanced-placement courses take a nationwide end-ofyear test. That test provides not only a measure of student achievement, but also a measure, if an imperfect one, of instructor effectiveness. Without the undeniable achievements of his mostly minority students on the advanced-
Page 303
placement test, Escalante's unconventional teaching techniques might well have been viewed as simply eccentric and probably ill-advised.
TRAINERS AND ORGANIZATIONS
The extent to which a trainer can maximize his or her effectiveness as a teacheror will even try to do sodepends heavily on the organizational attitudes and structures that characterize the work environment. To optimize training there needs to be communicationbetween instructors, across administrative levels, and between former trainees and current training personnel. And there needs to be cooperation rather than competition: that is, there need to be mechanisms to share knowledge, solutions, and innovations that appear promising on the basis of posttraining results. In actual practice, however, such communication and cooperation is frequently impeded by the attitude that the ability to teach is an innate talent, not a skill to be learned, and by administrative structures that isolate instructors or put them in competition with each other.
Teaching as a Skill
Teaching is a complex skill. To be a maximally effective instructor is itself a continuing and demanding learning process. Staying current with respect to the knowledge and skills that are to be taught is one necessary aspect of the learning process, of course, but doing only that much is far from sufficient. One needs also to work toward mastering the craft of instruction, which is a multifaceted and life-long process. To be most effective, an instructor needs to stay abreast of advances in high technology tools for trainingsuch as computer-assisted devices of one kind or another, needs to stay current with respect to research findings that have significant implications for training methodologies, and needs to explore systematically the relative effectiveness of alternative technologies and techniques in the particular training context. Beyond those aspects of the process, there are important things to learn about one's self as a teacher, about the overall mission of one's institution or organization, and about one's students or trainees. There are many styles of teaching, for example, and it may take some time and effort for a person to determine which of those styles is most personally effective and comfortable. The most effective style may also differ as a function of the age, background, and goals of one's trainees. Finally, understanding how the knowledge and skills to be taught "fit in," so to speak, from an organizational standpoint-in terms of the demands one's trainees can be expected to face or in terms of other training those trainees are receiving or will receiveis also an important, and continuing, process.
Page 304
Management personnel, and instructors themselves for that matter, are prone to view teaching not as a craft to be learned, but, rather, as a gift bestowed on certain individuals. The potential negative consequences of such an attitude are considerable. To the extent that an individual instructor views the ability to teach as an innate talent, criticism of his or her teaching, however constructive and specific, will tend to be either rejected as a kind of personal attack or accepted as evidence of limited potential, for example. One will be disinclined to seek advice and feedback, and to explore alternative techniques and methods.
At an implicit or explicit level, the notion that the ability to teach well is an innate talent is remarkably prevalent. Even in university settings, teaching tends not to be viewed as a skill to be learned. At lunch and elsewhere, professors talk to each other about research, politics, sports, the weather, and the stock market, among other things, but rarely, if ever, about teaching strategies and techniques. It is as though talking about such matters is off limitspossibly because one is at risk of implying that a colleague has failings as a teacher or that one has an elevated opinion of one's own "gifts" as a teacher.
From a management standpoint as well, the attitude that teaching is mostly or entirely an innate talent has negative consequences. One such consequence is a decreased likelihood of support for programs to upgrade and refine the skills of training personnel. The notion that the ability to teach well is a gift creates instead a tendency to simply hope that individuals selected as trainers have the "right stuff" in the first place. Another consequence of the failure to view teaching itself as a difficult skill is the tendency of organizations to recruit experts in a given domain to be instructors in that domainwithout regard to their credentials or experience as teachers.
Expertise in a given domain hardly disqualifies one as a teacher, of course, but experts may not only lack experience and knowledge of those teaching principles that transcend particular domains, but may also lack an understanding of their own skill or be unable to adopt the perspective of a novice. A high level of expertise in golf, or writing computer code, or preparing tax forms, for example, is no guarantee that one can effectively teach those skills. Someone who grew up on skis may be less able to explain to a beginner how to turn, or stop, or get up again than a less expert person who learned to ski as an adult. At a very high level of expertise and practice, many aspects of complicated skills become automatic, which can make them unavailable to conscious analysis without special effort. One reason that teaching one's own children to drive an automobile is alternately frustrating, humorous, and terrifying is that so many aspects of skills as experienced drivers have become automatic over the years. When asked about the appropriate timing and sequencing of the shifter, clutch, and gas
Page 305
pedal in a standard-transmission car, for example, a person is often reduced to trying to observe what he or she does when shifting, which, typically, alters and disrupts the process.
In sum, being an effective instructor in a given domain goes beyond having expertise in that domain. It seems quite obvious that being good at something is not the same thing as being an effective teacher of that somethingafter all, it is common for elite musicians and athletes to have teachers and coaches who are not themselves elite performersbut that perception persists. Such a perception may explain, for example, why the manuals accompanying personal-computer software and hardware are frequently so frustrating and ineffective as instructional tools. It seems plausible that the writers of such manuals have frequently been selected primarily on the basis of their intimate knowledge of a given productan engineer or computer technician, perhaps, who played a significant role in designing or refining that productwithout regard to their skills, or lack thereof, as a writer or instructor or their skill in adopting the perspective of a learner.
Administrative Structures
Another contributor to nonoptimal training is organizational structures that act to isolate instructors. If it is true, as has been argued by a number of influential writersparticularly W. Edwards Demingthat the behavior of individuals within an organization is more heavily determined by that organization's structure than by characteristics of those individuals, then many instructors, unfortunately, are working in settings where they will never achieve their potential as teachers. In corporate, military, and educational settings, instructors can find themselves denied the types of communication and cooperation necessary to optimize the training for which they are responsible.
A number of historical and institutional factors may contribute to the isolation of teachers and trainers within organizations. One such factor is an assembly-line mentality toward training. Students or trainees are viewed as needing to be "fitted" with skills and knowledge that will later be demanded of them. Given that view, it may seem optimal to subdivide training into a number of nonoverlapping and narrowly defined programs or classes, the goal being to achieve a kind of mass-production efficiency. Trainees or students can be sent to different training programs or classes, as necessary, where it is an instructor's job to attach to those trainees skills or knowledge of some type. Over time, however, such a structure will frequently not only act to isolate instructors, but may also put them in competition with each other. A given class or training program becomes the province of an instructor or staff of instructors, who then come to view their primary goal as being more highly rated by supervisors and trainees than
Page 306
are other instructors, a goal that is not commensurate with optimizing the long-term effectiveness of training.
Whatever the factors that act to isolate instructors within organizations, the effect of that isolation is to prevent or slow the rate of desirable changes and innovations within training programs. Instructors need the opportunity to learn from each other, and individuals in key management positions need to view themselves as partners with instructors in the training enterprise. In fact, some of the innovations required to optimize the total training mission of an institution-introducing technological tools to enhance training, for example, or changing how individual training programs are interleaved and interrelated-can only be accomplished at the management level. Administrators, who possess the power and, unfortunately, often the inclination to stop innovation, also frequently possess the power to foster and implement desirable changes.
CONCLUDING COMMENTS
Having focused on certain impediments to effective training, and having attempted to illustrate those impediments, we have perhaps painted an excessively gloomy picture of what the committee encountered during its many site visits. Were it the goal of this chapter to provide examples of real-world training environments that are exemplary in one or more respects, that would not be difficult to do. We were impressed, in fact, by the potential for innovation, communication, and cooperation we saw illustrated across the range of military, commercial, educational, governmental, and sports settings we visited. In short, although it became clear to the committee that the impediments to effective training we have identified in this chapter are commonplace in real-world environments, it also became clear that they need not exist.
A final point that merits comment is that the impediments to effective training summarized in this chapter are not entirely independent of each other. The counterproductive attitudes, values, and structures that impede training arise, to a greater or lesser extent, from a common root: a misunderstanding of the characteristics and potential of humans as learners. The body of research on the cognitive and social processes that underlie the learning and performance of individuals and teams has grown to the point that it is a far better guide to training than is intuition or standard practice. In an era of global competition and information superhighways, when the survival value of being able to learn and change is greater than ever before, it is critical to draw on that resource to enhance training.