3
Diagnosing the Health of Postsecondary Training

Postsecondary training in America is a large and multi-faceted activity, as Chapter 2 described. Is it working well or badly? Answering this question about so diverse an enterprise is not easy. There are areas of strength and areas of serious weakness. An accurate diagnosis, however, is crucial to the committee's ultimate goal of assessing the need for new or revised federal policies.

In evaluating the health of postsecondary training, the committee found itself struggling with two sometimes opposing tasks: analyzing each institution and program or evaluating postsecondary training as a system. It is easier, and tempting, to assess the performance of each institution and each program and to recommend how each should be improved. This is the way most research and analysis in the field has been done. But such a piecemeal perspective begs the critical question: Is the sum of all the activities embodied in existing institutions and programs a postsecondary training system that is accessible, effective, efficient, and adequate to America's needs in a rapidly changing and ever more competitive global marketplace?

While not ignoring the evidence about the individual parts, the committee chose to approach its diagnosis by giving primary emphasis to the system perspective. We will therefore evaluate the postsecondary training institutions and programs by organizing our discussion around five questions relating to the characteristics of a well-functioning system: Is training accessible, and do people have the information they need to select among training options? What do we know about the results of different kinds of training and what works for which people? Do the incentives in the system



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy 3 Diagnosing the Health of Postsecondary Training Postsecondary training in America is a large and multi-faceted activity, as Chapter 2 described. Is it working well or badly? Answering this question about so diverse an enterprise is not easy. There are areas of strength and areas of serious weakness. An accurate diagnosis, however, is crucial to the committee's ultimate goal of assessing the need for new or revised federal policies. In evaluating the health of postsecondary training, the committee found itself struggling with two sometimes opposing tasks: analyzing each institution and program or evaluating postsecondary training as a system. It is easier, and tempting, to assess the performance of each institution and each program and to recommend how each should be improved. This is the way most research and analysis in the field has been done. But such a piecemeal perspective begs the critical question: Is the sum of all the activities embodied in existing institutions and programs a postsecondary training system that is accessible, effective, efficient, and adequate to America's needs in a rapidly changing and ever more competitive global marketplace? While not ignoring the evidence about the individual parts, the committee chose to approach its diagnosis by giving primary emphasis to the system perspective. We will therefore evaluate the postsecondary training institutions and programs by organizing our discussion around five questions relating to the characteristics of a well-functioning system: Is training accessible, and do people have the information they need to select among training options? What do we know about the results of different kinds of training and what works for which people? Do the incentives in the system

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy encourage efficiency and effectiveness, or do they leave elements open to abuse or inadequate performance? How and how well is training connected to employers? How well are the diverse pieces of a fragmented system articulated and coordinated with one another? ACCESS AND INFORMATION Osterman (1990:258) has observed that ''the array of institutions [devoted to postsecondary training in America] is impressive'' and that a strength of this diverse system is that it "is very much driven by individual initiative, and a person who wants to change careers or gain new skills can find numerous ways to do so. . . . [E]early choices are not binding and the system provides chances (for those with the resources) to start over or change direction." Most people live close to a community college, where prices are relatively low and a variety of training options is often available. Numerous other opportunities for training exist in most areas. Nevertheless, access to different kinds of postsecondary training is quite variable, and information about the options available, which would enhance access for individuals seeking training, is often inadequate. Access to Qualifying Training Postsecondary training that qualifies individuals for jobs, when offered on a formal rather than informal basis, is heavily school-based in the United States. Thanks to the wide availability of state-subsidized community colleges and vocational-technical schools with few or no admission requirements and to financial aid to help cover the costs of unsubsidized training programs in private and proprietary schools, there is generally good access to postsecondary qualifying training. Two recent studies, which approach the issue in very different ways, confirm the general accessibility of postsecondary qualifying training. McPherson and Schapiro (1991:188) undertook a comprehensive evaluation of the affordability of postsecondary schooling and the effects of student financial aid policy, reviewing earlier research as well as conducting original econometric analyses on the effects of the net price of an education on enrollments. They concluded that "the combination of state institutional subsidies and federal student aid makes some form of postsecondary education financially accessible to a wide range of Americans." Because prices are very different in different sectors of postsecondary education, however, access to various kinds of postsecondary experiences are constrained by incomes, with low-price community colleges the most readily accessible choice for lower-income people. This does not limit access to training for these people, since many vocation-oriented programs are offered at commu

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy nity colleges. Access to baccalaureate-level education, which takes longer to complete and occurs in more expensive institutions, is more likely to be uneven. Nevertheless, there are some state-to-state differences even in access to training-oriented institutions: not every state has an extensive community college or technical college system, and the willingness and ability of states to subsidize these institutions varies. In general, however, qualifying training is quite accessible for most people, even the economically disadvantaged. Tuma's (1992) review of cross-sectional national data on enrollments in postsecondary schools supports this conclusion. He conducted a detailed analysis of participation in vocation-oriented schooling in the academic year 1989-1990, using the National Postsecondary Student Aid Study (NPSAS) data generated by the National Center for Education Statistics (NCES).1 Like McPherson and Schapiro, he found that vocation-oriented postsecondary schools, even relatively expensive ones like proprietary institutions, were especially accessible to individuals with economic, educational, and other disadvantages.2 Several qualifications should be placed on this generally positive picture, however. Resource constraints at the state and federal levels are affecting both institutional subsidies and the availability of financial aid, which increases the net costs of postsecondary training to individuals and places higher economic barriers in the way, particularly for lower-income people. Restrictions on access to schools that provide postsecondary training are especially worrisome in light of the fact that there are few alternative routes to school-based qualifying training in the United States. Moreover, even if access to schools remains good, there is reason to believe that the relative absence of nonschool-based options is a problem. The Massachusetts Institute of Technology (MIT) Commission on Industrial Productivity (Dertouzos et al., 1989) studied the routes from school to jobs in two sets of countries. The first set, called "pattern A" countries, included the United States, Sweden, and Britain, all of which rely on formal educational institutions for most of the job-oriented preparation of their work forces, including teaching specialized as well as more general skills. In "pattern B" countries, such as Japan and Germany, both general and specialized skills are usually learned on the job. The commission found that ''pattern B countries find it easier to produce workers with the flexibility and skills needed to respond to rapid and unpredictable changes in technology and markets'' (Dertouzos et al., 1989:84). Whether worker training is best provided in schools or in firms is an old debate in the United States (Osterman, 1990:269-270). The question has become salient again, particularly because of deep concerns about the shortcomings of American secondary education and widespread interest in the so-called dual system in Germany, which involves the majority of young

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy people in firm-based, 3-year apprenticeship programs. It is generally acknowledged that many school-based training programs in the United States are divorced from the needs of the workplace, with no connection either to the knowledge and skills needed at work or to the ways in which knowledge and skills are used in the workplace.3 Learning in work-based rather than school-based settings is thought to have a number of advantages, especially for young people (Hamilton and Hamilton, 1992:18). Learning at work seems real and has an immediate payoff that is easy to identify. Knowledge and skills acquired in the context in which they will be used are learned more effectively and powerfully. Work-based preparation occurs among adults, which makes work and learning seem more serious and also allows young people to develop relationships with older mentors. However, the question of whether work-based or school-based training is more desirable is controversial. Berryman and Bailey (1992) cite a number of concerns about work-based learning. The current failure of many employers to provide much formal training to their employees, especially younger and less-educated ones, casts doubt on their interest and ability to serve as major providers of qualifying training. The quality of work-based learning can vary from excellent to awful, depending on the training skills and expertise of those doing the training. School-based educators have been slow to understand and adopt research findings about how to create powerful learning environments; it is unclear why employers would be likely to do better. Scribner and Sachs (1991) show that the key issue for the workplace as a learning place is no different than for school-based learning. Whether in the workplace or the schoolroom setting, what is emphasized and encouraged helps learners develop either a conceptual understanding or a highly routinized, inflexible set of responses. For example, a company that organizes work or a school that organizes learning as a set of segmented tasks will limit what its workers or its students learn. Companies organized for mass production will be more apt to structure learning as segmented tasks. Since most companies still organize work for mass production, America may face a Hobson's choice between two worlds, schools and the work-place, neither of which is well designed for powerful learning. Whether or not work-based training on a massive scale would be better than the current school-based system, the committee recognizes that the absence of significant work-based alternatives is unfortunate. Chapter 2 showed that traditional apprenticeships in the United States are not usually open to people in their late teens to early twenties and that innovations such as youth apprenticeships are still in the demonstration stage, reaching relatively few people. Cooperative education, where students alternate schooling and work, involved fewer than 3 percent of community college students

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy in the academic year 1989-1990 (U.S. General Accounting Office, 1991:19). The evident success of work-based approaches to qualifying training in Germany and elsewhere (see Chapter 4) as well as the unsatisfactory state of much of school-based training in the United States suggest that a more experimental strategy utilizing both approaches is called for. Access to Skills Improvement Training Unlike the other forms of training considered in this report, decisions about the extent of skills improvement training are left almost completely to the private sector; that is, to firms and their employees. Not surprisingly, then, access to skills improvement training for currently employed workers is extremely uneven. Some firms have embraced training as fundamental to their success and promote participation by a large proportion of their work forces. In some unionized companies, joint union-management programs mandate set-aside funding for training that involves substantial funds. More typical, though, appear to be the findings of the MIT Commission on Industrial Productivity (Dertouzos et al., 1989:86), which discovered in interviews with managers in many industrial settings that companies had low regard for training. The commission found "a systematic undervaluation in this country of how much difference it can make when people are well educated and when their skills are continuously developed and challenged." Chapter 2 indicated that expenditures by firms on employee training are sizable. The evidence indicates, however, that the receipt of training is quite uneven for different categories of workers and for employees of different firms. Women, minorities, young people, employees of smaller firms, lower-level workers, and workers with lower levels of formal education tend to receive disproportionately low amounts of skills improvement training (Kochan and Osterman, 1991:26-27; Lynch, 1992). Many firms, especially small and new companies, appear reluctant to invest in training for a variety of reasons (Stern and Ritzen, 1991; Lynch, 1994; National Research Council, 1993). Chapter 1 described some general features of the American labor market that help explain this reluctance. In addition, small and new companies confront special problems. They face capital constraints that prevent them from borrowing. Borrowing to invest in training—even training that can be shown to be a good investment—is also likely to be difficult because of the uncertainty associated with training investments and because of its intangibility, which means there can be no collateral for such loans. Small firms also face deterrents because of their small size. Training programs require certain fixed costs (determining what training is necessary, developing an appropriate curriculum or teaching method), regardless of how many individuals are to be trained; as a result, training at small firms will cost more per worker than similar training in large firms.

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy One partial solution has been for community colleges and other training providers to offer standardized courses—in, for example, literacy, English as a second language, the math necessary for statistical process control, or total quality management—in which small firms can enroll their employees without incurring high fixed costs. Here, though, another problem surfaces: inadequate information. Firms may not be aware of what is available or be in a position to judge its quality or its potential payoff to the company. In addition, shifting from one "equilibrium" to another can pose problems. To use the terms of America's Choice: High Skills or Low Wages! (Commission on the Skills of the American Workforce, 1990), a distressingly large number of firms appear to be in what can be characterized as a low-skills equilibrium: they are profitable and endure with a relatively undereducated labor force. In such workplaces, firm-based training is rare and largely confined to upper-level managers and professionals, and production methods have been adapted to the low skill level of the labor force. While there may be another, high-skills equilibrium—characterized by greater skills, a different organization of work stressing greater responsibilities for front-line workers, and more education and training—there may be no good way for firms to move from one equilibrium to the other on their own. Such a change would require too much lost time in reorganizing production and retraining workers; it would be impossible to obtain loans to cover the capital and training costs; there would be serious problems of timing, deciding in what order changes in technology, organization, and skill development should take place; and competitive pressures would make it difficult for managers to chart a long-run strategy. Access to Retraining Access to the two major federal dislocated worker retraining programs is currently limited, primarily by the need for dislocated workers to meet specific eligibility criteria and the size of appropriations to finance the training. The ability of individuals to participate in the programs may also be limited by amount of income support available and the timeliness of service offerings. Of the two federal programs, the Trade Adjustment Assistance (TAA) program has the more restrictive eligibility requirements. Workers are eligible only if a significant number of workers in a company have lost (or are threatened with the loss of) their jobs because imports of like or directly competitive products contributed importantly to a decrease in that company's sales or production. Other dislocated workers in the same community who may have lost jobs because of a local economic decline resulting from layoffs at the affected company are not eligible. Similarly, workers dislo

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy cated from separate companies supplying the affected company are not eligible. The other program, Economic Dislocation and Worker Adjustment Assistance (EDWAA)—Title III of Job Training Partnership Act (JTPA)—can provide services to all unemployed workers who lack the job search and basic and occupational skills necessary for reemployment. However, all workers losing jobs because of a plant closing or substantial layoff are eligible without having to demonstrate a lack of those skills. The emphasis in EDWAA is to provide a rapid response to the needs of the latter workers. As with TAA, workers who lose jobs because of the secondary effects of a plant closing or mass layoff do not have the same access to EDWAA services. Appropriations can also limit access to these programs. Although TAA job training is an entitlement, it is currently subject to an $80 million appropriation cap. The fiscal 1991 appropriation for EDWAA was $527 million. Lack of income may also inhibit dislocated workers from using job training assistance. TAA provides income support to eligible dislocated workers for 52 weeks after exhausting unemployment insurance. Although EDWAA also allows some income support, few participants receive any (U.S. General Accounting Office, 1992b). The GAO reports that, in the three states it examined, TAA participants were more likely to enter and remain in longer-term training than EDWAA participants. However, since the final 26 weeks of TAA income support is dependent on enrollment in training, it is not clear whether the income support enables the training or the training enables the income support. Meaningful access to training may also be dependent on the speed with which assistance is provided. The GAO report indicates that, while over 50 percent of EDWAA participants entered training within the first 15 weeks of unemployment, substantially more than 50 percent (97 percent in one state) of TAA participants did not get training services until they had been unemployed for at least 15 weeks. The GAO concludes that the necessity to certify the cause of dislocation is an important reason for the delay in the TAA program. Although the U.S. Department of Labor is required to complete action on a worker's petition for certification within 60 days, it also takes time to prepare a petition for filing and to inform workers of eligibility after approval. Moreover, the GAO (1992c) found that the 60-day requirement has forced the department to take shortcuts; as a result, 63 percent of the petitions that were investigated in 1990 and 1991 were flawed. Access to Second-Chance Training Like retraining programs, access to second-chance training programs is limited because benefits are targeted to specific populations and funding

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy has never been adequate to meet the needs of those who meet the eligibility requirements. This is particularly true of the three largest second-chance programs: Title II training for youth and adults in JTPA, the Job Opportunities and Basic Skills Act (JOBS), and adult education programs. JTPA funding has never been adequate to serve more than a small fraction of an eligible population that is defined quite broadly. The version of JTPA Title II-A in effect prior to the 1992 reauthorization established five broad criteria under which an individual could be considered "economically disadvantaged" and therefore eligible for JTPA services. Sandell and Rupp (1988:34-36) used these criteria and data from the Job Training Quarterly Survey for program years 1984 and 1985 along with information from the Current Population Survey to estimate JTPA participation rates. They found that an average of 39 million people met formal Title II-A criteria each year. Of this total, 31.7 million were 16 through 64 years old. Within this age group, only 12 percent were unemployed, both without jobs and actively seeking work; thus there were 3.9 million unemployed eligibles actively seeking work. These were the people most likely to be interested in JTPA services. In program year 1985, JTPA II-A served 498,800 unemployed people aged 16 to 64, or just 13 percent of the eligible population most likely to take advantage of the program. In that same year, JTPA II-A served 738,200 of the 31.7 million eligibles aged 16 to 64, a participation rate of just 2.33 percent. When Congress created the JOBS program in 1988, it enacted specific participation rate goals and timetables. Aid to Families with Dependent Children (AFDC) recipients between the ages of 15 and 59 are required to participate in the JOBS program, unless they have been exempted. (Recipients working at least 30 hours a week, attending high school, or caring for young children are among those exempted.) Congress set a schedule that states had to meet of gradually increasing, minimum participation rates. Under the schedule, states were required to serve 7 percent of the total number of nonexempted AFDC recipients within the state during fiscal 1991.4 Because of the stringent definition of participation used in JOBS, many more people are involved in the program than are actually counted as participants. The required participation rate rises to 11 percent in 1992 and 1993, 15 percent in 1994, and 20 percent in 1995. States not meeting these rates stand to lose a portion of their federal funds for JOBS programs. For fiscal 1991, all but one state met the 7 percent minimum each month (U.S. General Accounting Office, 1992e). Early indications are that many states will not have the money to meet the 20 percent participation requirement. A serious gap between the size of available programs and the need for services also exists in the field of adult basic skills training. Estimates of the number of adults in need of basic skills education vary, but the number is generally believed to be more than 20 million (Chisman, 1989; Grubb et

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy al., 1991). In findings incorporated into the National Literacy Act of 1991 (PL 102-73), Congress put the number at closer to 30 million. Whatever the exact number, it is clear that existing programs do not meet the potential demand for services. Chapter 2 showed that fewer than 4 million people annually have been served by the programs authorized by the Adult Education Act. In the National Literacy Act, Congress said that public and private literacy programs taken together serve approximately 19 percent of those in need of help. As with other programs, of course, not all of those officially considered to be in need of help may have an interest in receiving services. Nonetheless, one study from 1987 (Mikulecky, 1989:221-222) reported waiting lists for 30 percent of literacy programs nationwide and for 47 percent of programs in urban areas. Information About Training Options We began our discussion of access to and information about postsecondary training by acknowledging the benefits of multiple providers and multiple routes into occupations. A drawback of this approach, however, is that the variety of options available can be confusing, and information to help individuals make appropriate choices is often either unavailable or difficult to use. Various kinds of information would be useful to those contemplating training, such as an overview of the training opportunities available in a geographic area, the success of various training providers in preparing people for jobs, the training and certification requirements of specific occupations, the likely demand by employers for new workers with specific kinds of training, and the wages and career paths that characterize various fields of work. Though we did not as a committee undertake a careful study of the information issue, experience tells us that quite a bit of information exists, thanks in part to requirements of various federal programs, such as the Perkins Act and JTPA; a federal and state data system operated under the auspices of the Bureau of Labor Statistics; and the work of the federally supported National Occupational Information Coordinating Committee, its state counterparts, as well as state and private efforts. Nevertheless, the pieces are not complete in terms of the needs of potential trainees. They are not easily available to many who need them, especially adults. And there are important gaps, such as information on training results (see below) and the needs of the local labor market. Later in this chapter, we briefly discuss information needs from the viewpoint of employers, who also lack easy access to many things they would like to know about trainees and trainers. Chapter 6 returns to the subject of information at greater length.

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy RESULTS5 A key question about all training institutions and programs is, What difference do they make? Training is ultimately supposed to help people get better jobs at higher pay than they would without it. But does it help and how much? And what kind of training makes a difference, and for whom? It is frustrating that, for much of the American system of postsecondary training, the answer is all too often "we don't know." With the exception of the major second-chance programs, which have been increasingly subject to formal evaluations based on experimental designs, surprisingly little effort has been directed at assessing the results of postsecondary training (Mangum et al., 1990). Assessing the results of training is not easy. Even comparatively straight-forward information on outcomes (What proportion of trainees finish programs? How many get jobs? How much are they paid? Do the jobs reflect the skills individuals obtained in training?) is hard to find. The necessary record-keeping is often not done by training providers. Where public funds are not involved (as in firm-based skills improvement training), there is no requirement that any data be kept, and often they are not. Most publicly supported institutions and programs have some kind of record-keeping requirements. These naturally vary, given the decentralization of training among the many institutions that must be responsive to widely varying state and local rules as well as to differences in requirements among multiple federal programs. Going beyond outcomes to determine what impact (or "value added") training has is even more difficult. What we would really like to determine is not just what happens to trainees, but whether the training itself is responsible for whatever difference we can observe between those with training and those without. Evidence about impacts is often lacking, even in instances where something is known about training outcomes. In this case, the ability to determine what difference training makes depends on judgments about whether outcome data reveal useful information about the value added by training. The next section explores this issue in more detail. The absence of good information about results has several implications. As suggested above, it means that individuals seeking training have to select among available options without knowing much about the track record of different training routes or providers. The lack of reliable evidence regarding impacts also makes it impossible to judge the cost effectiveness of much of postsecondary training. This would be true even if sufficient information on costs were available, which is frequently not the case. Finally, this lack of information about results makes it difficult for policy makers to allocate public resources to programs that are most likely to help their intended audiences.

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy These comments should not be taken to mean that there is no evidence on the results of postsecondary training. Rather, the evidence is less complete than we would like and is often more indirect than direct. Nevertheless, using the framework described in the next section, we have evaluated what the available evidence can tell us about the benefits of training. A Framework for Analysis: Value Added Versus Outcomes Our interest in assessing the success of existing training efforts centers on their impact or value added, i.e., the change in relevant outcomes due solely to the training experience. In other words, value added is the increment in the proportion of people getting a job as a result of training as opposed to the percentage of program trainees getting a job. That percentage may or may not be the result of training. Value added is the increment in the wages of trainees—the gain in wage due to the skills they learned in training relative to the gain comparable individuals would make without the program—not their absolute wage level or their wage compared with some crude alternative. This definition distinguishes between the value that training adds, on the one hand, and simple outcomes as measured only by the behavior of people who were in a training program, on the other. The best way to determine the value added of training is through a random assignment, controlled experiment (Betsey et al., 1985; Burtless and Orr, 1986; Job Training Longitudinal Survey Research Advisory Panel, 1985; Barnow, 1987). In such an experiment, applicants for training are randomly assigned to a treatment group that is allowed access to training or to a control group that is excluded from training. Random assignment ensures that the treatment and control groups do not differ in any systematic way except in their access to the program. Therefore, any subsequent differences in outcomes between the two groups can be confidently attributed to the training.6 Increasing evidence on the feasibility and credibility of this approach (Gueron and Pauly, 1991; Hollister et al., 1984; Wiseman, 1991) has prompted the use of random assignment in many government-and foundation-sponsored evaluation efforts, particularly for JTPA and welfare-related training programs (see, e.g., Gueron and Paul, 1991; Bloom et al., 1993; Manpower Demonstration Research Corporation, 1980; Bell et al., 1987; and Puma et al., 1990). Where they exist, we rely on random assignment experiments as the best indicator of program effectiveness.7 But it is not always possible to do a random assignment experiment. For instance, it is impossible to forbid some students from going to community colleges. As a result, almost all research on the results of school-based postsecondary training has been limited to nonexperimental studies in which statistical methods are used to attempt to correct for pre-existing differences between training participants and a nonrandom comparison group. Evi

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy employment and earnings of participants and a reduction in their welfare dependency. The U.S. Department of Labor, in consultation with state and local program officials, developed a series of separate measures and standards for adults and youth participating in the program by which the activities of each local service delivery area could be judged. Governors decided which of the measures and standards to adopt. The department also developed adjustment models that governors could use to modify expected performance to take account of differences in participant characteristics and local labor-market conditions affecting individual delivery areas. While the existence of performance measures and standards added to the credibility of JTPA (no small feat, given problems with the reputation of CETA), implementing them has also illustrated the difficulty of designing standards that in fact lead to desirable behavior. Cost and outcome standards in JTPA, for example, appeared to lead to "creaming" (selecting recipients who were the most job-ready and the least expensive to train); cost standards were eventually dropped.12 Over time, standards were also modified to replace measures of the status of participants at termination by post-program measures, because the former appeared to encourage short-term approaches such as job-search activities rather than longer-term education and training. New measures were added, and the adjustment models were refined by adding variables to make them increasingly sensitive to differences in participant characteristics. Despite the complexities suggested by the JTPA experience, the performance-management approach to ensuring quality is spreading to other training programs. The Family Support Act of 1988, which created the JOBS program, requires the Secretary of Health and Human Services to report to Congress on performance standards in JOBS by October 1, 1993. The 1990 reauthorization of the Perkins Act requires each state board receiving assistance to develop and put into effect a statewide system of core performance measures and standards. Educational institutions participating in student assistance programs are required to report certain outcome data for the first time under the Student Right-to-Know Act of 1990. What might be called rudimentary performance standards (e.g., provisions denying eligibility to participate in student loan programs to institutions with default rates above a specified level) have been adopted in recent years as part of federal efforts to crack down on default problems in student-assistance programs. In addition to these congressionally mandated steps, measurements and standards are being incorporated into other federal programs by administrative action. Although the popularity of the performance-management approach to quality assurance is growing, we found no evidence from the evaluation research on JTPA and the welfare-to-work experiments of the 1980s that programs with performance standards produced greater impacts than programs without them. In fact, performance management is not oriented to

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy ward impact or value-added. As discussed earlier, outcomes and impacts are not necessarily correlated with one another. Barnow (1992:279) explains the reason for performance management's emphasis on outcomes: Performance management should also be distinguished from evaluation. Programs can be evaluated in terms of their impacts on an occasional or one-shot basis, but performance management is an ongoing feature that continuously provides feedback to the program managers and the agencies and organizations responsible for monitoring the programs. Thus, while evaluations often include one or more years of post-program follow-up, performance management systems must rely on shorter post-program periods to provide reasonably quick feedback. While evaluations answer the question "What is the impact of the program?" performance management systems generally seek answers to simpler outcome and process questions that are associated with the goals of the program; the issue is more one of accountability than impact. Finally, evaluations of human service programs are generally costly and require the use of comparison or control groups to identify what would have occurred in the absence of the program. Performance management systems are generally less intrusive, but they then must sacrifice including impact measures. The use of performance-management system is growing, even though very little is known about how institutions respond to different approaches and the varying incentives and disincentives they create. The objectives cited above by Barnow are praiseworthy. Yet the fact that program outcomes and program impacts are not always highly correlated creates a profoundly important dilemma: Does the spreading influence of performance management systems actually lead to more effective programs? This issue is crucial; Chapter 6 analyzes the complexities involved in deciding how best to develop mechanisms for ensuring the quality of postsecondary training. CONNECTIONS WITH EMPLOYERS A well-functioning training system needs to be closely connected to employers. In general, employer connections with training, except for the training that they themselves sponsor, tend to be weak in the United States. There are different kinds of possible connections, though, and it is useful to distinguish among them in evaluating existing arrangements. A key role for employers is as customers for the "products" of the training system. The evidence is mixed as to how much employers consider postsecondary training when they make hiring decisions. The generally positive economic returns to qualifying training, at least for those who get degrees or certificates, suggest that employers do place value on what schools teach. On the other hand, research by Grubb et al. (1992) indicates that, in what they call "sub-baccalaureate" labor markets, employers often rely more

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy on experience than schooling in making hiring decisions. This tendency is encouraged by the relative absence of information at the local labor-market level about who is being trained in what kinds of programs. (As Grubb et al. note, sub-baccalaureate labor markets are almost entirely local.) Further complicating the picture is the general lack of information about the content of training programs and the hiring requirements that employers must follow. Nationally recognized credentials or occupational-skills standards would provide information about program content, but these exist in only a few fields. Licensing requirements similarly serve to connect employers and providers, but they exist only in health and a few other occupations in the sub-baccalaureate labor market. This mixed picture about the value employers place on training in their hiring processes has been identified as one reason why many students are not highly motivated to do well in school. Bishop (1990) has discussed the effects of the failure of American employers to utilize transcripts and other indicators of school success when making hiring decisions. While his attention has been focused on the high school, there is no evidence that employers place more emphasis on performance in postsecondary schools. As we shall see in Chapter 4, this contrasts strongly with the message German and Japanese firms convey to young people about the importance of taking their schoolwork seriously. Participants in second-chance training have been stigmatized in the eyes of employers by their enrollment in programs labeled as serving the disadvantaged. Osterman (1988, 1990) has emphasized the stigmatization problem in job training. Eisner (1989) cites evidence that workers labeled disadvantaged have been less likely to be hired by employers in other public employment programs, such as the Targeted Job Tax Credit. Similarly, use of the U.S. Employment Service by employers as a central labor exchange fell dramatically after the mid-1960s, when the service's emphasis was altered to focus on the hard to employ (Bendick, 1989). Employers can be connected to training in various ways besides hiring. They can provide work-based training and instructors for school-based training; make state-of-the-art equipment available to students; contribute to training curricula and pedagogy by helping educators understand industry and occupational skill demands; serve on advisory and governing boards (including boards of directors at large proprietary schools with corporate structures); and exert quality control over training by helping to define industry skill standards and rewarding skill certificates in their hiring processes. Employers do all of these things, but there is very limited data about how often or how well. For example, although there is no national system of defining and certifying boards for occupational and industry skills, some, but not many, industries have established standards and certificates. Similarly, there are widely varying reports about how extensively employers influence the

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy curricula of community colleges. Grubb et al. (1992:41-42) say that employer advisory committees were not very active in this regard in the four labor markets they studied. But the U.S. Office of Technology Assessment (1990:148) observed that community colleges in North Carolina match their curricula to the changing needs of employers through a process called DACUM (developing a curriculum). The process involves employees working with a college coordinator to identify lists of competencies needed in a specific occupation. Members of the committee observed the same process in use in community colleges in South Carolina. We should note that employer contributions to school-based training curricula need to be viewed cautiously. Ironically, with some notable exceptions, employers are often poor sources of information about the skill demands of their industry or occupations. Economic studies of restructuring companies find substantial variation within the same company in response to questions about what generic skills the company needs. The variation is related to the respondent's position within the company (Thomas R. Bailey, Columbia University, personal communication). Studies of on-the-job training find that employers often do not recognize the cognitive complexity of their own jobs, especially when these jobs are perceived as "low skill." Employers often do not understand the skill implications of restructuring in their own workplaces. For example, they may introduce just-in-time inventory arrangements, but they do not appreciate how such changes increase the skill requirements of such jobs as stockroom clerk (Scribner and Sachs, 1990). Despite these cautions, we agree with the general view that employer involvement in the planning and design of occupation-oriented programs is important and below optimal levels in the United States. This involvement is needed to ensure that the skills developed will be those needed in the workplace and that the available training programs match the likely needs of the local labor market. We also note strikingly positive findings in the welfare-recipient training area in two studies of sites that were particularly attentive to working closely with private-sector employers: Riverside County in the GAIN evaluation (Friedlander et al., 1993) and the Center for Employment Training in the JOBSTART and Minority Female Single Parent studies (Cave et al., 1993; Burghardt et al., 1992). OVERLAP, DUPLICATION, COORDINATION, AND ARTICULATION13 The fragmented, decentralized nature of training institutions and programs has led to somewhat of a preoccupation, especially among federal policy makers, with overlap, duplication, and the need for better coordination among publicly supported programs. The observation is frequently

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy made by both scholarly and policy-oriented observers that the United States does not have a true training system, but a loose collection of relatively autonomous parts operating within a variety of local delivery structures. We found that fragmentation is not as bad as it appears on the surface. At the same time, we found a number of reasons to believe that improved training requires a much more systematic approach than currently exists. Grubb and McDonnell (1991) have discovered that, despite fragmentation and program proliferation, existing institutions and programs in fact often behave in a variety of system-like ways. But there are many imperfections in these informal systems that develop in ad hoc ways in local communities. Contrary to conventional wisdom, at the local level there is considerable coordination among programs. In most communities, administrators of every program are familiar with one another, and extensive referral and contracting among programs takes place. In the most typical pattern—what Grubb and McDonnell call the standard model—secondary and postsecondary vocational programs are linked by articulation agreements and so-called two plus two plans linking the last 2 years of secondary and the first 2 years of postsecondary; JTPA and welfare-to-work programs subcontract with community colleges to provide some (though not all) of their classroom-based training and remediation, with adult schools usually providing the lion's share of remediation; and community colleges provide customized training with their own resources as well as funds from state economic development efforts and payments from sponsoring firms. There are, to be sure, variations among communities. In another pattern, educational institutions are linked through articulation agreements while JTPA and welfare-related programs collaborate, but the educational "subsystem" and the JTPA and JOBS programs do not interact much. Still another pattern emerges in a few communities, whereby a local community college dominates all education and training services. And, of course, there are some communities where programs operate virtually alone, and coordination is almost absent, but these appear to be quite rare. Such coordination among programs, which can be described as collaborative service delivery, is typically the result of local initiatives rather than that of federal requirements or state policies for collaborative planning. The motives for local collaboration vary. These include cost-shifting, where programs with limited funding (especially JTPA and welfare programs) refer clients and thereby shift costs to community colleges and adult schools with enrollment-based, open-ended funding or student aid funds; the general dislike of competition and a pervasive feeling that cooperation will increase total resources; the search by most programs for distinct niches where they need not compete with others; and local brokers—including

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy coordination councils, chambers of commerce, other business groups, and sometimes Private Industry Councils—that serve to link different programs. Where coordination fails to take place, the barriers include differences in the choice of services offered, particularly where JTPA and welfare programs have concentrated on job-search assistance and on-the-job training rather than the more intensive programs typical in community college; dissatisfaction with particular providers, especially in communities where the community college is viewed as too academic and rigid in offerings and scheduling; JTPA performance standards, which have discouraged some educational institutions from competing for contracts;14 and local politics, which dictate the allocation of JTPA resources in some communities and create a cleavage between educational institutions, on the one hand, and JTPA and welfare-related programs, on the other. These barriers are not the kind that can be overcome by federal or state coordination mandates. They are more fundamental, embedded in the basic purposes and structure of different programs. In another sense, communities have systems of education and training: a wide range of services are available. Virtually every community, at least in urban and suburban areas, has a continuum of remediation programs leading to the GED (within adult education) or college-level courses (within community colleges); a variety of job-specific training ranging from short-term, entry-level programs in area vocational schools and community-based organizations to 2-year associate degree programs in technical and other well-paid fields; and ancillary services, like job-search assistance and placement services. In theory, individuals can enter various points in this system, make their way through a variety of programs (including remedial programs as necessary), and emerge with various kinds of occupational credentials or with the academic requirements to transfer to 4-year colleges and the rest of the educational system. Finally, a system of specialization often operates at the local level that avoids the duplication of services and the resulting waste and inefficiency so worrisome to federal policy makers. Grubb and McDonnell (1991) found, in fact, that there are remarkably few cases of outright duplication in vocational education and job training. One reason is that programs vary in their services, with vocational programs concentrating on longer-term certificate and associate programs, while JTPA and welfare programs emphasize short-term training, on-the-job training, and job-search assistance. Institutions and programs also specialize in the individuals they serve: community colleges and technical institutes serve those students not attending 4-year colleges, as well as those in search of retraining, and experimenters trying to decide about their futures (Manski, 1989); JTPA serves a group with less formal education and labor-market experience; JOBS provides certain services for welfare clients; and adult education serves those in need of basic

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy skills remediation and English as a second language. In addition, an overall shortage of resources means that there is generally greater demand than existing programs can supply. Where apparent overlap and duplication exist, especially in federal programs, it tends to occur because programs put different weight on seemingly similar goals. Given these differences in emphasis, duplication in resulting activities or conflicts between like-sounding offerings are not surprising and indeed may be legitimate. On the other hand, there is reason to be concerned about problems of coordination and the lack of a systematic approach. One issue is the proliferation of federal programs that state and local training providers face. Table 2.1 details several dozen postsecondary training programs; these are only a fraction of the 125 employment and training programs identified by the U.S. General Accounting Office (1992d). While only a few of these programs are substantial, the existence of so many multiplies the administrative and coordinating tasks facing state and local program officials. GAO found that of 125 employment and training programs, 4 had annual funding of over $1 billion each, but 72 had funding of less than $50 million. Even in the larger programs, a patchwork of incompatible federal requirements inhibits state and local efforts to plan and deliver services in a coordinated fashion. A survey of state and local human resource administrators and policy makers by the National Governors' Association (1991) identified a variety of barriers to coordination created by federal laws and regulations. Major impediments included the lack of common or compatible definitions; procedures for determining eligibility; and fiscal, administrative, and planning requirements. Despite the large amount of coordination and referral that in fact occurs at the local level, there is also reason to be concerned about the near absence of mechanisms following individuals through the postsecondary education and training system, helping them make transitions among programs, providing them assistance if they falter, providing them information about the alternatives available, or helping them gain portable credentials that demonstrate competencies valued by employers. As a result, referral among programs—e.g., from JTPA to adult education, or from JOBS to the local community college—is likely to result in individuals becoming lost. This happens commonly even in welfare-to-work programs, where case workers are responsible for tracking individuals (Riccio et al., 1989). In a few communities, the dominance of the community college (or, less often, a particular adult school) has consolidated all services in one institution, facilitating tracking and referral among services; and some states, notably Wisconsin, have experimented with one-stop education and training centers. Such efforts have, until recently, been rare, however. In most communities, what could be a well-articulated system with a continuum of remedial and

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy job-specific education has proved to be a patchwork of disconnected pieces for those who want training. Years of mandating various coordination requirements among training and employment programs and creating ad hoc working arrangements among federal departments have not adequately alleviated these problems. Marion Pines, who headed a JTPA advisory committee to the U.S. Department of Labor that stressed the need for more coordinated human investment approaches and a more rational human resource delivery system in the United States, recently observed (Pines, 1992:30): Many states and local areas are working assiduously to make their diverse systems ''user-friendly'' by developing one-stop intake, assessment, and case-managed resource brokering to individuals and families. But they are doing it with much pain, and a degree of risk for audit exceptions. The variety of [federal] legislative initiatives that have been enacted have created the kinds of barriers to service integration that many prudent and conscientious people fear to buck. We find the continuing federal barriers to systemic planning and coordination particularly disturbing, because we, like Pines, are hopeful about new state efforts to restructure their approaches to training. A spate of recent developments suggests that a number of states are moving aggressively to weave the fragments of their own and federal training programs into integrated work force development systems. Governors and business leaders closely involved with such endeavors recognize the importance of decentralizing decision making to the substate level so that solutions can be tailored to the needs of employers and individuals within local labor markets. Nevertheless, they have also recognized the need to restructure state approaches to training, to ensure more comprehensive and coordinated planning, and to "shift the state role from a regulatory one to one of policy guidance and capacity building" (National Governors' Association, 1992:10-11). They have done so in various ways. Even before the 1992 JTPA amendments granted governors the discretion to establish human resource investment councils, which consolidated numerous advisory councils required by various federal human resource programs, at least 10 states had established "super councils" of one sort or another (National Governors' Association, n.d.). Thanks to the JTPA amendments, this approach seems certain to spread. Some states have rationalized responsibilities for training programs within and among state agencies. New Jersey, for example, has consolidated 64 programs previously operated by 6 departments into 15 program areas in 3 departments. At least one state, Indiana, created a new, consolidated, executive branch department. The Indiana Department of Workforce Development brings under one roof the Commission on Vocational and Technical

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy Education, the Indiana Department of Employment and Training Services, and the Office of Workforce Literacy. Many states are also strengthening their information and accountability systems and developing new ways of assessing the outcomes of training programs (see National Alliance of Business, 1992, and National Governors' Association, 1992). These initiatives typically involve strong leadership from governors and enlist the business community as important partners. They go beyond the requirements for coordinating and advisory councils that are often found in federal training programs. They appear to be vigorous, creative, and promising approaches to the problems that we have identified in this study. We find these new state developments extremely encouraging because it is clear that states occupy the key position in giving public coherence and direction to postsecondary training. Most of the public funding for training institutions is theirs or flows through them. They are in a far better position than the federal government to oversee the development of arrangements that will result in systematically available training opportunities that are attuned to the specific and varying needs of local labor markets. The federal government, as an important player in the postsecondary training arena, must find ways to support these efforts and not inhibit their development. SUMMARY This chapter is long and complicated, but its major conclusions can be briefly stated. The United States does not have a true postsecondary training system. We have a variety of providers and programs that supply very good training to some citizens, but less effective or no training to many others. Qualifying training is readily accessible to many people, though the range of options is undoubtedly confusing to some. Qualifying training relies on schools almost exclusively, and many individuals who enroll do not complete any formal credential, though it is not clear how much the failure to stay enrolled long enough to earn a postsecondary training credential hurts individuals in the labor market. Opportunities to use workplaces as learning sites are inadequately developed. For different reasons, skills improvement training, retraining, and second-chance training are all less widely available. Evidence about the results of training is less thorough than we would like, but on balance the results are positive, if sometimes modest. Some kinds of training, particularly for disadvantaged youth, do not appear to be working. The quality of training is mixed, and the institutional structures for ensuring quality are underdeveloped. Linkages to employers need improvement. At the crucial level of local labor markets, programs and providers work together in more harmony than the fragmented national picture would suggest, but the pieces do not add up to anything that can be described as a system. The federal government, with its prolifera

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy tion of programs and lack of unifying approach, bears part of the blame for this situation. NOTES 1.   While it would be preferable to examine the access question using longitudinal studies that include nonenrolled as well as enrolled individuals, existing longitudinal studies are dated and involve much smaller samples than NPSAS. NPSAS results, while limited by their cross-sectional nature, are recent and revealing. 2.   He concluded that "the financial aid system works relatively well at removing the direct economic barriers to postsecondary enrollment" and observed that the vocation-oriented schools were more accessible "not only because of less restrictive admissions practices and lower direct costs [than 4-year schools], but also because they offer short-term programs and flexible scheduling (for example, part-time enrollment) that reduce the opportunity costs that students must incur" (Tuma, 1992:iii-iv). 3.   Lynch (1993) has summarized several papers from her forthcoming volume on training and the private sector that use data from the United States, Britain, the Netherlands, and Norway to show that firm-based training results in greater economic payoffs to individuals than does school-based training (see Blanchflower and Lynch, 1994; Elias et al., 1994; and Groot et al., 1994). The committee was not able to examine these studies, but we note that the U.S. data on school-based training apparently are based on secondary-level vocational education and exclude training received in postsecondary schools such as community colleges and proprietary institutions. 4.   According to guidelines set by the U.S. Department of Health and Human Services, countable participants as a group must be scheduled for an average of 20 hours a week of JOBS services and must attend most of those hours. States may serve exempted welfare recipients who desire services and may include them in order to meet the minimum participation rate. 5.   Parts of this section are taken from a background paper prepared for the committee by David Stern. 6.   The major shortcoming of nonrandomly controlled research designs stems from what evaluation experts call the problem of selection or selectivity bias. There may be unmeasured differences (in motivation levels, for example, or in job-relevant capabilities not measured by standardized admission or ability tests) among individuals choosing different training paths or between training participants and nonparticipants. Unmeasured differences may affect subsequent success on the job, thus confounding efforts to determine how much impact training itself has on job success. The evaluation problem created by the selection process is well-documented for welfare-to-work and mainstream employment and training programs (Gueron and Pauly, 1991; Ashenfelter, 1987; Burtless and Orr, 1986; Betsey et al., 1985; Lalonde and Maynard, 1987; Job Training Longitudinal Survey Research Advisory Panel, 1985). Selection bias also occurs among those pursuing various postsecondary schooling options (Grubb, 1990; Hollenbeck, 1992; Kane and Rouse, 1993), though these authors find that selection effects do not affect the estimates of the returns to schooling. From a research standpoint, however, it is clear that the study of selection effects in education has not advanced very far, and that further investigation into selection biases in nonexperimental studies on returns to school-based training is needed. 7.   For all their advantages, however, random assignment experiments share some of the shortcomings of nonexperimental research. For example, because the experiment does not have laboratory control conditions, results from one study may not be valid for other circumstances (a training program may work better when the job market is tight but not when it is very soft); few studies follow people long enough to determine if there are permanent changes

OCR for page 64
Preparing for the Workplace: Charting a Course for Federal Postsecondary Training Policy     in circumstances; and many follow trainees and controls for just 1 or 2 years, requiring extrapolation of end-period results into the future. Manski and Garfinkel (1992) and Heckman (1992) question the emphasis that has been given to random assignment field experiments, and we will see that they may not be practical in some circumstances and may not address all of the relevant policy questions. Still, where it exists, we believe that evidence from the experimental approach is the best available. 8.   Cross-sectional income data are found in the Current Population Surveys and the Survey of Income and Program Participation, both conducted by the Bureau of the Census. The U.S. Department of Education conducted two longitudinal research projects, the National Longitudinal Study of the high school class of 1972 and High School and Beyond, studying the high school classes of 1980 and 1982. 9.   For example, Adelman (1992) presents evidence from the National Longitudinal Study of 1972 on the infrequency of degree completion. He then concludes that students "seemed to make of the community college what they wanted to make of it," but without any evidence from the data that those who leave have in fact achieved their purposes. 10.   The research findings cited in this and subsequent paragraphs relate to average earnings effects. Katz and Murphy (1992) have pointed out that earnings inequality within groups defined by education, age/experience, and gender is substantial and has been steadily rising, reaching a level in 1987 that was 30 percent higher than the level in 1970. This suggests that the returns to education vary significantly among students. 11.   Data from the U.S. Department of Labor, unpublished 5-year analysis of JTPA Title III program performance, March 23, 1993. 12.   Barnow (1992:297) indicates that performance standards likely led to creaming in JTPA but acknowledges that it is difficult to separate out the influence of performance standards from other factors that could also have encouraged the selection of easy-to-place participants. These include the increased role of the private sector in overseeing local implementation and limitations on supporting services and stipends. 13.   Parts of this section are taken from a background paper prepared for the committee by Richard F. Elmore. 14.   However, the discouraging effects of performance standards are clearly much weaker than often claimed, since the most common form of cooperation between vocational education and JTPA involves JTPA subcontracts with community colleges and technical institutes.