Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 281
~ Central Recommendations In this chapter the committee presents a core set of recommendations that emerge from the earlier discussions throughout the report. Chapter 14 is not a comprehensive list of all recommendations in Chapters 4 through 13, but is limited to those that bear most directly on the design and implementation of the VG-GATB Referral System. Although a thorough understanding of the committee's statements here depends on the discussions in Chapters 4 through 13, policy makers will find here a summation of the most essential points. The findings and conclusions on which these recommendations are based appear at the ends of the appropriate chapters, as do further recommendations for research. OPERATIONAL USE OF THE VG-GATB REFERRAL SYSTEM A thorough evaluation of the General Aptitude Test Battery (GATB) leads us to conclude that the test has modest levels of validity for predicting job performance, and that these predictive validities are strong enough to produce some enhancement of worker performance for indi- vidual employers who use test information in selecting employees. We accept, as a general approach, the theory of validity generalization, whereby validities estimated for some jobs may reasonably be expected to hold for similar jobs not studied. We conclude that the range of GATE validities found in the 500 jobs studied would roughly generalize to the kinds of jobs typically handled by the U.S. Employment Service (USES). At the same time, we note that the GATE was not designed to function as 281

OCR for page 281
282 CONCLUSIONS AND RECOMMENDATIONS the centerpiece of a widely used referral system, and it is not currently supported by a research and development program that would justify its use in this way. 1. On the basis of thesefindings, the committee recommends that any expansion of the VG-GATB Referral System be accompanied by a vigorous program of research and development. Two inadequacies in the testing program must be corrected: a. Test Security: It is essential that measures be taken to provide for test security to ensure fairness to examiners. Most important is the regular development of alternate forms of the test and frequent replacement of old forms. (As a point of comparison, the Department of Defense develops three new forms of the Armed Services Vocational Aptitude Battery every four years.) In addition, USES must produce, and the states must enforce, clearly specified security procedures of the kind used to maintain the confidentiality of other large-scale test batteries. b. Test Speededness: A research and development project should be put in place to reduce the speededness of the GATB. A highly speeded test, one that no one can hope to complete, is vulnerable to distortion from coaching. For example, scores can be im- proved by teaching test takers to fill in all remaining blanks in the last minute of the test period. In addition, preliminary evidence suggests the possibility of differential impact by race of highly speeded tests. If this characteristic of the GATB is not altered, the test will not retain its validity when given a gatekeeping function that is widely recognized. 2. We recommend that no job seeker be obliged to take the GATB; every local office that uses VG-GATB referral should maintain an alternative referral path for those who choose not to take the test. There are dangers in instituting a single, uniform testing system throughout the Public Employment Service-and such dangers would exist even if the test instrument were far superior to anything available today. Tests are not only fallible, but they also give a narrow reading on human capabilities. To permit only one route into the work force would result in its impoverishment. Giving the sort of primacy envisioned to a single instrument would also unnecessarily burden certain job seekers, for some people are simply not well served by cognitive tests because of test anxiety, certain kinds of handicapping conditions, or language problems. Overreliance on testing could also tend to create an underclass of low-scoring registrants who never get referred to jobs. In addition, there are large classes of jobs for which the test is not needed for various reasons. For example, employers with openings for

OCR for page 281
CENTR`4L COMMENDATIONS 283 unskilled labor will usually consider all applicants; testing the applicants would be an unnecessary burden. Nor is it appropriate to encourage employers to use the GATB if able workers can be better identified by other, more job-specific methods, as would be the case with jobs such as musician or master electrician that require special skills or training. 3. Because tests provide only partial information about future job performance, we recommend that Job Service offices that adopt the VG-GATB Referral System continue to use multiple criteria in choosing which applicants to refer. Employment Service personnel should help employers who elect to use the VG-GATB system decide how test scores and other job-related information, such as experience, skills, or education, should be used in establishing a referral decision rule. The employer should also be encour- aged to consider able applicants who have not taken the test. The best service to employer and job seeker alike will be obtained by using multiple criteria, tailored as much as possible to each job situation. REFERRAL METHODS Our examination of USES validity studies confirms that there are sizable differences in mean scores on the GATB, ranging from one-half to one standard deviation, between blacks or Hispanics and the majority group. As a consequence, referrals made in the order of unmodified GATB scores would adversely affect the employment chances of minority job seekers. Furthermore, because the GATB has only modest predictive validity correlations (our conservative estimate is that they average .3, corrected), low-scoring applicants who could have been successful per- formers will be screened out (see Chapter 131. Because greater propor- tions of minority applicants fall in the low-scoring group, exclusive use of unadjusted GATB scores would result in referring able minority workers in much lower proportions than majority workers at the same level of job performance. 4. The committee recommends the continued use of score adjust- ments for black and Hispanic applicants in choosing which applicants to refer to an employer, because the effects of imperfect prediction fall more heavily on minority applicants as a group due to their lower mean test scores. We endorse the adoption of score adjustments that give approx- imately equal chances of referral to able minority applicants and able majority applicants: for example, within-group percentile scores, perfor- mance-based scores, or other adjustments. Given the modest current levels of GATB validities, such adjustments are necessary to ensure that able black and Hispanic workers will not

OCR for page 281
284 CONCLUSIONS ED RECOMMENDATIONS experience higher rejection rates than workers of the same level of performance in the majority group. Referral by within-group percentile scores is one elective way to balance the dual goals of productivity and racial equity. 5. If the within-group score adjustment strategy is chosen, a. we recommend that USES undertake research to develop more adequate norming tables. The data on Native Americans are particularly weak, but all the norming samples are idiosyncratic convenience samples. As a conse- quence, there is reason to doubt that the particular constant factors added to minority scores are the most appropriate ones. b. An attempt should be made to develop norms for homogeneous groups of jobs, at the least by job family, belt if possible by more cohesive clusters of jobs in Job Families I V and V. To correctly compute within-group percentiles, USES must esti- mate the average difference between the majority group scores and minority group scores in applicants for homogeneous groups of jobs. 6. We also recommend that USES study the feasibility of what we call a Combined Rules Referral Plan, under which the referral group is composed of all those who would have been referred either by the total-group or by the within-group ranking method. This method of referral is attractive because it does not curtail the chances of any majority group applicants in order to increase the opportunities of minority applicants. In addition, when combined with a complementary score reporting system, it gives employers a choice. Depending on their affirmative action posture, they can choose to ignore race entirely and select solely on the basis of predicted performance, or to select from an enriched pool that includes the highest-scoring minority- group members available. The method does require referring a somewhat larger number of applicants for each job order, and so increases the selection task for the employer. SCORE REPORTING The uppermost concern in reporting GATE scores should be to provide the most accurate and informative estimate of future job performance possible. Used in isolation, percentile scores (whether computed for the whole population or by specified subgroups) can be misleading when test validities are modest, because they appear to say much more about expected job performance than is warranted. The sole use of total-group percentile scores would compound the problem vis-a-vis minority groups by encouraging the incorrect inference that differences in job performance

OCR for page 281
CENTRAL RECOMMENDATIONS 285 between high and low scorers will be as great as the differences in test scores. Reporting only scores that have been adjusted to reflect standing within a racial or ethnic group, however, is also deceptive the more so as test validity increases since it masks group differences in predicted performance. 7. The committee recommends that two scores be reported to employ- ers and applicants: a. a within-group percentile score with the corresponding norm group identified and b. an expectancy score (derived from the total-group percentile scored equal to the probability that an applicant's job perfor- mance will be better than average. This combination of scores provides information on how well an applicant performed on the test with reference to others of the same subpopulation while also indicating the probability of above-average performance irrespective of group. In other words, it provides both a within-group and a total group comparison. In addition, the expectancy score is more informative than other scoring methods because it reflects the predictive accuracy of the test as well as the performance of the applicant. With a test of modest validities like the GATE, this scoring method helps to prevent the incorrect inference that large differences in test scores reflect similarly large differences in performance on the job; employers and applicants are informed that even low scorers on the test have a reasonable chance of being above-average workers. An example of our recommended scoring protocol, using a test validity of .3, follows. (For some classes of jobs different validities might be appropriate.) Within-Group Total-Group Expectancy Percentile Computed Score: Chance of Being Name for "Black" Group* Better-Than-Average Worker Grace Birley 16 25 James Jones 50 40 Shelton Pike 84 50 Within-Group Total-Group Expectancy Percentile Computed Score: Chance of Being Name for "Other" Group* Better-Than-Average Worker Nancy Rathouse 16 40 William Cole 50 50 Theresa Brewer 84 60

OCR for page 281
286 CONCLUSIONS ED RECOMMENDATIONS Within-Group Total-Group Expectancy Percentile Computed Score: Chance of Being Name for "Hispanic" Groups Better-Than-Average Worker Juan Gomez 16 33 Chester Alverez 50 44 Olivia Gerber 84 56 *GATE subpopulation norms exist for `'black," "Hispanic," and "other" groups. PROMOTION OF THE VG-GATB REFERRAL PROGRAM USES technical reports make overly optimistic projections of the effects of VG-GATB referral. Perhaps as a consequence, much of the promotional literature that we have seen overstates the psychometric quality and predictive power of the GATB, underestimates the vulnera- bility of the referral system to legal challenge, and exaggerates the economic impact of preemployment testing. 8. Given the modest validities of the GATB for the 500 jobs actually - studied; given our incomplete knowledge about the relationship between this sample and the remaining I I ,500 jobs in the U.S. economy, given the Department of Justice challenge to the legality of within-group scoring and the larger philosophical debates about race-conscious mechanisms ancl the known problems of using a test with severe adverse impact, given the primitive state of knowleclge about the relationship of individual performance and productivity of the firm, we recommend that the claims for the testing program be tempered and that employers as well as job seekers be given a balanced view of the strengths and weaknesses of the GATB and its likely contribution in matching people to jobs. 9. Given the primitive state of knowledge about the aggregate eco- nomic elects of better personnel selection, we recommend that Employ- ment Service officials refrain from making any dollar estimates of the gains that would result from test-based" selection. 10. The Employment Service should make clear to employers using the VG-GATB Referral System that responsibility for the relevance of selec- tion criteria and the effects of selection on the composition of their work force lies directly with the employer. Use of tests approved by the U.S. Employment Service does not alter this allocation of responsibility under federal civil rights law. We have seen Employment Service literature that could be understood to say that use of the VG-GATB protects employers from legal challenge of their employee selection procedures. We have heard from a number of employers that they believed this to be the case because the VG-GATB

OCR for page 281
CENTRAL COMMENDATIONS 287 has been promoted by the federal government. USES should take pains to correct this misapprehension and to inform employers routinely that they are not relieved of their responsibility for maintaining nondiscriminatory selection procedures by using the U.S. Employment Service. EFFECTS OF THE VG-GATB SYSTEM There is too little evidence based on controlled, rigorous studies of the effects of using the VG-GATB Referral System for the committee to be able to assure policy makers at the Department of Labor that anticipated improvements have indeed occurred. This is not to say that they have not occurred. The evidence simply does not exist to establish the case scientifically. For the moment, policy decisions about the future of the VG-GATB Referral System will have to be made on the basis of more impressionistic and experiential information. 11. If USES decicles to continue the VG-GATB Referral System, it should undertake a series of carefully rlesigned studies to establish more solidly the e;/~iciencies that are believed to result. 12. This research will need to be a cooperative effort, involving federal and State Employment Service personnel and employers. USES should encourage state Employment Security Agencies that deal with large employers (e.g., Michigan) and states that have fully articulated VG systems in place (e.g., Virginia, Utah, Oklahoma) to take a leading role in conducting studies to demonstrate the efficacy of the VG-GATB Referral System. 13. We also recommend that the employer community, as a potentially major beneficiary of an improved referral system, take an active part in the effort to evaluate the VG-GATB Referral System. The Employers' National Job Service Committee can help to identify appropriate employ- ers who are willing to commit the resources necessary to study the effects of VG-GATB referral. SPECIAL POPULATIONS Veterans Like members of protected minority groups, military veterans have been the object of federal law and policy intended to increase their participation in the work force. The Wagner-Peyser Act creating the Public Employment Service in 1933 placed responsibility for veterans' employment with this system. The act also stipulated that qualified veterans should have priority over qualified nonveterans in employment

OCR for page 281
288 CONCLUSIONS ED RECOMMENDATIONS and training services. The language of the legislation and regulations conferring preference or priority on military veterans consistently uses the terminology "qualified veterans." We infer from this wording that the intent of Congress was to balance considerations of productivity with preferential treatment for veterans. 14. If government policy is to strike a balance between maximizing productivity and preference for veterans in employment referral through the VG-GATB Referral System, the Employment Service should adjust veterans' VG-GATB scores by adding a veterans' bonus of some number of points before conversion to percentiles. Unadjusted expectancy scores should also be reported to employers and job seekers. It should be noted on the referral slip that the percentile score has been adjusted for veterans' preference. If the federal rule is followed, the size of the adjustment would range from one-eighth to one-quarter of a standard deviation, corresponding to 5 and 10 percentile points, depend- ing on disability status. 15. The Employment Service should continue to meet the needs of disabled veterans through individualized counseling and placement serv- ices. People with Handicapping Conditions When tests are modified to accommodate visual, hearing, motor, or other handicaps, questions are raised about the comparability of the modified and regular instruments and about the meaning of the resulting scores. Even in the best of circumstances, very few data exist to answer these questions empirically; for the GATE, the research base is meager when it exists at all. Special administrations are offered for people with hearing problems, but the test has not been modified for people with visual handicaps. Extreme caution is clearly required in interpreting test results from special administrations or regular administrations to people with handicapping conditions. 16. For applicants with handicapping conditions, we recommend the continued use of job counselors to make referrals. 17. Measures should be taken to ensure that no job order is filled automatically and solely through the VG-GATB system. Job counselors who serve handicapped applicants, disabled veterans, or other popula- tions with special needs must have regular access to the daily flow of job orders. 18. To ensure that handicapped applicants who can compete with tested applicants are given that opportunity, the GATB should be used when feasible to assess the abilities of handicapped applicants. But the

OCR for page 281
CENTRAL COMMENDATIONS 289 test should be used to supplement decision making, not to take the place of counseling services. 19. Because special expertise in assessing the capabilities of people with handicaps is necessary and available, we recommend that the Department of Labor encourage closer coordination between state reha- bilitation agencies and the State Employment Service Agencies. States should consider placing state rehabilitation counselors in local employ- ment service offices that serve a sizable population of handicapped people.

OCR for page 281