|
Items in cart [0] |
Questions? Call 888-624-8373 |
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 148
7
INTRODUCTION
An Assessment of the
Dictionary of
Occupational Titles
as a Source of
Occupational Information
In the preceding chapter, procedures used to compile the most recent
edition of the Dictionary of Occupational Titles are described, and several
concerns are raised about the quality and characteristics of the fourth
edition in light of the way it was produced. To fulfill the committee's
charge to make recommendations about whether future editions of the
DOT should be produced and what kinds of occupational research should
be conducted to produce them, an evaluation of the quality and
characteristics of the DOT is presented in this chapter. The results of this
assessment, coupled with knowledge about use, have helped to inform us
as to how well the data contained in the DOT meet the purposes for which
they are intended and/or used. This assessment is also a basis for the
committee's recommendations about whether data collection and analysis
activities used in compiling future editions of the DOT should differ
substantially from what has been done in the past.
Establishing the quality and characteristics of data contained in the DOT
is not a straightforward task. First, as already mentioned, data collection
procedures were not well documented. As a result the possibilities are
Pamela S. Cain had primary responsibility for the preparation of this chapter.
148
OCR for page 149
An Assessment of DOT as a Source of Occupational Information 149
limited for systematic secondary analysis of the procedures themselves or
of their implications for the resulting data. Second, most of the data
contained in the DOT are unique, so no readily available bench marks exist
against which to compare and assess them. In fact, a great deal of
occupational research takes the DOT as the bench mark or standard of
comparison, a fact that makes the assessment of DOT data even more
important. In this chapter we present the results of several analyses that
were designed to explore in detail and systematically the nature of the
process by which the DOT was produced and the quality and characteris-
tics of the resulting data.
SAMPLING PROCEDURES
As described in chapter 6, the industry designations developed by the
occupational analysis program provide the "sampling frames" from which
establishments are selected for on-site visits. The underlying assumptions
of the procedure are that jobs vary by industry, by region, and by size (i.e.,
number of employees) and that these criteria provide the soundest basis for
achieving reasonable coverage of all jobs and for discovering significant
variations among jobs within occupations. Within the establishments
chosen, emphasis is put on analyzing those jobs that appear to be unique to
the work performed in establishments of the type that the selected one
represents.
No bench mark data on the "population" of jobs exist, and the
procedures by which specific choices were made about which jobs to study
are not well documented. Consequently, it is not possible to establish
whether the DOT provides comprehensive and representative information
about jobs in the U.S. economy. Nevertheless, certain aspects of the
procedures and their outcomes raise serious questions about the success in
attaining representative coverage.
A total of 232 industry designations are used to delineate the
"universes" from which sample establishments are chosen. As we have
noted in chapter 2, several of these, notably the designation clerical and
kindred workers, are not in fact industries, and their use carries the
implicit assumption that such occupations do not vary significantly in
content among establishments of different types. As a consequence of this
treatment of a number of nonproduction occupations the majority of the
232 industry designations that provide the universes from which establish-
ments are selected are in the manufacturing sector. In contrast, the current
version of the Standard Industrial Classification denotes 1,005 industries at
its most detailed level, and less than half are in manufacturing. Viewed in
this context then, the DOT cannot be said to be based on job analyses
OCR for page 150
lso
WORK, JOBS, AND OCCUPATIONS
conducted in establishments representing the entire spectrum of U.S.
industry types.
Comparable establishment-level data for the DOT and the U.S. economy
can be used to yield a crude indicator of the direction in which the job
analysis efforts for the fourth edition DOT were channeled. In themselves
these data do not constitute an evaluation of the DOT'S coverage, since the
critical issue, under the assumptions of the procedure currently used, is the
variety of types of establishments rather than the number of establishments
(or the number of employees). Nevertheless, comparison of the two
distributions reinforces the impression of a disproportionate emphasis on
manufacturing.
Data for DOT establishments were obtained from a set of staffing
schedules that were recently computerized and made available to us by the
national office of the Division of Occupational Analysis. As noted in
chapter 6, in the course of fourth edition production, staffing schedules
were not prepared for all establishments entered or for all jobs analyzed.
Furthermore, computerization of the schedules had not yet been com-
pleted at the time of the committee's study. Thus the data employed in our
analysis cover only 2,063 establishments; schedules for an estimated 1,100
to 1,200 establishments are still outstanding.)
The characteristics of establishments in which staffing schedules were
not completed or of establishments whose schedules had not yet been
computerized cannot be determined. As far as we can ascertain, there is no
reason to believe that there are marked differences between the characteris-
tics of establishments for which data are and are not available, especially
since analysts were supposed to complete staffing schedules for every
establishment in which they analyzed a significant number of jobs. Given
the procedures by which staffing schedules were filled out and their
purpose, however, we conjecture that analysts may have been more likely
to complete the schedules in larger, more bureaucratic establishments,
especially those with personnel offices.
Data on the national population of establishments were obtained from
tables in County Business Patterns, 1974 (U.S. Bureau of the Census,
1977~. This publication is compiled by the Census Bureau using data from
the administrative records of the Internal Revenue Service and the Social
Security Administration. Information is available on establishments,
payroll, and employment by industrial classification, size class, and county
for all types of employment covered by the Federal Insurance Contribu-
tions Act. In 1974 these data covered approximately 90 percent of U.S.
This information was obtained through personal communication with staff at the national
office and the North Carolina field center.
OCR for page 151
An Assessment of DOT as a Source of Occupational Information 151
establishments and 75 percent of the employed population. Not covered
were some government employees; self-employed persons; and certain
types of farm, domestic service, and railroad workers.
In order to compare the DOT data with the published data on the
national population of establishments, the staffing schedules were recoded
to the categories used in County Business Patterns. The DOT establishments
in public administration (N = 59) were excluded from tabulations, as were
establishments for which data were missing. These exclusions resulted in
the loss of 113 establishments and a final total of 1,950 establishments in
the DOT sample.
Table 7-1 presents a comparison of the percentage distribution of DOT
and U.S. establishments by sac major industry division. The two
distributions exhibit marked dissimilarities. The largest discrepancy occurs
in the manufacturing category: 67 percent of the DOT establishments are in
manufacturing industries, although this category accounts for only 8
percent of all U.S. establishments and for 32 percent of total employment.
Underrepresentation is most pronounced in the retail trade and services
divisions. Retail trade accounts for a mere 4 percent of the DOT
establishments, although nationally, it includes 29 percent of establish-
ments and employs 20 percent of the labor force. Only 7 percent of the
DOT establishments are in the services division, an industry division that
accounts for 27 percent of all U.S. establishments and for 20 percent of
U.S. employment. Both retail trade and services include establishments
engaged in a great variety of activities. It seems highly improbable that the
disparity in coverage between these major industry divisions and the
manufacturing division reflects a real difference in the heterogeneity of
occupations.
As previously noted, the wide disparity between the two distributions
cannot be interpreted as conclusive evidence; but it does suggest that the
procedures used to select establishments for the fourth edition DOT
resulted in an overrepresentation of establishments in manufacturing
industries. This overrepresentation occurred primarily at the expense of
the retail trade and service industries, which include 40 percent of all
workers. Moreover, the comments and observations of field center
personnel lend additional support to the general impression that job
analysis activities have tended to place emphasis on manufacturing
industries.
Size was another important criterion of establishment selection accord-
ing to the occupational analysts, one for which national data are also
available from County Business Patterns, 1974 (U.S. Bureau of the Census,
1977~. In Table 7-2 the percentage distribution of establishments by size
class (number of employees) is presented for the DOT and for the U.S.
OCR for page 152
152
WORK, JOBS, AND OCCUPATIONS
TABLE 7-1 Percentage Distribution of Establishments by SIC
Industry Division: Comparison of DOT Sample a and U.S. Labor Force b
Establishments
U.S. Labor
DOT,U.s.,Force, c
sac DivisionDOT, Npercentagepercentagepercentage
Agricultural services,
forestry, fisheries1618.30.90.3
Mining271.40.61.1
Contract construction522.49.16.2
Manufacturing1,30967.27.632.1
Transportation and
utilities954.93.56.4
Wholesale trade402.18.77.0
Retail trade824.229.019.6
. . .
. finance, insurance,
real estate442.39.06.8
Services1407.226.819.6
Nonclassifiable ~00.04.80.9
TOTAL1,950100.0100.0100.0
a DOT data taken from establishment staffing schedules. For purposes of comparison with
U.S. data, establishments in public administration were eliminated from tabulation.
bSOURCE: Coun~BusinessPatterns,1974(U.S.BureauoftheCensus,1977:TablelB).
C Workers employed in the establishments covered, not the employed civilian labor
force.
Included in this category are establishments that could not be classified because of
insufficient information. Typically, these were new businesses.
population of establishments. This comparison also reveals discrepancies
between the DOT sample and the national population; the discrepancy is
particularly large in the smallest size class. Establishments employing one
to four workers made up 59 percent of all U.S. establishments but only 6
percent of the DOT establishments. Generally, small establishments with
fewer than 20 employees were underrepresented in the DOT sample, while
intermediate (20 to 249 employees) and large (250 or more employees)
establishments were overrepresented in relation to the U.S. distribution of
establishments. There is a rather close correspondence, however, between
the DOT distribution of establishments and the distribution of U.S.
employment.
Once again, we point out that the implications of these results for the
OCR for page 153
An Assessment of DOT as a Source of Occupational Information 153
TABLE 7-2 Percentage Distribution of Establishments by
Employment-Size Class: Comparison of DOT Sample a and U.S. Labor
Force b
Establishments
U.S. Labor
DOT,U.S., Force, c
Size DOT, Npercentagepercentage percentage
1-4 1256.458.7 7.2
5-9 1497.618.0 8.2
10-19 20010.311.3 10.4
20-49 36718.87.5 15.3
50-99 27714.22.4 11.4
100-249 33817.31.4 13.6
250-499 21611.10.4 9.6
500-999 1206.20.2 8.3
1,000+ 1588.10.1 16.0
TOTAL 1,9501 ~.01 00.0 1 00.0
a DOT data taken from establishment staffing schedules. For purposes of comparison with
U.S. data, establishments in public administration were eliminated from tabulation.
bSOURCE: Coun~Business Patterns, 1974(U.S. Bureau ofthe Census, 1977: Table 1B).
C Workers employed in the establishments covered, not the employed civilian labor
force.
coverage of jobs are not straightforward. If the assumption that industry
type is the proper basis for sampling establishments is correct, then an
important first step might be to revise the industry list so that it provides
coverage of all unit items in the sac. In this frame of reference the number
of establishments in each industry would not be relevant, since the
objective would be to obtain adequate minimum coverage for each separate
type of establishment. On the other hand, if jobs in manufacturing are
more diverse than those in other sectors, then oversampling of manufac-
turing enterprises is quite appropriate. The DOT analysts would be
expected to devote more of their attention to establishments (and
presumably jobs) in these industries. Furthermore, if jobs tend to be
similar in large and small establishments, undersampling small establish-
ments and oversampling large estabishments would be justified on grounds
of cost effectiveness.
The difficulty is that there is no evidence at all regarding the relationship
between type of establishment and the variability of job content. We do not
OCR for page 154
154
WORK, JOBS, AND OCCUPATIONS
know whether manufacturing jobs are more heterogeneous than other jobs
or whether jobs in small establishments differ from ostensibly similar jobs
in large establishments or in other small establishments.
In addition to considering the types and sizes of establishments
providing the base data for the DOT, it is also possible to compare the
distribution of occupational units in the DOT with the distribution of
workers. This approach also has very obvious limitations, since some
occupational units include large numbers of workers and others include
relatively few. Nevertheless, the data presented in Table 7-3, in which DOT
coverage and labor force employment by major occupational category are
shown, reveal very marked discrepancies. Some 60 percent of all base titles
fall in the processing, machine trades, and benchwork categories, although
these categories include only about 12 percent of the labor force. Taken in
conjunction with the finding (documented in Table 7-5 below) that a
substantial proportion of occupational titles are supported by one (or no)
job analysis schedule, the skewness of the distribution in Table 7-3 raises
the conjecture that the choice of jobs for analysis has a major impact on
the number of occupations identified and that therefore the concentration
of attention on manufacturing establishments has an important impact on
the entire classification structure. To state this more explicitly, if there is a
strong tendency for each job analysis to result in the identification of a
separate occupation (as Table 7-5 seems to imply), the selection of job
analysis sites and of the jobs to be analyzed at these sites becomes the
crucial decision of the occupational analysis program.
As noted above, the procedures for selecting sites for job analysis were
not carefully developed. Analysts drew heavily on the third edition DOT to
guide their job analysis activities. This practice might well have led them
to concentrate more on jobs in established manufacturing industries
(which were well represented in earlier editions) and to devote less
attention to jobs in newly emerging or rapidly growing sectors of the
economy, such as services or retail trade. In addition, it was clear to us in
talking with the analysts that many were oriented almost exclusively
toward the study of production jobs. Undoubtedly, this orientation is a
historical outgrowth of the program, rooted in tradition, but other reasons
may be salient, such as the ease of access to manufacturing establishments.
Similarly, the emphasis on large establishments may have come about
because of the relative efficiency of analyzing many jobs in a few large
establishments versus a few jobs each in many small ones.
For whatever reasons the concentration on manufacturing and relatively
large establishments came about, and whatever its implications are for the
coverage of jobs, the results of the foregoing comparisons raise questions
about exactly how sampling for the DOT should proceed. Previous
practices were relatively unsystematic, virtually uninfonned by empirical
OCR for page 155
An Assessment of DOT as a Source of Occupational Information 155
TABLE 7-3 Comparison of Percentage Distributions of DOT Titles
and Labor Force by DOT Occupational Categories
Percentage of
Base Titles Percentage of
DOT Occupational Category(N = 12,099) Labor Force
Professional, technical, and managerial12 25
Clerical and sales8 25
Service4 16
Agriculture, fishing, and forestry2 4
Processing23 2
Machine trades18 6
Benchwork19 4
Structural work7 9
Miscellaneous7 8
TOTAL100 99
SOURCE: Labor force data derived from April 1971, Current Population Survey; sample
(N = 60,441) includes currently employed workers and experienced unemployed for
whom a census code could be assigned. Excluded are 12 percent of sample for whom WT
codes could not be assigned. Data on distribution of DOT titles by category provided by
the Department of Labor occupational analysis program.
data, and resulted in relative inattention to several sectors that include
large proportions of workers. The distributions of workers or of establish-
ments that we have had to use as crude indicators are not the basic
relevant criteria, of course; a more desirable goal would be the iden-
tification of the types of organizations that have unique types of jobs, with
at least minimum coverage of these unique types of jobs.
A sampling strategy that would ensure adequate coverage of the job
content of the American economy will not be easy to develop, but it is
essential that work on this problem be initiated immediately if the DOT is
to serve the many demands that are made of it.
SOURCE DATA
Chapter 6 observes that the amount and type of source data supporting
DOT titles and definitions vary and that the quality of the data appears to
be uneven. These conclusions were based on examination of the source
data, on reports from analysts involved in writing definitions, and on
findings of the Booz, Allen & Hamilton, Inc. (1979) management review.
In this section a more systematic and detailed inquiry into the quality of
OCR for page 156
156
WORK, JOBS, AND OCCUPATIONS
source data is undertaken to determine the extent to which departures
from standard procedures occurred and whether such departures vary by
period or across certain types of jobs. As is evident from the discussion in
chapter 6, there are numerous points at which departures could have
occurred. The nature of these departures is important to the extent that
they have deleteriously affected the quality and comparability of the data
in the DOT.
To assess the quality of DOT documentation, we used a set of data
collected by Booz, Allen & Hamilton as part of its management review.
Because the only information available on the procedures by which the
DOT was produced is anecdotal and impressionistic, Booz, Allen &
Hamilton conducted a special study of DOT source data in November
1978. Analysts at the North Carolina field center were requested to record
information on the documentation available for a sample of 307 DOT base
titles. The sample was systematically selected by choosing every fortieth
title in the DOT. However, there was an occasional departure from this
procedure. If the title selected was not a base title, a substitution was
made, but the procedure by which this was done is unclear.
Even though the sample is slightly unsytematic, the difficulties of
conducting another similar study justify the use of these data to get an idea
of the quality of DOT documentation. As a check on the Booz, Allen &
Hamilton sample, the percentage distribution of base titles by DOT major
occupational categories for the sample was compared with that of the DOT.
The comparison, in Table 7-4, reveals that the two distributions are very
similar. Hence on this criterion at least, the sample appears to be quite
representative of the population from which it was drawn.
The distribution of DOT titles by the kind of documentation available for
each is shown in Table 7-5. The summary information at the end of the
table shows that 11 percent of the DOT titles had no supporting
documentation other than the third edition definition, which was based on
job analyses conducted prior to 1965. Seventy-one percent of titles were
supported by job analysis schedules only, 8 percent by schedules and
occupational code requests, and the remaining 10 percent by other
combinations of data. Thus job analysis schedules constituted the bulk of
the data base for the DOT, other types of information making up a
relatively small percentage of the source data.
The quality of the definitions for the 11 percent of titles lacking any sort
of documentation other than the third edition is particularly questionable,
. .. . ~ . . ~ ~ , .
since there Is no way ot knowing whether and to what extent changes in
the content of these jobs occurred between the third and fourth editions.
The quality of definitions based solely (5 percent) or in part (14 percent) on
information other than job analysis schedules may also be questionable.
OCR for page 157
An Assessment of DOT as a Source of Occupational Information ·57
TABLE 7-4 Percentage Distribution of DOT Titles
by Major Group: The DOT versus the Booz, Allen &
Hamilton Sample
Category DOT
Booz, Allen
& Hamilton Sample
0-11213
289
345
4
52321
61818
71919
878
976
TOTAL1~1~
N(12,099)(307)
Occupational code requests, for example, are essentially employers' job
orders, which are taken over the phone and may not be verified on site. As
a result the job specifications contained in code requests probably reflect
hiring requirements rather than the functional requirements of jobs, as
would have been determined via on-site analysis. Similarly, information
obtained through letters from trade associations (which are, in part,
advocacy groups) is perhaps more likely to depict the ideal job than the
average or typical one. For both sources of information, skill and other
requirements of the job may be inflated or biased upward, in relation to
what would have been determined through on-site analysis. If these data
continue to be used to support DOT definitions, steps should probably be
taken to determine their properties and possible biases and their
comparability to data obtained via on-site observations and interviews.
Table 7-5 shows the distribution of titles by the number of job analysis
schedules available for each. Sixteen percent of DOT occupations are
unsupported by job analysis schedules (11 percent of these are completely
unsupported, and 4.5 percent are supported by other types of information).
Of the total number of occupations an additional 29 percent are supported
by only one schedule, 19 percent by two schedules, and the remaining 37
percent by three or more schedules.
The small number of jobs analyzed per title raises additional questions
about the inclusiveness and accuracy of the occupational information
OCR for page 158
158
WORK, JOBS, AND OCCUPATIONS
TABLE 7-5 Percentage Distribution of DOT Titles
by Number and Type of Supporting Documentation
Documentation
Percentage
Number of job analysis schedules (]AS)
o
2
4
6
8+
TOTAL
Number of occupational code requests (OCR)
o
3+
TOTAL
Number of othera sources
o
1
2
3+
TOTAL
All forms of documentation
None
JAS only
OCR only
Other only
WAS and OCR
JAS and other
JAS, OCR, and other
TOTAL
TOTAL N
16
29
19
4
2
13
101
90
6
2
2
100
89
8
2
1
100
1 1
4
8
101
307
a Other includes comments from trade associations, job descriptions
from employees, etc.
SOURCE: Tabulated using data from Booz, Allen & Hamilton study
of DOT documentation.
OCR for page 185
An Assessment of DOT as a Source of Occupational Information 185
TABLE 7-13 (continued)
Factor
Variables 1 2 3 4 s 6
Eigenvalue 10.86 4.98 2.18 1.20 1.09 0.63
Percentage variance 49.30 22.60 9.90 5.40 4.90 2.90
Cumulative percentage 49.30 72.00 81.90 87.30 92.20 95.10
a Factor loadings greater than or equal to .4 are in boldface.
b Where necessary, scores on variables were reflected so that high scores represent high
levels of the trait.
relationship in the DOT between the substantive complexity of occupations
and their managerial responsibilities.
The fifth and sixth factors account for 5 and 3 percent of the shared
variance in the matrix, respectively. Factor 5, which is composed of only 4
items, might be labeled "interpersonal skills." An inspection of the items'
content reveals that this dimension involves working with feelings and
ideas and sensory or judgmental criteria and that it involves influencing
people and dealing with their social welfare. The sixth factor, although it
accounts for only 3 percent of the variance, is readily interpretable as
reflecting undesirable aspects of the working conditions of occupations.
By and large, the results of this factor analysis are straightforward.
Several variables did load on more than one factor: as noted, there is some
overlap between factors 1 and 4; factors 1 and 2 also share two items in
common. Only five variables (COLORDIS, PUS, COLD, WET, and NOISE),
failed to load significantly on any of the factors. Of these five variables, all
but COLORDIS are dichotomous variables with limited variance. The
variable COLORDIS (occupations requiring an aptitude for color discrimina-
tion) appears to tap a unique occupational dimension. Presumably, many
occupations require similar special aptitudes, but since each aptitude is
probably required of only a few occupations, it would be preferable to
include such information as part of the occupational definition.
These results can be interpreted in two ways. The most straightforward
interpretation is simply that there is a great deal of redundancy among
DOT indicators. Alternatively, the factor patterns just presented could
result from the procedures used in making DOT ratings. In rating
occupations for these traits, occupational analysts might have forced
consistency among them. It is true that many of the functions and traits
appear to tap nearly identical phenomena (e.g., GED and INTELL).
However, it is also the case that the way in which the ratings were made-
OCR for page 186
186
WORK, JOBS, AND OCCUPATIONS
TABLE 7-14 Factor Analysis of Fourth Edition DOT Occupational
Characteristics: Items and Loadings for Six Major Factors
Variable
Label description
Loading
Factor 1: substantive complexity, 49.3 percent a
GED general educational development
svP specific vocational preparation
INTELL
DATA
REPCON
NUMER
VERBAL
ABSTRACT
MVC
CLERICAL
SPATIAL
PEOPLE
FORM
TALK
pop
VARCH
DATACOM
intelligence b
complexity of functioning with data b
repetitive or continuous processes
numerical aptitude b
verbal aptitude b
abstract and creative versus routine, concrete activities
measurable or verifiable criteria
clerical perception b
spatial perception b
complexity of functioning with people b
form perception b
talking
direction, control, and planning
variety and change
communication of data versus activities with things
Factor 2: motor skills, 22.6 percent a
FINGDEX finger dexterity b
MOTOR motor coordination b
MANDEX manual dexterity b
THINGS
FORM
SPATIAL
SEE
REACH
STS
MACHINE
complexity of functioning with things b
form perception b
spatial perception b
seeing
reaching
set limits, tolerances, or standards
activities involving processes, machines versus
social welfare
Factor 3: physical demands, 9.9 percept a
LOCATION outside working conditions
STOOP stooping, kneeling, crouching, crawling
EYEHAND
CLIMB
STRENGTH
eye-hand-foot coordination b
climbing, balancing
lifting, carrying, pulling, pushing
Factor 4: management, 5.4 percept a
DEPL dealing with people
pop direction, control, planning
PEOPLE complexity of functioning with people b
TALK talking
.86
.86
.83
.81
.81
.78
.76
.68
.64
.64
.55
.47
.46
.44
.43
.42
.41
.69
.68
.67
.66
.52
.47
.43
.42
.37
.33
.67
.53
.52
.49
.48
.78
.74
.70
.64
OCR for page 187
An Assessment of DOT as a Source of Occupational Information 187
TABLE 7-14 (continued)
Variable
Label Description Loading
SCIENCE
DATA
TANGIBLE activities resulting in tangible satisfaction versus prestige
scientific, technical activities versus business contact
DATACOM communication of data versus activities with things
complexity of functioning with data b
Factor 5: interpersonal skills, 4.9 percept a
sac
FIF
IN FLU
sensory or judgmental criteria
feelings, ideas, facts
influencing people
MACHINE activities involving processes, machines versus
social welfare
Factor 6: undesirable working conditions, 2.9 percent a
HAZARDS hazardous conditions
ATMOSPHR fumes, odors, dust, poor ventilation
HEAT extreme heat
63
57
.49
.44
.51
.41
.41
37
.52
.42
.37
a Percentage of common variance explained.
b Sign reflected on this variable.
all ratings assigned at one time by a single analyst-could have inflated the
degree of consistency among the scores for each occupation and hence the
degree of correlation between variables measured over occupations. This is
called a "halo effect," the tendency of one judgment to be affected by
another. It is well known that when several ratings are made at a single
time by a single judge, they tend to be more consistent than when the
ratings are made independently of one another (Selltiz et al., 1959:351-
352~.
Evidence that the rating procedure itself is an important source of the
high degree of interrelationship among the DOT variables is offered by the
results of a similar factor analysis performed by using third edition data
(Barker, 1969~. For the third edition, different analysts rated each of the
traits: one analyst rated occupations for aptitudes, another for tempera-
ments, etc., a procedure that would mitigate the tendency to force
consistency among the ratings. In an analysis of third edition ratings,
Barker found that 11 factors emerged and that the factor loadings,
commonalities, and percentage of common variance explained were all
much lower than the estimates presented here. Although other reasons
could account for the differences between his findings and ours (e.g.,
differences in the underlying distribution of occupations), the suspicion is
strong that the differences are attributable to the change in the rating
OCR for page 188
188
WORK, JOBS, AND OCCUPATIONS
procedures from the third to fourth edition, that is, that the high
covariation among the worker functions and worker traits is an artifact at
least in part of the procedures used to rate DOT occupations. If this is so,
these findings suggest that a modification of current rating procedures is
needed along with a careful examination of the content of the items
themselves.
These results suggest that the more reliable indicators of the features of
occupations tapped by the worker traits and worker functions variables
could be created by developing factor-based multiple-item scales to
represent the various dimensions revealed by the factor analysis. Such
scales would have the advantage of greater internal reliability and
consistency than single indicators or scales created by simple summing of
items without knowledge of their factor structure. In Appendix F we
present scores for scales constructed in this way for the categories of the
1970 U.S. Census detailed occupational classification.
SEX BIAS IN THE RATING OF OCCUPATIONS
Recently, the DOT has come under attack for alleged sex bias. It has been
claimed that in the third edition DOT both the occupational descriptions
and the ratings of occupational characteristics undervalued jobs held
mainly by women (Wits and Naherny, 1975~. In particular, it has been
asserted that third edition ratings of the complexity of work in relation to
data, people, and things reflect traditional stereotypes regarding the
relative complexity of the kinds of jobs typically held by women and those
typically held by men (Wits and Naherny, 1975~. Consideration of a few
examples is sufficient to legitimate the charge of sex bias in the third
edition. In it the DATA, PEOPLE, and THINGS variables included as the
lowest response level a judgment that an occupation had "no significant
relationship" to data, people, or things. Typist, a job held mainly by
women, was coded as having no significant relationship to things, whereas
Typesetting-Machine Tender, a job held mainly by men, was coded at a
higher level of complexity. Such jobs as Nursery School Teacher and
Practical Nurse were coded as having minimal or no significant relation-
ship to data, people, and things, while such jobs as Dog Pound Attendant
were rated as functioning at a higher level of complexity.
According to informants in the national office the no significant
relationship category for the worker functions was dropped in the fourth
edition in response to the charge of sex bias in the third edition.
Occupations that had been scored at the lowest complexity levels in the
third edition were assigned new worker function scores. In addition, in
OCR for page 189
An Assessment of DOT as a Source of Occupational Information 189
some instances other scores were changed, presumably to reflect changes
in job content or to correct other errors in the third edition.
In order to document the changes made between the third and fourth
editions and to determine whether the ratings of occupations commonly
pursued by women had been upgraded as claimed, we conducted an
analysis of third and fourth edition worker function ratings. This was done
by utilizing the April 1971 Current Population Survey (cPs) of a
representative sample of the labor force. This data set contains, among
other variables, both the third and fourth edition DOT codes for the job
held at the time of the survey and the sex of each worker. The cPs data set
includes data for 60,441 members of the labor force. Third edition DOT
codes were assigned to each occupational response by trained occupational
analysts in the occupational analysis field centers. Fourth edition codes
were subsequently added to the data, using a map prepared by the Division
of Occupational Analysis that related fourth edition DOT codes to third
edition codes. By comparing third and fourth edition scores on the DATA,
PEOPLE, and THINGS variables separately for men and women, we can
determine the effect of scoring changes between the third and fourth
editions on the relative status of male and female workers. Note that our
sample for this analysis is composed of workers, not jobs. However,
neither workers nor jobs changed, only the classification of jobs in the DOT
scheme and hence the scoring of the worker function variables. An
analysis of the nature of these changes permits an indirect inference about
the extent of sex bias remaining in the fourth edition DOT.
We begin by considering the labor force as a whole (see Table 7-15~. In
1971, about a third of both the male and female labor force were in
occupations that were judged in the third edition to have no significant
relationship to data. In contrast, a much larger proportion of men than
women were in occupations having no significant relationship to people,
and a much larger proportion of women than men were in occupations
with no significant relationship to things. The second line of the table,
which gives the mean fourth edition score for occupations with "no
significant relationship" in the third edition, shows what happened to these
occupations in the fourth edition. On average, the occupations held by
men and those held by women were assigned similar scores on the DATA
and PEOPLE variables, but on the THINGS variable the occupations held by
women were judged to be more complex than the occupations held by
men. In short, the major effect of the abolition of the no significant
relationship category was to upgrade substantially the complexity in
relation to things of occupations held by women. This conclusion is also
evident in the "difference in means" row, which shows the difference in the
average score between the third and fourth editions. Since a low score
OCR for page 190
l9o
WORK, JOBS, AND OCCUPATIONS
means greater complexity, the fact that all the numbers in the row are
negative indicates an average upgrading of complexity levels between the
third and fourth editions. The only change of substantive importance,
however, is the upgrading of occupations held by women on the THINGS
variable.
The remaining point to note concerning the total labor force is that
except for changes required by the abolition of the no significant
relationship codes, there were few changes in ratings between the third and
fourth editions. More than 90 percent of the scores remained unchanged
between the two editions, as perhaps was to be expected, given the way in
which DOT occupational data were generated.
Inspection of the second section of Table 7-15 allows us to identify a
major source of change in the THINGS ratings: the upgrading of clerical
and sales jobs held by women. Most clerical and sales jobs (whether held
by men or women) were identified in the third edition as having no
significant relationship to things. However, the occupations held by
women were coded substantially differently on the THINGS variable in the
fourth edition from those held by men; on average, the clerical and sales
occupations held by women were judged as having much greater
complexity than those held by men. No doubt this reflects the greater
propensity of female clerical and sales workers than male clerical and sales
workers to operate office machines. Whereas in the third edition the task of
typing was rated as not involving a significant relationship to things (level
8), in the fourth edition it was rated as involving the "operating-
controlling" of things (level 2~. The same sort of coding change was made
for a large number of positions involving the operation of office machines.
Hence while both clerical and sales occupations held by women and those
held by men tended to be upgraded in the fourth edition, the upgrading
was much greater for the jobs held by women. Thus on the basis of fourth
edition scores the average female clerical and sales worker is scored as
doing more complex work in relation to things than the average male
clerical and sales worker.
In contrast to the clerical and sales sector the service and benchwork
sectors included here because they are also large employers of women
do not exhibit radically different patterns of upgrading for jobs held by
men and those held by women, although they do show significant
differences in the proportion of occupations in the third edition with no
significant relationship to data, people, and things.
What do these results tell us about sex bias in the fourth edition DoT?
Although no definitive judgment is possible in the absence of an external
criterion of job complexity against which to assess the DOT ratings, the
relative similarity in the mean scores for male and female workers is
OCR for page 191
An Assessment of DOT as a Source of Occupational Information 191
certainly consistent with an inference that these variables are largely bias
free. For the total labor force, the means for the DATA variable vary by
only about half a point, and the means for PEOPLE and THINGS by even
less. Although the means are lower for men, indicating that they work in
occupations with greater complexity than those held by women, the size of
the differences is within what would be expected from well-known patterns
of occupational segregation by sex. Hence there is no reason to believe that
the kink! of work women do is undervalued in the fourth edition DOT, at
least with respect to the worker function ratings. Of course, the possibility
exists that the work that women do is overvalued and that if unbiased
scores were available, the mean difference between male and female
workers would be even greater. However, this is unlikely, given otter
evidence demonstrating that men and women are equally well educated on
the average and hold jobs with similar average prestige (Treiman and
Terrell, 1975a, b), that the average GED levels of the jobs held by men and
by women are virtually identical (the means are 3.14 and 3.20), and that
the average svP levels of the jobs held by men and by women differ by only
about a half a point (the means are 4.70 and 4.14~. These results imply that
the worker function ratings in the fourth edition-but not the third
edition-can be used to assess sex differences in occupational attainment
without undue distortion (see chapter 4 for a discussion of such analyses).
CONCLUSION
This chapter deals with two major issues, the adequacy of the source data
used to create the DOT and the adequacy of the data on occupational
characteristics created in conjunction with the DOT. These issues are, of
course, not unrelated, since the adequacy of the source data determines, in
part, the adequacy of the resulting occupational characteristics scales. Still,
it is useful to consider them separately.
The chapter documents the very uneven coverage of the labor force in
the basic data collection process. First, the DOT includes many more
production process occupations, relative to the number of individuals in
the labor force employed in such occupations, than clerical, sales, and
service occupations. While it may be that production process occupations
are, in fact, more finely differentiated in the economy than are other
occupations, there is no evidence that this is so. An equally plausible
explanation is that DOT data collection procedures, which tend to
concentrate on manufacturing plants, create a bias toward more detailed
coverage of production process occupations than of other types of work.
At present, there is no way of resolving this question, since there exist no
principles for determining the boundaries of occupations and hence no
OCR for page 192
192
o
._
._
LO
so
._
a'
a'
3
CQ
By
Cal
LO
o
o
of
._
so
o
Cal
Cal
Cal
1 C-
O
LL1 ~
a'
m ~
EM o
Ct
Lie
V]
V ~
~Ct
:: ~
o
:L
6
E~
ro
an ~
_..
Cto
a,
Ct
Ct
Cal 0\ V) rid
-, At ~ in, '.,) -~ As, ~
To ~ 1
_.
en ~ ~ ~ rid ~ ~ rat oo so
ON ~ - , ~ ~ ~ ~ 1- ~
rid ~ ~OF
Go No ~ ~No
r ~ oo ~ ~ ~ ~ O ~O
oo - ~ ~ ~ 1 °°
oo
-
. . . . . . .
_ r~ ~ ~ \0 ~ I
~I
~ _
~O ~ 0 r~ oo ~ ~
~,
_ - - O
r~
oo
oo c~
~oo
O
-
oo _
_ ~ oo ~ O
~r ~_ O _ _ v, ~ 00 ~O
r ~v) oo ~ r~ _ ~ ~ ~_ l- _
oo ~ _ ~ (~, _ 00 ~O O t
- ~ ~ 1 ~ oo oo
=: =:
3 ._
.o ~o o ~o
::
v, .o: - ~ o ~ ~ =^ o
· - s ~ - c o .o
s ~ - s ~ ~ ~ s s ~ ~ ~ ~ ~ s
~ D - 9', ~ D - ~y D~ 9'. e - s
OCR for page 193
193
OO ~ ~ rot red
cr
1
Can
...
1 ~
...
, Cry
1 ~
red
...
, ~
1
red~
oo
..
1~
11-
o ~
·- C33
.- (,
._
~ Cal
_`~ To
.~ s
V' .= ._
~u, 3
.= O ~
~0 0
It,) ~ _
~ C_,
_ ~ O
~ a=^=
Q)
.= .
·= ~ .P
~ S ~
O ~ ~
.0
.O
O
O O O O ~ ~
Hi 0 0 0 ~ Is) 1
O
Cal
i:
.
U.
~0
.
U.
· O
0^
at;}
11 ~
X
a,
° o
D ~O
Pi ~O
U.
·- ~ P.
- , - ~ . -
o s ~ ==
a' ~ l_ O
sp o-s ~
.- ~ ~ cd
.=
Op-= o o
~ ~L, ~ ~
. -
~S O '~ ~ 3
3 ^- 0'
Ce
. =. ~ . ~ 8 C , , , ~
o oo o
o 1 1
· ~ o
~ 1
o. ~ ~ ~ ~ oo
~, o
oo 0- 1
-
3
o
~:
-
O ~
=^ O
O ._ ~ ~
·- a' O . °
. ~ ._
S
.= ,=
p. ~l,
~ ~ O
O
~ .0 .~,
_ ~ ~ 1
3 5 t;
_ ~ r c-;
~ ~ _ 4_
·u,
a
o
_
_ '_
O e_
1.
1 o o
c~ ~ .~=
oo ~ ~ ~
=3 1 1 .=
O3
o o ~ S
m
V)
v
z
:r
o
00
1
o
o
a
o
00
o
c
~00 00 0
00 ~ _ ~ ~ ~ ~ ~
00 ~ ~ ~ ~ _
oo 1
00 ~ ~
C~ ~ ~ ~ ~ ~ _
00 ~ _ ~ ~ ~ _
oo ~1
OCR for page 194
194
WORK, JOBS, AND OCCUPATIONS
unambiguous procedures for aggregating jobs into occupations. The
development of such principles and of procedures for using them in the
data collection process should be given high priority in preparation for
future editions of the DOT.
Second, some occupations in the fourth edition DOT were analyzed
many times, while others were not analyzed at all. Given the heterogeneity
of jobs included within a single occupational category (which is confirmed
by the substantial "job description" effect on the reliability of worker trait
and worker function ratings), procedures need to be developed to ensure a
more even sampling of jobs within occupations in order to be certain that
each occupational description is based on data froin a sufficient number of
job analyses to produce representative data.
What constitutes an "occupation" and how much heterogeneity in the
content of a set of jobs justifies a single occupational title is a difficult
question. Historically, the DOT has tended to define occupations by their
titles rather than by their content. Jobs with similar titles have been
grouped unless the evidence strongly indicated that they differed in
content, and occupations with different titles have been defined as being
different, regardless of similarity in content. At the same time, each job
analysis tends to produce a new DOT occupation, while jobs with titles
similar to titles already existing in the DOT tend not to be analyzed at all,
making it impossible to determine their degree of similarity. Occupational
titles are also used inconsistently in the DOT to define very specific or very
heterogeneous groups of jobs. Branch manager, for example, describes a
wide variety of jobs, all of which involve coordination and control
functions but vary enormously in terms of the specific tasks performed.
Tool and Die Maker, by contrast, describes basically the same job
regardless of where tool and die makers are employed.
Consideration should be given to developing a clear and unambiguous
way of defining occupations.
The analysis in this chapter also raises serious questions regarding the
adequacy of the worker trait and worker function variables. First, it is
unclear whether the 46 variables on which data are collected adequately
represent the kind of information needed by users within and outside the
Employment Service. Our conjecture is that they do not. Many of the DOT
variables, especially the aptitudes, interests, and temperaments, are not
heavily used, as we have seen in chapters 3 and 4. Oddly, other
information collected on job analysis schedules but never subsequently
recorded, i.e., information on promotion ladders and lateral transfer
routes, is often mentioned by users outside the Employment Service as a
major lack in the DOT. Obviously, consideration should be given to the
inclusion of such information in the DOT occupational descriptions. More
OCR for page 195
An Assessment of DOT as a Source of Occupational Information ·95
generally, a careful conceptual review should be undertaken of the sort of
information needed for matching workers with jobs (e.g., data on the
transferability of skills), for counseling job applicants about occupational
requirements, for assessing the comparability of occupations for the
resolution of equal employment opportunity disputes (better data on the
responsibilities entailed in occupational performance, for example), and for
occupational research of various kinds. Once the major dimensions of
occupations on which data are needed are identified, scales measuring
these dimensions should be developed following standard psychometric
practices. In particular, consideration should be given to the development
of factor-based multiple-item scales, the use of which would go a long way
toward overcoming the reliability problems identified in Appendix E and
summarized in this chapter.
Despite the deficiencies in the fourth edition worker trait and worker
function variables identified here, they remain the most comprehensive set
of occupational characteristics currently available. As such, their use
should be encouraged. To facilitate this use, Appendix F provides data on
eight DOT variables aggregated to match the categories of the 1970 U.S.
Census detailed occupational classification and four factor-based scales
derived from the DOT variables. Researchers should find these data a
useful supplement to data on the average characteristics of workers that
can be derived from census occupational statistics. Moreover, one
potential major threat to the usefulness of these data can be discounted on
the basis of our analysis: so far as we can tell, the fourth edition worker
function variables do not undervalue occupations held mainly by women
as the third edition worker function variables apparently did.
Representative terms from entire chapter:
third edition
