An Assessment of Research-Doctorate Programs in the United States Mathematical and Physical Sciences (1982) / Chapter Skim
Currently Skimming:
IX. Summary and Discussion
IX. Summary and Discussion
Pages 159-192
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 159... ...
of the reputational survey ratings and an examination of some factors that might possibly have influenced the survey results. '~ ~r ~u~'uu~ we ~uyy=~`ur~ studies of this kind -- with particular attention given to measures one would like to have available for an assessment of research-doctorate programs.
|
From page 160... ...
and the mean values for each measure in the six mathematical and physical science disciplines. As can be seen, the mean values reported for individual measures vary considerably among disciplines.
|
From page 161... ...
161 TABLE 9.1 Mean Values for Each Program Measure, by Discipline Computer Geo Chemistry Sciences sciences Math Statistics/ Physics Biostat. Program Size 012316163328 12 025120192435 15 037541253556 22 Program Graduates 04.23.17.26.25.26 .32 055.96.57.06.67.1 6.7 06.76.80.77.61.66 .78 07.33.38.22.25.26 .43 Survey Results 082.52.52.92.72.7 2.8 091.61.51.81.61.7 1.6 101.11.11.11.21.1 1.1 11.9.9.9.8.7 .9 University Library 12.1.4.4.1.1 .5 Research Support 13.48.36.47.32.36 .25 141788117139966162943 NA Publication Records 1578344439106 12 Total Programs1455891115123 64
|
From page 162... ...
Differences in the mean ratings derived from the reputational survey are small. In all six disciplines the mean rating of scholarly quality of program faculty (measure 08)
|
From page 163... ...
In this chapter selected correlations to determine the extent to which coefficients are comparable in the six disciplines are presented. Special attention is given to the correlations involving the number of FY1975-79 program graduates (measure 02)
|
From page 164... ...
Program Size 01 .68.62.42 .50.77.53 03 .92.52.72 .85.92.48 Program Graduates 04 .02.05-.01 .08-.02.00 05 .38-.07.29 .31.32.04 06 .23.12.05 .18.40.00 07 .13-.05.36 .46.41-.03 Survey Results 08 .83.66.64 .70.76.55 09 .81.68.67 .68.73.63 10 .23-.02.06 .01-.17.17 11 .83.61.67 .72.78.59 University Library 12 .61.44.43 .45.47.11 Research Support 13 .57.34.40 .35.13.06 14 .72.58.25 .41.66N/A Publication Records 15 .83.85.73 .75.85.52 16 .86.84.74 .81.86.48
|
From page 165... ...
Tee pp. Table 9.3 shows the correlation coefficients for measure 08, the mean rating of the scholarly quality of program faculty, with each of the other variables.
|
From page 166... ...
Program Size 01 .64 .54 .45.48.68 .63 02 .83 .66 .64.70.76 .55 03 .81 .50 .61.64.75 .40 Program Graduates ' 04 .11 .35 .08.30.15 .19 05 .47 .14 .50.57.42 .32 06 .28 .21 .24.19.42 .15 07 .30 .17 .58.63.58 .25 Survey Results 09 .98 .98 .97.98.96 .95 10 .35 .29 .29-.01-.15 .30 11 .96 .97 .87.96.96 .93 University Library 12 .66 .58 .58.65.67 .53 Research Support 13 .77 .59 .72.70.24 .53 14 .79 .63 .27.42.61 N/A Publication Records 15 .80 .70 .75.75.85 .70 16 .86 .77 .77.83.86 .67 1
|
From page 167... ...
Correlations of ratings of faculty quality with measure 06, the fraction of program graduates with definite employment plans, are moderately high in physics and somewhat lower, but still positive, in the other disciplines. In every discipline except computer sciences the correlation of measure 08 is higher with measure 07, the fraction of graduates having agreed to employment at a Ph.D.-granting institution.
|
From page 168... ...
The pattern of relations is quite similar for programs in chemistry, computer sciences, and physics: moderately high correlations with measures of program size and somewhat higher correlations with both reputational survey results (except measure 10) and publication measures.
|
From page 169... ...
Program Size 01 .43.44.61 .18.54 N/A 02 .72.58.25 .41.66 N/A 03 .66.43.28 .44.68 N/A Program Graduates 04 .18.22.22 .29.04 N/A 05 .35-.21-.05 .17.31 N/A 06 .31-.03-.04 .23.25 N/A 07 .20-.16.06 .22.31 N/A Survey Results 08 .79.63.27 .42.61 N/A on .74.61.25 .42.61 N/A 10 .14-.02.13 -.12-.08 N/A 11 .77.64.18 .43.58 N/A University Library 12 .45.16.33 .33.33 N/A Research Support 13 .55.10.20 .18.07 N/A Publication Records 15 .70.66.42 .35.80 N/A 16 .78.73.35 .42.80 N/A r
|
From page 170... ...
Program Size 01 .65.61 .36 .63 .72.49 02 .86.84 .74 .81 .86.48 03 .84.52 .64 .78 .85.50 Program Graduates 04 .03.20 .07 .15 .05-.29 05 .41-.04 .31 .40 .38.37 06 .22.14 .00 .16 .43.11 07 .23-.01 .39 .50 .48.30 Survey Results 08 .86.77 .77 .83 .86.67 09 .82.75 .75 .80 .82.63 10 .33.05 .09 .05 -.14.15 11 .88.74 .70 .83 .86.66 University Library 12 .56.52 .66 .59 .61.36 Research Suppor t 13 .60.35 .51 .51 .21.56 14 .78.73 .35 .42 .80N/A Publication Records 15 .95.98 .97 .90 .99.98
|
From page 171... ...
Nonetheless, certain similarities may be seen in the pattern of correlations among the measures. High correlations consistently appear among measures 08, 09, and 11 from the reputational survey, and these measures also are prominently related to program size (measures 01, 02, and 03)
|
From page 172... ...
172 TABLE 9.6 Distribution of Responses to Each Survey Item, by Discipline Chem- Computer Geo- Statistics/ Survey MeasureTotalistry Sciences science"Math Physics Biostat. 08 SCHOLARLY QUALITY OF PROGRAM FACULTY Distinguished 7.2 6.37.56.5 7.7 7.98.3 Strong 15.9 15.112.519.1 15.5 13.620.3 Good 21.2 22.420.422.8 19.2 19.622.7 Adequate 16.3 19.519.413.4 14.5 14.616.2 Marginal 7.8 10.49.84.7 6.9 6.97.3 Not Suf f ic lent for Doctoral Education 2.2 3.03.0.8 2.5 1.32.7 Don ' t Know Well Enough to Evaluate 29.4 23.327.432.7 33.8 36.122.4 TOTAL 100.0 100.0100.0100.0 100.0 100.0100.0 O 9 EFFECTIVENESS OF PROGRAM IN EDUCATING SCIENTISTS Extremely Effective 8.0 B.77.98.3 7.4 7.87.2 Reasonably Effective 28.7 32.525.734.1 22.1 27.029.0 Minimally Effective 13.2 15.015.712.1 11.3 11.115.1 Not Effective 3.1 3.64.61.7 3.4 2.03.8 Don't Know Well Enough to Evaluate 47.0 40.246.143.8 55.8 52.145.0 TOTAL 100.0 100.0100.0100.0 100.0 100.0100.0 10 CHANGE IN PROGRAM QUALITY IN LAST FIVE YEARS Better 11.5 12.715.714.2 9.3 9.29.2 Little or No Change 29.4 33.925.927.1 25.8 28.432.5 Poorer 6.2 8.48.26.6 3.5 5.15.1 Don't Know Well Enough to Evaluate 52.9 44.950.152.1 61.5 57.353.2 TOTAL 100.0 100.0100.0100.0 100.0 100.0100.0 11 FAMILIARITY WITH WORK OF PROGRAM FACULTY Considerable 20.0 20.920.222.3 17.9 16.324.0 Some 41.1 43.142.840.7 38.8 38.243.6 Little or None 37.2 34.634.635.4 41.8 43.031.1 No Response 1.7 1.42.31.6 1.5 2.51.3 TOTAL 100.0 100.0100.0100.0 100.0 100.0100.0 NOTE: For survey measures 08, 09, and 10 the "don't known category includes a small number of cases for which the respondents provided no response to the survey item.
|
From page 173... ...
As shown in Table 9.7, survey respondents in each discipline were much more likely to furnish evaluations for programs with high reputational standings than they were for programs of lesser distinction. For example, for mathematical and physical science programs that received mean ratings of 4.0 or higher on measure 08, almost 95 percent of the evaluations requested on measure 08 were provided; 85 and 77 percent were provided on measures 09 and 10.
|
From page 174... ...
, and a separate mean rating was computed for each group. The last column in Table 9.8 reports the correlations between the mean program ratings of the two groups and is not corrected for the fact that the mean ratings of each group are based on only half rather than a full set of the responses.9 As the reader will note, the coefficients reported for measure 08, the scholarly quality of program faculty, are in the range of .96 to For a discussion of the interpretation of "split-half" coefficients, see Robert L
|
From page 175... ...
Deviation Correlation ~Group A Group B Group A Group B N r Chemistry 1.05 1.06 .22 .23 145 .76 Computer Sciences 1.14 1.11 .28 .29 57 .82 Geosciences 1.15 1.13 .28 .30 91 .77 Mathematics 1.12 1.14 .22 .22 114 .62 Physics 1.10 1.11 .26 .25 122 .64 Statistics/Biostat. 1.06 1.07 .28 .27 63 .85 MEASURE 11: FAMILIARITY WITH WORK OF PROGRAM FACULTY Discipline Mean Rating Std.
|
From page 176... ...
176 TABLE 9.9 Comparison of Mean Ratings for 11 Mathematics Programs Included in Two Separate Survey Administrations Survey All Evaluators Measure First N ~ X Second N X Evaluators Rating the Same Program in Both Surveys First Second N X N X Program A 08100 4.9114 4.950 4.950 4.9 0990 2.7100 2.842 2.743 2.7 1074 1.283 1.238 I.134 1.2 11100 1.6115 1.650 1.550 1.6 Program 8 0894 4.6115 4.648 4.650 4.5 0981 2.691 2.540 2.639 2.5 1069 1.082 1.037 1.036 0.9 1198 1.4116 1.450 1.550 1.5 Program C 0886 3.4103 3.642 3.444 3.5 0956 2.066 2.128 2.129 2.0 1055 1.162 1.330 1.227 1.4 1199 1.0116 1.150 1.150 1.0 Program D 0874 3.093 3.037 2.838 2.9 0950 1.848 1.627 1.716 1.6 1046 1.452 1.524 1.423 1.5 1190 1.0113 0.946 1.046 0.9 Program E 0869 3.095 3.139 3.046 3.1 0940 1.860 1.925 1.830 1.8 1036 0.858 0.924 0.829 0.9 1196 0.8115 0.952 0.952 1.0 Program F 0863 2.990 3.026 3.032 3.1 0935 1.846 1.710 1.613 1.8 1032 1.143 1.111 1.312 1.2 1195 0.7115 0.843 0.744 0.7 Program G 0869 2.792 2.839 2.739 3.0 0935 1.745 1.617 1.719 1.7 1036 1.143 1.217 1.119 1.2 1185 0.9116 0.846 0.946 0.9 Program H 0858 2.273 2.536 2.237 2.4 0932 1.343 1.322 1.219 1.3 1030 1.539 1.520 1.717 1.4 1190 0.7116 0.651 0.752 0.6 Program I 0855 2.074 1.930 1.930 2.0 0933 1.041 0.919 1.018 0.8 1027 1.231 1.115 1.113 1.2 1199 0.5115 0.550 0.550 0.5 Program J 0851 1.567 1.526 1.428 1.4 0931 0.836 0.714 0.614 0.7 1026 1.223 1.114 1.212 1.3 1196 0.5113 0.349 0.448 0.4 0833 1.248 1.217 1.121 1.4 0919 0.821 0.511 0.68 0.4 1012 0.815 0.95 1.05 0.8 1199 0.2114 0.248 0.247 0.2
|
From page 177... ...
This issue can be addressed in more than one way. It is evident from the data reported in Table 9.10 that mean ratings of the scholarly quality of program faculty tend to be higher if the evaluator has considerable familiarity with the program.
|
From page 178... ...
For example, if we exclude evaluations provided by those who confessed "little or non familiarity with particular programs, then the revised mean ratings would be correlated at a level of at least .99 with the mean ratings computed using all of the data.~° (This similarity arises, in part, because only a small fraction of evaluations are given on the basis of no more than ~little" familiarity with the program.) The third column in Table 9.10 presents the correlation in each discipline between the array of mean ratings supplied by respondents claiming Considerable familiarity and the mean ratings of those indicating isomer or "little or non familiarity with particular programs.
|
From page 179... ...
in the previous five years. Since earlier reputational surveys had not provided such information, 10 percent of the sample members, randomly selected, were given forms without faculty names or doctoral data, as a "control group." Although one might expect that those given faculty names would have been more likely than other survey respondents to provide evaluations of the scholarly quality of program faculty, no appreciable differences were found (Table 9.11)
|
From page 180... ...
There is enough regional traffic in academic life that one might expect proximate programs to be better known than those in distant regions of the country. This hypothesis may apply especially to the smaller and less visible programs and is TABLE 9.12 Mean Ratings of Scholarly Quality of Program Faculty, by Type of Survey Form Provided to Evaluator MEAN RATINGS CORRELATION Names No Names r N Chemistry 2.53 2.66 .93 145 Computer Sciences 2.49 2.61 .93 57 Geosciences 2.93 3.01 .88 90 Mathematics 2.62 2.72 .94 113 Physics 2.62 2.88 .85 122 Statistics/Biostat.
|
From page 181... ...
Furthermore, the high correlations found between the mean ratings of the two groups indicate that the relative standings of programs are not dramatically influenced by the geographic proximity of those evaluating it. Another consideration that troubles some critics is that large programs may be unfairly favored in a faculty survey because they are likely to have more alumni contributing to their ratings who, it would stand to reason, would be generous in the evaluations of their alma 2 See Appendix I for a list of the states included in each region.
|
From page 182... ...
e ~ Novell `~1 ~= l"~=L U`~1C, however, the fraction of alumni providing ratings of a program is always quite small and should have had minimal impact on the overall mean rating of any program. To be certain that this was the case, mean ratings of the scholarly quality of faculty were recalculated for every mathematical and physical science program -- with the evaluations provided by alumni excluded.
|
From page 183... ...
The program ratings TABLE 9.15 Mean Ratings of Scholarly Quality of Program Faculty, by Evaluator's Field of Specialty Within Physics or Statistics/Biostatistics PHYSICS: Group A includes evaluators in elementary particles and nuclear structure; Group B includes those in atomic/ molecular, solid state, and other fields of physics. STATISTICS/BIOSTATISTICS: Group A includes evaluators in biostatistics, biometrics, and epidemiology; Group B includes those in all other fields of statistics.
|
From page 184... ...
This generality has self-evident appeal. On the other hand, it is wise to keep in mind that these reputational ratings are measures of Perceived program quality rather than of "quality" in some ideal or absolute sense.
|
From page 185... ...
At the very least, the survey results provide a snapshot of these impressions from discipline to discipline. Although these impressions may be far from ideally informed, they certainly show a strong degree of consensus within each discipline, and it seems safe to assume that they are more than passingly related to what a majority of keen observers might agree program quality is all about.
|
From page 186... ...
The consequences of these differences in survey coverage are quite apparent: in the committee's survey, evaluations were requested for a total of 593 research-doctorate programs in the mathematical and physical sciences, compared with 444 programs in the Roose-Andersen study. Figures 9.1-9.4 plot the mean ratings of scholarly quality of faculty in programs included in both surveys; sets of ratings are graphed for 103 programs in chemistry, 57 in geosciences, 86 in mathematics, and 90 in physics.
|
From page 187... ...
FIGURE 9.1 Mean rating of scholarly quality of faculty (measure 08) versus mean rating of faculty in the Roose-Andersen study -- 103 programs in chemistry.
|
From page 188... ...
FIGURE 9.2 Mean rating of ~cholarly quality of faculty (measure 08) versus mean rating of faculty in the Roo~e-Andersen study -- 57 programs in geosciences.
|
From page 189... ...
F$GURE 9.3 Mean rating of scholarly quality of faculty (measure 08) versus mean rating of faculty in the Roose-Andersen study -- 86 programs in mathematics.
|
From page 190... ...
FIGURE 9.4 Mean rating of acholarly quality of faculty (measure 08) versus mean rating of faculty in the Roos - Andersen study -- 90 programs in physics.
|
From page 191... ...
is the only one plotted here. Although the results of both surveys are reported on identical scales, some caution must be taken in interpreting differences in the mean ratings a program received in the two evaluations.
|
From page 192... ...
How do program graduates compare with regard to their publication records? Also desired might be measures of the quality of the students applying for admittance to a graduate program (e.g., Graduate Record Examination scores, undergraduate grade point averages)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.