National Academies Press: OpenBook
« Previous: Front Matter
Suggested Citation:"Executive Summary." National Research Council. 2000. Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology. Washington, DC: The National Academies Press. doi: 10.17226/9865.
×

EXECUTIVE SUMMARY

Background

Interest in predicting demand and supply for doctoral scientists and engineers began in the 1950s, and since that time there have been repeated efforts to forecast impending shortages or surpluses. As the importance of science and engineering has increased in relation to the American economy, so has the need for indicators of the adequacy of future demand and supply for scientific and engineering personnel. This need, however, has not been met by databased forecasting models, and accurate forecasts have not been produced.

Forecast error may proceed from many sources. Models may be based on incorrect assumptions about overall structure, included variables, lag structure, and error structure. Data used for estimation may be flawed or aggregated at an inappropriate level. Further, unanticipated events beyond those considered in the model may occur that could ruin the accuracy of even the best forecasts. As Leslie and Oaxaca (1993) have described in their thorough review article, virtually all models of demand and supply have been flawed by at least one (and, in many cases, all) of these problems.

In order to assess the methodology of forecasting the demand and supply of doctoral scientists and engineers, the National Science Foundation (NSF) and Sloan Foundation funded the National Research Council to assemble a committee of experts for a workshop on the topic. The task of the committee was not to find fault with past efforts, but to provide guidance to the NSF and to scholars in this area about how models (and the forecasts derived

Suggested Citation:"Executive Summary." National Research Council. 2000. Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology. Washington, DC: The National Academies Press. doi: 10.17226/9865.
×

from them) might be improved and what role NSF should play in their improvement.

Another issue for the committee was the responsible reporting of forecasts to policymakers. Virtually no forecast is error-free, and some uncertainty is always associated with them. Policymakers who use the results of forecasting are usually not technically proficient in the arcana of the forecaster's art. If forecasts are to be used responsibly, policymakers need to be informed about the assumptions upon which the forecasts rest, and forecasters need to track the validity of their assumptions and the accuracy of their forecasts over time.

The committee then assessed the information on these issues provided at the workshop as well as in the forecasting literature and arrived at the following recommendations.

Recommendations

The forecasting process is ongoing. Forecasters must learn from their mistakes. The whole forecasting exercise needs to be placed within an administrative framework that facilitates an evaluation process and a process to correct errors. The Science Resources Study Division (SRS) of NSF is the locus of action in the federal government to bring about the improvement of data and forecasts of the market for doctoral scientists and engineers. The committee has therefore directed most of its recommendations to SRS, since this division should be able to encourage improvements in the construction and use of forecasting models for highly trained scientists and engineers, even when SRS should not carry out the work itself.

Recommendation 1. The producers of forecasts should take into account the variety of consumers of forecasts of demand and supply for scientists and engineers.

Suggested Citation:"Executive Summary." National Research Council. 2000. Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology. Washington, DC: The National Academies Press. doi: 10.17226/9865.
×

NSF should recognize that there are five distinct communities of clients for data and projections on the supply and demand of scientists and engineers, each with different needs and interests. NSF's data collection and forecasting activities should keep the needs of these different communities in mind. They are:

  1. students making career decisions,

  2. federal, state, and private funding agencies,

  3. industrial and academic employers of scientists,

  4. Congress in its role as funder and policymaker, and

  5. the scientific community that conducts research and produces studies of the market and its participants.

Representatives of all these user communities spoke at the workshop. All expressed dissatisfaction with the current state of data and forecasting.

Students need qualitative projections of likely career outcomes and probabilities of success, with particular attention to the state of the job market in a few years when they will seek employment. These qualitative projections require timely data, but they need not be based on broad surveys or censuses.

Research funding agencies and Congress need relatively long-term projections of supply and demand factors by specific discipline that can be used to guide policy on training support and institutional development. These projections should take careful account of the ease with which one kind of labor can be substituted for another and the incentives and behavioral responses that operate. The importance of contingencies and of forecast uncertainty should be emphasized.

The needs of employers are varied, but they also benefit from an early warning about shortages. As producers of Ph.D.s, academic institutions can take steps to expand or contract Ph.D. enrollment given convincing evidence of emerging labor market trends.

Suggested Citation:"Executive Summary." National Research Council. 2000. Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology. Washington, DC: The National Academies Press. doi: 10.17226/9865.
×

Finally, the forecasting community should be more forthcoming about the appropriate use of forecasts and the nature of their underlying assumptions.

Recommendation 2. The NSF should not produce or sponsor “official” forecasts of supply and demand of scientists and engineers, but should support scholarship to improve the quality of underlying data and methodology.

NSF should not produce or sponsor “official” forecasts of supply and demand in the markets for scientists and engineers, but should continue to take the lead in collecting and making available data on these markets. A clear organizational separation should be made between data collection and modeling/forecasting activities undertaken for NSF's own policy use or for use by federal agencies. For example, convert the SRS into a National Center for Science Statistics on the model of the National Center for Health Statistics (NCHS), the National Center for Educational Statistics (NCES), or the Energy Information Agency (EIA) and remove modeling and forecasting activities to a separate policy unit. Or, for example, forecasts could be produced by an outside agency with statistical expertise. Agencies such as the Bureau of the Census or Bureau of Economic Analysis may be well suited to undertake such forecasts.

If asked to produce forecasts of scientific and engineering personnel for its own use or the use of other agencies, the NSF policy unit should avoid endorsing or emphasizing “gap” models that do not incorporate behavioral adjustment to demand and supply and consequently may give unwary users a misleading impression of likely market outcomes. NSF should avoid suggesting that there is a single best level of detail and model complexity for the forecasts needed by various users and should instead maintain that model structure will depend on user needs and objectives.

The committee reviewed the history of NSF projections, most notably those in The State of Academic Science and Engineering.

Suggested Citation:"Executive Summary." National Research Council. 2000. Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology. Washington, DC: The National Academies Press. doi: 10.17226/9865.
×

In response to the perception that there could be a connection between NSF funding and its projections, especially regarding projections of shortages, the committee believed that NSF should limit itself to data collection and dissemination, and use external “arms-length ” forecasts for its policy needs to avoid the possible conflict of interest that might occur if it produced its own forecasts.

Recommendation 3: Undertake a comprehensive review of data collection in the light of forecasting needs.

NSF's SRS should undertake a comprehensive review of its data management program, preferably in coordination with Bureau of Labor Statistics (BLS), NCHS, and NCES. It should seek the production of more timely and useful data on the market for scientists and engineers. In addition, it should coordinate definitions and categories across agencies to facilitate a consistent picture of the different stages in the market, from student training and degree choice to mid-career transitions across and our of science and engineering fields. Moreover, sample sizes have been reduced since the late 1970s, which makes modeling difficult for small fields, specific employment sectors, and for rare events (such as mid-career changes).

Recommendation 4. Data that enhance forecasts should be widely available and be disseminated on a timely basis.

NSF's SRS should establish three high-priority data objectives. These are: (1) production of timely, descriptive statistics on employment and salaries by field of training, occupation, and sector, in a consistent time-series format that permits tracking and projections of trends; (2) production of an individual-level Public Use Sample, containing a consistent time series of cross-sections of doctoral recipients, and when available, nondoctoral recipients; and (3) production of a Public Use dataset panel of scientists and

Suggested Citation:"Executive Summary." National Research Council. 2000. Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology. Washington, DC: The National Academies Press. doi: 10.17226/9865.
×

engineers to analyze transitions from the educational system to employment and transitions across fields, occupations, and activities. NSF should process these data, establish rules for access so that they are widely available, and institute less burdensome mechanisms to protect confidentiality. For example, in producing a Public Use panel, NSF might recruit respondents who are willing to provide vitae in a standard format without confidentiality restrictions. This format could be made available on the web; to make the process even easier for respondents, NSF could code respondent vitae. This would enhance modeling of individual and institutional behavior in response to changes in funding, demographically driven demand, compensation, etc. Standard coding of vitae would permit collection of more detailed data than the Survey of Doctorate Recipients form.

Both modelers and policy advisors at the workshop complained that data were not timely. At present, there is no way of knowing in a timely way whether market mechanisms are working to alleviate shortages or gluts. Policies calibrated to outdated evidence may provide too little too late, or too much too late. Furthermore, if indicators show that market-clearing mechanisms are working rapidly, the need for policy change might be obviated altogether.

NFS's SRS should collect and disseminate data that enable a variety of forecasting exercises that differ along some or all of the following dimensions:

  1. Unconditional vs. conditional (“What if”) forecasts.

  2. Multiple levels of disaggregation by field, e.g., physical sciences, physics, solid state physics, materials science.

  3. Optional variables to forecast, e.g., jobs, salaries, quality, productivity, occupation.

  4. Various sectors to forecast, e.g., academic tenure track/ other, postdoctoral positions, or industrial sectors.

  5. What to forecast, e.g., stocks, flows, transitions, or careers.

  6. Permit forecasts to go beyond means and standard errors

Suggested Citation:"Executive Summary." National Research Council. 2000. Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology. Washington, DC: The National Academies Press. doi: 10.17226/9865.
×

to more complete descriptions of the distribution of possible outcomes.

  1. Various time horizons for forecasts.

Finally, the NSF's SRS website currently focuses almost exclusively on providing relatively simple tabulations that are useful for casual policy analysis but not very useful for either career planning by students or for research on the science and engineering market. The data management program and website should be redesigned to service these neglected user communities (or in the case of students, the public and private organizations and associations that provide career guidance), and to provide links to other data sources from BLS and NCES that are important for analysis of the markets for scientists and engineers.

Recommendation 5. NSF should develop a research program to improve forecasting.

The Directorate of Social, Behavioral and Economic Sciences of the NSF should commission behavioral studies of scientists and engineers early in their careers, as well as studies of forecasting methods and evaluations of past forecasting exercises. Particular emphasis should be placed on critical parameters of market response, such as wage-sensitivity of field and occupation switching, and the determinants of the ability of employers to restructure research jobs in response to supply conditions in various fields. Emphasis should be placed on the difficult issues of measuring the quality and productivity of scientists and engineers, the quality of worker-to-job matches, and quality of life for scientists and engineers. This work should be conducted through the ordinary peer-reviewed research support process already in place at NSF. SRS should facilitate the dissemination of results of these studies but should stop short of sponsoring or endorsing specific forecasts or methods.

Suggested Citation:"Executive Summary." National Research Council. 2000. Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology. Washington, DC: The National Academies Press. doi: 10.17226/9865.
×
This page in the original is blank.
Suggested Citation:"Executive Summary." National Research Council. 2000. Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology. Washington, DC: The National Academies Press. doi: 10.17226/9865.
×
Page 1
Suggested Citation:"Executive Summary." National Research Council. 2000. Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology. Washington, DC: The National Academies Press. doi: 10.17226/9865.
×
Page 2
Suggested Citation:"Executive Summary." National Research Council. 2000. Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology. Washington, DC: The National Academies Press. doi: 10.17226/9865.
×
Page 3
Suggested Citation:"Executive Summary." National Research Council. 2000. Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology. Washington, DC: The National Academies Press. doi: 10.17226/9865.
×
Page 4
Suggested Citation:"Executive Summary." National Research Council. 2000. Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology. Washington, DC: The National Academies Press. doi: 10.17226/9865.
×
Page 5
Suggested Citation:"Executive Summary." National Research Council. 2000. Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology. Washington, DC: The National Academies Press. doi: 10.17226/9865.
×
Page 6
Suggested Citation:"Executive Summary." National Research Council. 2000. Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology. Washington, DC: The National Academies Press. doi: 10.17226/9865.
×
Page 7
Suggested Citation:"Executive Summary." National Research Council. 2000. Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology. Washington, DC: The National Academies Press. doi: 10.17226/9865.
×
Page 8
Next: 1 Background »
Forecasting Demand and Supply of Doctoral Scientists and Engineers: Report of a Workshop on Methodology Get This Book
×
Buy Paperback | $39.00 Buy Ebook | $31.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This report is the summary of a workshop conducted by the National Research Council in order to learn from both forecast makers and forecast users about improvements that can be made in understanding the markets for doctoral scientists and engineers. The workshop commissioned papers examined (1) the history and problems with models of demand and supply for scientists and engineers, (2) objectives and approaches to forecasting models, (3) margins of adjustment that have been neglected in models, especially substitution and quality, (4) the presentation of uncertainty, and (5) whether these forecasts of supply and demand are worthwhile, given all their shortcomings. The focus of the report was to provide guidance to the NSF and to scholars in this area on how models and the forecasts derived from them might be improved, and what role NSF should play in their improvement. In addition, the report examined issues of reporting forecasts to policymakers.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!