National Academies Press: OpenBook

Experiments in International Benchmarking of US Research Fields (2000)

Chapter: Appendix B: Excerpts from National Academies Reports

« Previous: Appendix A: Committee on Science, Engineering, and Public Policy: Member and Staff Biographical Information
Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

APPENDIX B
EXCERPTS FROM NATIONAL ACADEMIES REPORTS

Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
This page in the original is blank.
Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

APPENDIX B-1
EXCERPT FROM: SCIENCE, TECHNOLOGY AND THE FEDERAL GOVERNMENT: NATIONAL GOALS FOR A NEW ERA

Chapter 3
National Goals for Science

Science and technology are closely linked to important national objectives in such areas as economic growth, health care, national security, and environmental protection. These linkages implicitly raise the question of what goals the nation should have for science and technology and how those goals should be reflected in levels of investment, organizational arrangements, and subjects for research and development. In this chapter, we examine these issues for science; in the next chapter, we do so for technology.

In setting national goals for science, several observations must be kept in mind. The first is that it has proved impossible to predict reliably which areas of science will ultimately contribute to important new technologies. History is rich in examples of scientific research that have led to practical applications in areas far removed from the original work. Fundamental research on electromagnetism contributed directly to the development of modern communications. Investigations in solid-state physics enabled the invention of the transistor. The recombinant DNA technology that led to the biotechnology industry arose from studies of unusual enzymes in bacteria. Mathematics, often regarded as highly abstract, is at the core of applications as diverse as aircraft design, computing, and predictions of climate change.

The second observation involves the importance of research for which applications are not yet known. For investigators who do such fundamental research, primarily in universities, the original motivation

Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

is seldom to develop new applications; rather, it is the desire to discover and to understand natural processes. Nevertheless, this motivation—enlarging the store of human knowledge—in the end brings advances and applications that cannot be made any other way. A substantial redirection of such fundamental research toward goal-directed work would reduce the potential for advances of economic and social importance without necessarily leading to solutions for the problems being addressed.

A third observation concerns the cultural significance of science. The application of modern scientific research, beginning in the seventeenth century, is one of the most profound events in human history. Modern research has done more than change the material circumstances of our lives. It has changed our ideas about ourselves and our place in the universe, about human history and the human future.

A final observation has to do with leadership. The great modern expansion of scientific knowledge has led to changes that have been of enormous benefit to humanity. Adjusting to these changes has in some cases proved disruptive to society. Nonetheless, leadership in science has become one of the defining characteristics of great nations. The United States has risen to a position of global prominence in part through its strengths in science and technology. Those strengths can continue to contribute greatly to U.S. leadership.

In light of the above observations, we believe that the federal government, in partnership with the private sector and with other levels of government, should adopt explicit national goals for science. Our first recommendation is:

The United States should be among the world leaders in all major areas of science.

''Major areas" refers to broad disciplines of science (such as biology, physics, mathematics, chemistry, earth science, and astronomy) and to their major subdisciplines (such as the neurosciences, condensed-matter physics, and seismology). "Among the world leaders" means that the United States should have capabilities and infrastructures of support that are not exceeded elsewhere. Of course, there will be specific areas or skills in which other nations lead the world. But in considering the major subdisciplines in which such areas belong, the United States should meet world standards.

There are several rationales for this goal:

  • Excellence. When U.S. researchers are working at world levels in all disciplines, they can bring the best available knowledge to bear on problems related to national objectives, even if that knowledge appears unexpectedly in a field not traditionally linked to that objective. For

Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

example, by being among the world leaders in the areas of virology, immunology, and molecular biology, U.S. researchers were able quickly to devise a test for AIDS antibodies that helped ensure the safety of the blood supply.

  • Receptiveness. By being among the world leaders in all disciplines, U.S. researchers can quickly recognize, extend, and utilize significant research results that occur elsewhere. For example, high-temperature superconductivity was discovered in Switzerland, but U.S. researchers were able to repeat and extend these findings within a matter of days.

  • Education. Only by working in the presence of world leaders can students in American colleges and universities prepare to become leaders themselves and to extend and apply the frontiers of knowledge.

  • Personnel. Maintaining excellence in a field is the best way to attract the brightest young students to that field and thus ensure its continuing excellence.

In general, being among the leaders in each area of science means that U.S. scientists understand and participate in expanding the frontier of human knowledge. The United States could not have been the early home of the semiconductor industry without having been among the world leaders in solid-state physics. It could not have been the home of the emerging biotechnology industry without having been a world leader in molecular biology.

In addition to being among the world leaders in all areas of science, the United States will wish to excel in certain areas on a national level. Therefore, the committee's second recommendation is:

The United States should maintain clear leadership in some major areas of science.

The rationales for maintaining a clear lead in selected areas of science go beyond those listed above. Among the criteria that would call for clear leadership in a field are the following:

  • The field is demonstrably and tightly coupled to national objectives that can be met only if U.S. research performers are clear leaders. For example, the field of condensed-matter physics drives technological advances in such industrial sectors as microelectronics, advanced materials, and sensors.

  • The field so captures the imagination that it is of broad interest to society. An example in astronomy is the recent detection of differences in the cosmic background radiation related to the creation of the universe.

Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
  • The field affects other areas of science disproportionately and therefore has a multiplicative effect on other scientific advances, especially those where clear leadership is the objective. For example, molecular biology is critical to advances in health care, biotechnology, agriculture, and industrial processes.

The selection of those fields in which the United States wishes to maintain clear leadership will be made by government decisionmakers with appropriate advice from various interested groups. These decisions must be fully informed by the comparative assessments of different scientific fields, discussed below, and by the extent to which different fields meet the criteria for clear leadership. These decisions thus differ in character from decisions about the most promising directions for research within an area of science, which are made most effectively by researchers themselves and should be insulated from the political process.

Implications of The Performance Goals

The federal government needs a better way to gauge the overall health of research—as a whole and in its parts—and to determine whether it is adequately supporting broad national objectives. Such indicators as dollars spent or numbers of scientists supported are, by themselves, inadequate. Nor can such indicators determine the adequacy of overall funding or the appropriate distribution of funds among different fields.

The committee believes that comparative international assessments of scientific accomplishment are a better yardstick for policy decisions. This concept, like others in this report, has been discussed in theory and applied in specific cases, but the committee believes that it now deserves a central place in national considerations of science policy. To this end, we have developed it in much greater detail.

The committee believes that it is feasible to monitor U.S. performance with field-by-field peer assessments. Researchers in many fields have, in recent years, identified research opportunities and even set funding priorities. The processes that they have used could be adapted for more general application.

The committee recommends the establishment of independent panels consisting of researchers who work in a field, individuals who work in closely related fields, and research "users" who follow the field closely. Some of these individuals should be outstanding foreign scientists in the field being examined.

The panels would assess the performance of U.S. research scientists in a given field and compare it with performance of researchers in other nations. To do this, the panels might, for example, judge where the most exciting and promising ideas are emerging, consider where the

Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

best new talent is locating, and examine the comparative capabilities of research facilities or equipment. The panels could identify key factors enhancing—or blocking—performance within different fields and project trends into the future. They would assess both the internal performance of the field and its relationship to other fields of science. Finally, the panels could recommend actions for both the performers and the supporters of research (see box).

Quantitative measurements, such as movements of individuals, literature citation counts, and quantity and quality of instrumentation, would be important tools for the panels. But the most valuable contribution by the panels would be the qualitative judgments of panel members working in each field. Scientists immersed in a particular field are best qualified to appraise the true quality of the work being done, to identify the most promising and exciting advances, and to project the status of the field into the future. The participation of representatives from other fields of science and from the users of research would help to allay concerns that the panels' recommendations are self-serving.

We believe that assessments of fields will prove useful in the alloca-

RESULTS OF THE PANELS' DELIBERATIONS

The independent panels of researchers and research users, drawing on their assessments of the comparative performance of U.S. researchers, could make several kinds of reports to the broader research community and to the federal government. Here are three hypothetical outcomes:

  1. In one area of science, an assessment panel might find that the U.S. research effort is not at world levels of performance. The panel would diagnose the reasons for the deficiency—perhaps inferior facilities or a shortage of qualified young researchers. It could then translate its findings into proposals for the funders, performers, and users of research that would help bring U.S. performance up to world standards.

  2. In another field, an assessment panel might find that U.S. performance is at world levels and does not seem to be in danger of falling behind. In this case, the group could recommend actions that would keep the United States among the world leaders.

  3. In yet another area, an assessment panel might conclude that the United States leads the rest of the world. If that area is not one in which the United States should maintain clear leadership, the panel might recommend reductions in funding, which could then be applied to areas requiring additional support.

Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

tion of resources both within and among fields. Within fields, the assessments will help identify the key factors affecting the comparative performance of its researchers. The assessments will be much more useful than current budgetary criteria for analyzing issues such as the adequacy of the infrastructure and the optimal number of students entering the field. By providing long-term perspectives, the assessments will increase the predictability and stability that are essential to a focused and sustained effort in science.

Assessments of research performance would also help resolve debates over the support of megaprojects. The costs of a megaproject would be assessed in terms of the performance goals. If the area of the megaproject was one in which the United States chose only to be among the leaders, participation would almost certainly depend on international collaboration and cost sharing. For an area where clear leadership is justified, the United States might choose to pursue a megaproject even without international partners. One could envision this process being applied, for instance, to the Human Genome Project, the Superconducting Supercollider, NASA's Mission to Planet Earth, fusion reactors, synchrotron radiation light sources, and so on.

The goals that we recommend also have implications for the research infrastructure. Meeting the goals requires that appropriate elements of the U.S. research infrastructure remain second to none. Educational institutions are essential to this infrastructure; only by providing the finest instruction in mathematics and science can the United States produce world class young scientists and engineers.

The committee believes that these goals can be met within the existing overall federal R&D budget. First, because of its traditionally strong support for science, the United States is already a leader in most areas of science. Second, through the application of the goals outlined above, the federal government is likely to find that the United States has clear leadership in some areas of science in which we need only to be among the leaders; funds can be redistributed accordingly. Third, relatively minor reallocations of the federal government's R&D budget, which now exceeds $70 billion per year, could have a major effect on the research portion of the budget.

A New Framework for Funding

The performance goals stated above would provide the basis for a new approach to designing and enacting federal research budgets.3 Today, the federal R&D budget emerges from a process that is only loosely coordinated. Each federal agency supports research in pursuit of its individual mission, and research is often a relatively small part of that mission.

Guided by the performance goals of being among the world leaders in all areas of science and maintaining a clear lead in some, the Execu-

Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

tive Branch and Congress could take a more coherent approach to setting R&D budgets. In the Executive Branch, the assessments of scientific fields could guide initiatives designed to achieve specific scientific or technological goals. In the past few years, initial steps in this direction have been taken by the Federal Coordinating Council for Science, Engineering, and Technology under the Office of Science and Technology Policy. If a major field of science were found to be behind world standards, the Executive Branch could boost funding across all the agencies that support research in the field.

When the budget reaches Capitol Hill, the House and the Senate would conduct comprehensive reviews of the proposals for science and technology before disaggregating the budget for agency-by-agency examination. Limited versions of such reviews now take place in both houses, but they need to be structured so that their results have more impact on the decisions of individual appropriations subcommittees.

Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

APPENDIX B-2
EXCERPT FROM: ALLOCATING FEDERAL FUNDS FOR SCIENCE AND TECHNOLOGY

RECOMMENDATION 4. The President and Congress should ensure that the FS&T budget is sufficient to allow the United States to achieve preeminence in a select number of fields and to perform at a world-class level in the other major fields.8

The pool of approximately $35 billion to $40 billion in annual public support for FS&T is large and diverse. The committee believes that it is possible within that budget to reduce some programs, eliminate others, increase support of high-opportunity fields, and restrain federal spending—all while maintaining our nation's tradition of excellence in science and technology. To continue as a world leader, the United States should strive for clear leadership in the most promising areas of science and technology and those deemed most important to our national goals. In other major fields, the United States should perform on a par with other nations so that it is "poised to pounce" if future discoveries increase the importance of one of these fields. If the nation sets priorities in this way (see bulleted items below) and uses them in conjunction with the FS&T budget process, the result will be better decisions about reallocating and restructuring the U.S. research and development enterprise, preserving its core strengths, and positioning it well for strong future performance.

The international comparisons needed to assess U.S. achievement of its goals for leadership in research and development should be conducted by panels of the nation's leading experts under White House

Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

Box 1.4 Evaluating FS&T Opportunities and Making International Comparisons: How It Might Work

Every five years, panels are convened to evaluate the fields in each major area of science and technology (e.g., physics, biology, electrical engineering), their standing in the world, and the resources needed to reach and maintain world-class position. Evaluation focuses on outputs, such as important discoveries, and also on certain benchmarks of best practice, such as number of scientists and engineers and their training or the current state of the laboratories and research facilities. To avoid conflicts of interest, at least half of the panel will include a few nonscientists plus experts from fields outside but related to the fields being evaluated. The panel will also include specialists in the evaluated fields who are recruited from the United States and foreign countries. If any field within a major area is performing below world standards but is judged to be a national priority, the panel will recommend that its budget be augmented or other changes made to bring it up to par. At the same time, the panel will identify the other fields with declining scientific opportunities and obsolete federal missions from which resources should be reallocated. Opportunities for international cost-sharing will be examined to achieve optimal use of federal funds devoted to science and technology.

Evaluations will be commissioned by the National Science and Technology Council or its equivalent. The selection of fields for clear U.S. leadership from among those recommended by the panels will be made by the President and presidential advisors as part of the budget process. As an example, an extract of the President's budget message might read: "I propose that the United States need not be so far ahead in experimental particle physics, but should operate at world levels, in this case by contributing to construction of the particle accelerator in Geneva, sponsored by the CERN, and funding the participation of U.S. scientists in its design and research. On the advice of my Council of Advisors on Science and Technology, I propose that the United States should remain clearly preeminent in the molecular biology of plants and animals for the following reasons. . . . Accordingly, I will include the necessary additional funds in the FS&T budgets of the National Institutes of Health, the Department of Agriculture, and the National Science Foundation to achieve this goal. . . ."

auspices. Reallocation decisions should be made with the advice and guidance of these expert panels, capable of determining the appropriate scope of the fields to assess and to judge the international stature of U.S. efforts in each field (see Box 1.4 for a discussion of how international comparisons might work). These panels would recommend to the President, his advisors, and Congress:

Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
  • Which fields must attain or maintain preeminence, based on goals such as economic importance, national security, unusual opportunity for significant discoveries, global resource or environmental issues, control of disease, mitigation of natural disasters, food production, a presidential initiative (such as human spaceflight), or an unanticipated crisis;

  • Which fields require increases in funding, changes in direction, restructuring, or other actions to achieve these goals; and

  • Which fields have excess capacity (e.g., are producing too many new investigators, have more laboratories or facilities than needed) relative to national needs and international benchmarks.

The committee believes that designing the budget process so as to secure an FS&T budget sufficient to ensure preeminence in select fields and world status in others will allow the United States to maintain continued world leadership. The FS&T budget process must be coupled to systematic review of investments by the nation's best scientific and technical experts, reporting to the highest reaches of government, to produce an appropriately balanced mix of activities. The committee emphasizes that wise federal investments will lead to the creation of new wealth in the future to an even greater extent than they have in the past. As a result, these investments will help reduce the federal deficit in the long run. After a period of budget constraints, reconfiguration, and adjustment, national needs may justify increased investments in FS&T.

Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page 37
Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page 38
Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page 39
Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page 40
Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page 41
Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page 42
Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page 43
Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page 44
Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page 45
Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page 46
Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page 47
Suggested Citation:"Appendix B: Excerpts from National Academies Reports." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page 48
Next: Appendix C: Workshop on International Benchmarking of US Research »
Experiments in International Benchmarking of US Research Fields Get This Book
×
Buy Paperback | $87.00 Buy Ebook | $69.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

How can the federal government gauge the overall health of scientific research—as a whole and in its parts—and determine whether national funding adequately supports national research objectives? It is feasible to monitor US performance with field-by-field peer assessments. This might be done through the establishment of independent panels consisting of researchers who work in a field, individuals who work in closely related fields, and research "users" who follow the field closely. Some of these individuals should be outstanding foreign scientists in the field being examined. This technique of comparative international assessments is also known as international benchmarking.

Experiments in International Benchmarking of U.S. Research Fields evaluates the feasibility and utility of the benchmarking technique. In order to do this, the report internationally benchmarks three fields: mathematics, immunology, and materials science and engineering, then summarizes the results of these experiments.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!