Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 37
Experiments in International Benchmarking of US Research Fields APPENDIX B EXCERPTS FROM NATIONAL ACADEMIES REPORTS
OCR for page 38
Experiments in International Benchmarking of US Research Fields This page in the original is blank.
OCR for page 39
Experiments in International Benchmarking of US Research Fields APPENDIX B-1 EXCERPT FROM: SCIENCE, TECHNOLOGY AND THE FEDERAL GOVERNMENT: NATIONAL GOALS FOR A NEW ERA Chapter 3 National Goals for Science Science and technology are closely linked to important national objectives in such areas as economic growth, health care, national security, and environmental protection. These linkages implicitly raise the question of what goals the nation should have for science and technology and how those goals should be reflected in levels of investment, organizational arrangements, and subjects for research and development. In this chapter, we examine these issues for science; in the next chapter, we do so for technology. In setting national goals for science, several observations must be kept in mind. The first is that it has proved impossible to predict reliably which areas of science will ultimately contribute to important new technologies. History is rich in examples of scientific research that have led to practical applications in areas far removed from the original work. Fundamental research on electromagnetism contributed directly to the development of modern communications. Investigations in solid-state physics enabled the invention of the transistor. The recombinant DNA technology that led to the biotechnology industry arose from studies of unusual enzymes in bacteria. Mathematics, often regarded as highly abstract, is at the core of applications as diverse as aircraft design, computing, and predictions of climate change. The second observation involves the importance of research for which applications are not yet known. For investigators who do such fundamental research, primarily in universities, the original motivation
OCR for page 40
Experiments in International Benchmarking of US Research Fields is seldom to develop new applications; rather, it is the desire to discover and to understand natural processes. Nevertheless, this motivation—enlarging the store of human knowledge—in the end brings advances and applications that cannot be made any other way. A substantial redirection of such fundamental research toward goal-directed work would reduce the potential for advances of economic and social importance without necessarily leading to solutions for the problems being addressed. A third observation concerns the cultural significance of science. The application of modern scientific research, beginning in the seventeenth century, is one of the most profound events in human history. Modern research has done more than change the material circumstances of our lives. It has changed our ideas about ourselves and our place in the universe, about human history and the human future. A final observation has to do with leadership. The great modern expansion of scientific knowledge has led to changes that have been of enormous benefit to humanity. Adjusting to these changes has in some cases proved disruptive to society. Nonetheless, leadership in science has become one of the defining characteristics of great nations. The United States has risen to a position of global prominence in part through its strengths in science and technology. Those strengths can continue to contribute greatly to U.S. leadership. In light of the above observations, we believe that the federal government, in partnership with the private sector and with other levels of government, should adopt explicit national goals for science. Our first recommendation is: The United States should be among the world leaders in all major areas of science. ''Major areas" refers to broad disciplines of science (such as biology, physics, mathematics, chemistry, earth science, and astronomy) and to their major subdisciplines (such as the neurosciences, condensed-matter physics, and seismology). "Among the world leaders" means that the United States should have capabilities and infrastructures of support that are not exceeded elsewhere. Of course, there will be specific areas or skills in which other nations lead the world. But in considering the major subdisciplines in which such areas belong, the United States should meet world standards. There are several rationales for this goal: Excellence. When U.S. researchers are working at world levels in all disciplines, they can bring the best available knowledge to bear on problems related to national objectives, even if that knowledge appears unexpectedly in a field not traditionally linked to that objective. For
OCR for page 41
Experiments in International Benchmarking of US Research Fields example, by being among the world leaders in the areas of virology, immunology, and molecular biology, U.S. researchers were able quickly to devise a test for AIDS antibodies that helped ensure the safety of the blood supply. Receptiveness. By being among the world leaders in all disciplines, U.S. researchers can quickly recognize, extend, and utilize significant research results that occur elsewhere. For example, high-temperature superconductivity was discovered in Switzerland, but U.S. researchers were able to repeat and extend these findings within a matter of days. Education. Only by working in the presence of world leaders can students in American colleges and universities prepare to become leaders themselves and to extend and apply the frontiers of knowledge. Personnel. Maintaining excellence in a field is the best way to attract the brightest young students to that field and thus ensure its continuing excellence. In general, being among the leaders in each area of science means that U.S. scientists understand and participate in expanding the frontier of human knowledge. The United States could not have been the early home of the semiconductor industry without having been among the world leaders in solid-state physics. It could not have been the home of the emerging biotechnology industry without having been a world leader in molecular biology. In addition to being among the world leaders in all areas of science, the United States will wish to excel in certain areas on a national level. Therefore, the committee's second recommendation is: The United States should maintain clear leadership in some major areas of science. The rationales for maintaining a clear lead in selected areas of science go beyond those listed above. Among the criteria that would call for clear leadership in a field are the following: The field is demonstrably and tightly coupled to national objectives that can be met only if U.S. research performers are clear leaders. For example, the field of condensed-matter physics drives technological advances in such industrial sectors as microelectronics, advanced materials, and sensors. The field so captures the imagination that it is of broad interest to society. An example in astronomy is the recent detection of differences in the cosmic background radiation related to the creation of the universe.
OCR for page 42
Experiments in International Benchmarking of US Research Fields The field affects other areas of science disproportionately and therefore has a multiplicative effect on other scientific advances, especially those where clear leadership is the objective. For example, molecular biology is critical to advances in health care, biotechnology, agriculture, and industrial processes. The selection of those fields in which the United States wishes to maintain clear leadership will be made by government decisionmakers with appropriate advice from various interested groups. These decisions must be fully informed by the comparative assessments of different scientific fields, discussed below, and by the extent to which different fields meet the criteria for clear leadership. These decisions thus differ in character from decisions about the most promising directions for research within an area of science, which are made most effectively by researchers themselves and should be insulated from the political process. Implications of The Performance Goals The federal government needs a better way to gauge the overall health of research—as a whole and in its parts—and to determine whether it is adequately supporting broad national objectives. Such indicators as dollars spent or numbers of scientists supported are, by themselves, inadequate. Nor can such indicators determine the adequacy of overall funding or the appropriate distribution of funds among different fields. The committee believes that comparative international assessments of scientific accomplishment are a better yardstick for policy decisions. This concept, like others in this report, has been discussed in theory and applied in specific cases, but the committee believes that it now deserves a central place in national considerations of science policy. To this end, we have developed it in much greater detail. The committee believes that it is feasible to monitor U.S. performance with field-by-field peer assessments. Researchers in many fields have, in recent years, identified research opportunities and even set funding priorities. The processes that they have used could be adapted for more general application. The committee recommends the establishment of independent panels consisting of researchers who work in a field, individuals who work in closely related fields, and research "users" who follow the field closely. Some of these individuals should be outstanding foreign scientists in the field being examined. The panels would assess the performance of U.S. research scientists in a given field and compare it with performance of researchers in other nations. To do this, the panels might, for example, judge where the most exciting and promising ideas are emerging, consider where the
OCR for page 43
Experiments in International Benchmarking of US Research Fields best new talent is locating, and examine the comparative capabilities of research facilities or equipment. The panels could identify key factors enhancing—or blocking—performance within different fields and project trends into the future. They would assess both the internal performance of the field and its relationship to other fields of science. Finally, the panels could recommend actions for both the performers and the supporters of research (see box). Quantitative measurements, such as movements of individuals, literature citation counts, and quantity and quality of instrumentation, would be important tools for the panels. But the most valuable contribution by the panels would be the qualitative judgments of panel members working in each field. Scientists immersed in a particular field are best qualified to appraise the true quality of the work being done, to identify the most promising and exciting advances, and to project the status of the field into the future. The participation of representatives from other fields of science and from the users of research would help to allay concerns that the panels' recommendations are self-serving. We believe that assessments of fields will prove useful in the alloca- RESULTS OF THE PANELS' DELIBERATIONS The independent panels of researchers and research users, drawing on their assessments of the comparative performance of U.S. researchers, could make several kinds of reports to the broader research community and to the federal government. Here are three hypothetical outcomes: In one area of science, an assessment panel might find that the U.S. research effort is not at world levels of performance. The panel would diagnose the reasons for the deficiency—perhaps inferior facilities or a shortage of qualified young researchers. It could then translate its findings into proposals for the funders, performers, and users of research that would help bring U.S. performance up to world standards. In another field, an assessment panel might find that U.S. performance is at world levels and does not seem to be in danger of falling behind. In this case, the group could recommend actions that would keep the United States among the world leaders. In yet another area, an assessment panel might conclude that the United States leads the rest of the world. If that area is not one in which the United States should maintain clear leadership, the panel might recommend reductions in funding, which could then be applied to areas requiring additional support.
OCR for page 44
Experiments in International Benchmarking of US Research Fields tion of resources both within and among fields. Within fields, the assessments will help identify the key factors affecting the comparative performance of its researchers. The assessments will be much more useful than current budgetary criteria for analyzing issues such as the adequacy of the infrastructure and the optimal number of students entering the field. By providing long-term perspectives, the assessments will increase the predictability and stability that are essential to a focused and sustained effort in science. Assessments of research performance would also help resolve debates over the support of megaprojects. The costs of a megaproject would be assessed in terms of the performance goals. If the area of the megaproject was one in which the United States chose only to be among the leaders, participation would almost certainly depend on international collaboration and cost sharing. For an area where clear leadership is justified, the United States might choose to pursue a megaproject even without international partners. One could envision this process being applied, for instance, to the Human Genome Project, the Superconducting Supercollider, NASA's Mission to Planet Earth, fusion reactors, synchrotron radiation light sources, and so on. The goals that we recommend also have implications for the research infrastructure. Meeting the goals requires that appropriate elements of the U.S. research infrastructure remain second to none. Educational institutions are essential to this infrastructure; only by providing the finest instruction in mathematics and science can the United States produce world class young scientists and engineers. The committee believes that these goals can be met within the existing overall federal R&D budget. First, because of its traditionally strong support for science, the United States is already a leader in most areas of science. Second, through the application of the goals outlined above, the federal government is likely to find that the United States has clear leadership in some areas of science in which we need only to be among the leaders; funds can be redistributed accordingly. Third, relatively minor reallocations of the federal government's R&D budget, which now exceeds $70 billion per year, could have a major effect on the research portion of the budget. A New Framework for Funding The performance goals stated above would provide the basis for a new approach to designing and enacting federal research budgets.3 Today, the federal R&D budget emerges from a process that is only loosely coordinated. Each federal agency supports research in pursuit of its individual mission, and research is often a relatively small part of that mission. Guided by the performance goals of being among the world leaders in all areas of science and maintaining a clear lead in some, the Execu-
OCR for page 45
Experiments in International Benchmarking of US Research Fields tive Branch and Congress could take a more coherent approach to setting R&D budgets. In the Executive Branch, the assessments of scientific fields could guide initiatives designed to achieve specific scientific or technological goals. In the past few years, initial steps in this direction have been taken by the Federal Coordinating Council for Science, Engineering, and Technology under the Office of Science and Technology Policy. If a major field of science were found to be behind world standards, the Executive Branch could boost funding across all the agencies that support research in the field. When the budget reaches Capitol Hill, the House and the Senate would conduct comprehensive reviews of the proposals for science and technology before disaggregating the budget for agency-by-agency examination. Limited versions of such reviews now take place in both houses, but they need to be structured so that their results have more impact on the decisions of individual appropriations subcommittees.
OCR for page 46
Experiments in International Benchmarking of US Research Fields APPENDIX B-2 EXCERPT FROM: ALLOCATING FEDERAL FUNDS FOR SCIENCE AND TECHNOLOGY RECOMMENDATION 4. The President and Congress should ensure that the FS&T budget is sufficient to allow the United States to achieve preeminence in a select number of fields and to perform at a world-class level in the other major fields.8 The pool of approximately $35 billion to $40 billion in annual public support for FS&T is large and diverse. The committee believes that it is possible within that budget to reduce some programs, eliminate others, increase support of high-opportunity fields, and restrain federal spending—all while maintaining our nation's tradition of excellence in science and technology. To continue as a world leader, the United States should strive for clear leadership in the most promising areas of science and technology and those deemed most important to our national goals. In other major fields, the United States should perform on a par with other nations so that it is "poised to pounce" if future discoveries increase the importance of one of these fields. If the nation sets priorities in this way (see bulleted items below) and uses them in conjunction with the FS&T budget process, the result will be better decisions about reallocating and restructuring the U.S. research and development enterprise, preserving its core strengths, and positioning it well for strong future performance. The international comparisons needed to assess U.S. achievement of its goals for leadership in research and development should be conducted by panels of the nation's leading experts under White House
OCR for page 47
Experiments in International Benchmarking of US Research Fields Box 1.4 Evaluating FS&T Opportunities and Making International Comparisons: How It Might Work Every five years, panels are convened to evaluate the fields in each major area of science and technology (e.g., physics, biology, electrical engineering), their standing in the world, and the resources needed to reach and maintain world-class position. Evaluation focuses on outputs, such as important discoveries, and also on certain benchmarks of best practice, such as number of scientists and engineers and their training or the current state of the laboratories and research facilities. To avoid conflicts of interest, at least half of the panel will include a few nonscientists plus experts from fields outside but related to the fields being evaluated. The panel will also include specialists in the evaluated fields who are recruited from the United States and foreign countries. If any field within a major area is performing below world standards but is judged to be a national priority, the panel will recommend that its budget be augmented or other changes made to bring it up to par. At the same time, the panel will identify the other fields with declining scientific opportunities and obsolete federal missions from which resources should be reallocated. Opportunities for international cost-sharing will be examined to achieve optimal use of federal funds devoted to science and technology. Evaluations will be commissioned by the National Science and Technology Council or its equivalent. The selection of fields for clear U.S. leadership from among those recommended by the panels will be made by the President and presidential advisors as part of the budget process. As an example, an extract of the President's budget message might read: "I propose that the United States need not be so far ahead in experimental particle physics, but should operate at world levels, in this case by contributing to construction of the particle accelerator in Geneva, sponsored by the CERN, and funding the participation of U.S. scientists in its design and research. On the advice of my Council of Advisors on Science and Technology, I propose that the United States should remain clearly preeminent in the molecular biology of plants and animals for the following reasons. . . . Accordingly, I will include the necessary additional funds in the FS&T budgets of the National Institutes of Health, the Department of Agriculture, and the National Science Foundation to achieve this goal. . . ." auspices. Reallocation decisions should be made with the advice and guidance of these expert panels, capable of determining the appropriate scope of the fields to assess and to judge the international stature of U.S. efforts in each field (see Box 1.4 for a discussion of how international comparisons might work). These panels would recommend to the President, his advisors, and Congress:
OCR for page 48
Experiments in International Benchmarking of US Research Fields Which fields must attain or maintain preeminence, based on goals such as economic importance, national security, unusual opportunity for significant discoveries, global resource or environmental issues, control of disease, mitigation of natural disasters, food production, a presidential initiative (such as human spaceflight), or an unanticipated crisis; Which fields require increases in funding, changes in direction, restructuring, or other actions to achieve these goals; and Which fields have excess capacity (e.g., are producing too many new investigators, have more laboratories or facilities than needed) relative to national needs and international benchmarks. The committee believes that designing the budget process so as to secure an FS&T budget sufficient to ensure preeminence in select fields and world status in others will allow the United States to maintain continued world leadership. The FS&T budget process must be coupled to systematic review of investments by the nation's best scientific and technical experts, reporting to the highest reaches of government, to produce an appropriately balanced mix of activities. The committee emphasizes that wise federal investments will lead to the creation of new wealth in the future to an even greater extent than they have in the past. As a result, these investments will help reduce the federal deficit in the long run. After a period of budget constraints, reconfiguration, and adjustment, national needs may justify increased investments in FS&T.
Representative terms from entire chapter: