National Academy Press
2101 Constitution Ave., NW Washington, DC 20418
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. It is a result of work done by an independent panel appointed by the Committee on Science, Engineering, and Public Policy, which has authorized its release to the public. The members of the panel responsible for this report were chosen for their special competencies and with regard for appropriate balance.
This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee and by the Committee on Science, Engineering, and Public Policy. Both consist of members of the National Academy of Sciences, National Academy of Engineering, and Institute of Medicine.
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M. Alberts is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences in its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.
The Committee on Science, Engineering, and Public Policy is a joint committee of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. It includes members of the councils of all three bodies.
Sponsor: This study was funded with Federal funds from the National Science Foundation (NSF) under contract number LPA-9123428. The contents of this report do not necessarily reflect the views or policies of the NSF, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.
Library of Congress Catalog Card Number 94-66065
International Standard Book Number 0-309-05029-4
Copyright 1994 by the National Academy of Sciences. All rights reserved.
Available from:
National Academy Press,
2101 Constitution Avenue, N.W., Washington, D.C. 20418
B-274
Printed in the United States of America
PANEL ON NSF DECISIONMAKING FOR MAJOR AWARDS
ROBERT H. RUTFORD (Chair), President,
University of Texas at Dallas, and
Chairman of the Polar Research Board,
National Research Council
CLARENCE R. ALLEN, Professor of Geology and Geophysics, Emeritus,
California Institute of Technology
ALBERT A. BARBER, Special Assistant to the Chancellor,
University of California, Los Angeles—Washington D.C. Office
HARVEY BROOKS, Gordon McKay Professor of Applied Physics, Emeritus, in the Division of Applied Sciences, and Benjamin Peirce Professor of Technology and Public Policy, Emeritus,
John F. Kennedy School of Government, Harvard University
CHRISTOPHER COBURN, Director,
Public Technology Programs, Battelle Memorial Institute
SUSAN E. COZZENS, Associate Professor,
Science and Technology Studies Department, Rensselaer Polytechnic Institute
FRANK D. DRAKE, Professor of Astronomy and Astrophysics,
University of California, Santa Cruz, and
President,
SETI Institute
DONALD S. FREDRICKSON, President,
D.S. Fredrickson Associates, Inc.
FREDRICK S. HUMPHRIES, President,
Florida A&M University
ANITA K. JONES, Chair,
Department of Computer Science, University of Virginia (resigned May 31, 1993, to become Director of Defense Research and Engineering, Department of Defense)
LARRY K. MONTEITH, Chancellor,
North Carolina State University
DOUGLAS D. OSHEROFF, Professor,
Department of Physics, Stanford University
JUDITH A. RAMALEY, President,
Portland State University
LYLE H. SCHWARTZ, Director,
Materials Science and Engineering Laboratory, National Institute of Standards and Technology
Staff
MICHAEL McGEARY, Study Director
ELIZABETH BLOUNT, Senior Project Assistant
COMMITTEE ON SCIENCE, ENGINEERING, AND PUBLIC POLICY
PHILLIP A. GRIFFITHS (Chair), Director,
Institute for Advanced Study
ROBERT McCORMICK ADAMS, Secretary,
Smithsonian Institution
BRUCE M. ALBERTS, President,
National Academy of Sciences
(Ex-Officio)
ELKAN BLOUT, Harkness Professor,
Department of Biological Chemistry and Molecular Pharmacology, Harvard Medical School
FELIX BROWDER,
Department of Mathematics, Rutgers University
ROBERT A. BURT, Alexander M. Bickel Professor of Law,
Yale Law School
DAVID R. CHALLONER, M.D., Vice President of Health Affairs,
University of Florida
ALBERT M. CLOGSTON,* Member,
Center for Material Sciences, Los Alamos National Laboratory
F. ALBERT COTTON, Distinguished Professor of Chemistry; Director,
Laboratory for Molecular Structure and Bonding, Texas A&M University
ALEXANDER H. FLAX, Senior Fellow,
National Academy of Engineering
RALPH E. GOMORY, President,
Alfred P. Sloan Foundation
THOMAS D. LARSON, Consultant
JOHN L. McLUCAS,* Aerospace Consultant
MARY JANE OSBORN,
Department of Microbiology, University of Connecticut Health Center
C. KUMAR N. PATEL, Vice Chancellor,
Research Programs, University of California, Los Angeles
PHILLIP A. SHARP, Head,
Department of Biology, Center for Cancer Research, Massachusetts Institute of Technology
KENNETH I. SHINE, President,
Institute of Medicine
(Ex-Officio)
ROBERT M. SOLOW, Institute Professor,
Department of Economics, Massachusetts Institute of Technology
H. GUYFORD STEVER, Member,
Carnegie Commission on Science and Technology
MORRIS TANENBAUM, Vice President,
National Academy of Engineering
ROBERT M. WHITE, President,
National Academy of Engineering
(Ex-Officio)
SHEILA E. WIDNALL,* Associate Provost and Abby Rockefeller Mauze Professor of Aeronautics and Astronautics,
Massachusetts Institute of Technology
Staff
LAWRENCE E. McCRAY, Executive Director
BARBARA A. CANDLAND, Administrative Coordinator
Preface
Under the guidance of the National Science Board (NSB), the National Science Foundation (NSF) supports science and engineering research and education projects. NSF does not carry out these projects itself. It chooses the best proposals submitted by researchers in universities, colleges, and other research institutions. NSF uses a merit review process to identify the most promising projects to receive funding awards. Merit review has two distinctive features: it relies on independent outside peer reviewers to assess the quality of proposals, and it uses criteria that emphasize technical quality and also promote other goals of the nation’s research base such as equal opportunity, human resource development, and a broader geographic and institutional infrastructure.
Most of the awards made by NSF are to individuals or to small groups of scientists and engineers. This report addresses a small but important set of awards—very large awards for major research facilities, interdisciplinary research centers, and other large-scale research-related activities. Because of their budgetary impact and importance, it is critical that these major projects be carefully chosen on the basis of their contributions to the nation’s research enterprise and not according to political, bureaucratic, or other considerations. To achieve this, major award proposals are subjected to a merit review process. Merit review of major awards is more complicated and sometimes more controversial than that for individual investigator and small group awards.
This report is based on an 18-month study of the NSF-NSB system for making major awards. The study was undertaken by a broad-based expert group, which makes a series of recommendations
for improving the planning, selection, and renewal of such awards. The recommendations appear in the chapters on these topics and are summarized in the executive summary.
The panel would like to thank the individuals who took the time to meet with us and share their knowledge, experiences, and views. Special thanks go to Alan M. Gaines, assistant for science and technology to the director of NSF, and NSF liaison official for this study, who made sure we had full and timely access to the publicly available information needed for the study.
The panel was briefed on the decisionmaking process for major awards at its first meeting by NSF and NSB officials: Frederick M. Bernthal, deputy director (chair, Director's Action Review Board); William C. Harris, assistant director for mathematical and physical sciences; Mary E. Clutter, assistant director for biological sciences; Joseph Kull, chief financial officer (executive secretary, NSB Committee on Programs and Plans); and Marta Cehelsky, NSB executive officer. Then-director Walter E. Massey met with the panel at a later meeting. Warren J. Baker, chair, NSB Committee on Programs and Plans, also briefed the panel on how the NSB reviews major award proposals. Former NSF director John B. Slaughter, who recently chaired the site selection committee for the NSF-supported Laser Interferometer Gravitational Wave Observatory, graciously provided his views on the evolution of the major award review process in an interview with a panel member.
The staff would also like to thank others at NSF who provided information and insight: Robert P. Abel, Charles N. Brownstein, Thomas N. Cooley, Peter W. House, Madeleine E. Hymowitz, James M. McCullough, Lynn Preston, and Joanna E. Rom. Susan E. Fannoney of the NSB staff was especially helpful in locating and providing NSB documents relating to NSB review and approval of 10 case study awards; Florence Heckman, NSF librarian, pointed the way to materials on the history of proposal review at NSF; and George Mazuzan, NSF historian, provided access to the historical files of NSF.
The panel appreciates the efforts of Michael McGeary, the study director, who pulled together a remarkable amount of information on
NSF's merit review policies, procedures, and practices, as well as the 10 case study award decisions, and assisted the panel in drafting the report. Elizabeth Blount, staff associate, took care of the many administrative details of panel meetings and report production with skill, energy, and unfailing good cheer. Jeffrey D. Porro, consultant, edited early drafts of the report. We are also grateful for the support and assistance of the staff of the Committee on Science, Engineering, and Public Policy, including Lawrence E. McCray, executive director, and Barbara Candland, executive assistant, and of Philip M. Smith, executive officer, National Academy of Sciences. Florence Poullin was copy editor. National Academy Press staff who helped turn the report into a book included Stephen Mautner, Dawn Eichenlaub, and James Gormley.
Finally, I would like to thank the panel members for their willingness to devote considerable time to the study and for their contributions to this report. The recommendations reflect their vast experience and wisdom and their desire to give NSF and the NSB constructive advice for better decisionmaking on major awards. Panel member Anita K. Jones was able to participate substantially in the drafting of the report before resigning in May 1993 to become the director of defense research and engineering in the Department of Defense.
Robert H. Rutford
Chairman of the Study Panel