Major Award Decisionmaking at the National Science Foundation (1994) / Chapter Skim
Currently Skimming:
1 MAJOR AWARDS AT NSF
1 MAJOR AWARDS AT NSF
Pages 13-38
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 13... ...
that the National Academy of Sciences (NAS) study the criteria weighed by the NSF in making major award decisions and assess the roles in the merit review decisionmaking process of outside scientists and executive agency staff.
|
From page 14... ...
. Even the best merit review decisionmaking process for major awards cannot guarantee a perfect result; the projects are too complex and the knowledge is too imperfect for that.
|
From page 15... ...
. It studied the series of cycles that a major award undergoes the initial decision to launch a major project; the planning and implementation of the review process; the decisionmaking leading to the award; and subsequent decisions to renew, recompute, or terminate a project at the end of each award period focusing on the roles of peer reviewers, staff, outside advisory groups, and the NSB in merit review.
|
From page 16... ...
Most facilities, however, are set up to serve a particular field or subfield.2 The global seismic detector network operated by 2 Many of the facilities have in-house research staff who work part-time on their own research projects that do not go through the same merit review process an outside researcher undergoes to obtain an NSF award for a project using the facility. This opportunity to access unique facilities enables the facilities to recruit high-quality staff who can better assist outside researchers in using the facilities more effectively as well as develop state-of-the-art instrumen
|
From page 17... ...
IRIS, which is constructing and managing the global network of seismometers, has more than 80 member universities and colleges. The Ocean Drilling Program has seven international partners representing IS nations that are also members of an international scientific organization, the Joint Oceanographic Tnstitutions for Deep Earth Sampling, which provides scientific advice and direction to the program.
|
From page 18... ...
ERCs and STCs receive f~ve-year awards, but undergo a full merit review during the third year of each award and either receive a new five-year award or are phased out at the end of the current award period. After ~ ~ years, a center is supposed to be on its own or must reapply de novo in competition win other new proposals.
|
From page 19... ...
Merit review, when used for major awards, often involves more diverse criteria and a more complicated process (ultimately involving the NSB) than when it is used for the typical small research project grant.
|
From page 20... ...
It helps give award decisions credibility with the scientific community, Congress, and We public. The peer review aspects of merit review maximize the role of technical considerations in making awards and help shield the decisionmaking process against internal agency bureaucratic interests and outside political pressures.
|
From page 21... ...
Even if there were no controversial award decisions, a review of NSF's capacity to make wise decisions on major awards would be prudent and timely. The number of major awards and their share of the NSF budget have been increasing in response to several trends.
|
From page 22... ...
The better planned a major project is, and the better planned the review process is, the more likely is the award decision to be understood and supported in the affected communities. The NSB has at least three opportunities to review new largescale activities as they move through the major award process (Table ..6 First, in the case of physical facilities, the NSB approves early in He process a project development plan that establishes the need and technical feasibility of the facility.
|
From page 23... ...
Design Award General Program Approval for Center and Other Nonfacility Awards NSF cannot accept proposals unless the program within which the award will be made has been reviewed and approved by NSB. Approval of Solicitations for Major Awards NSB reviews and approves Requests for Proposals and other solicitations in which the resulting awards are expected to require NSB approval.
|
From page 24... ...
. The responsibility for approving major awards is also closely related to NSB's role in working with the director of NSF in long-range planning, setting program priorities, and developing NSF's annual budget request.
|
From page 25... ...
MAJOR AWARDS AT NSF TABLE 1.2: NSF Research Directorates and Divisions 25 Biological Sciences Directorate Molecular and Cellular Biosciences Integrative Biology and Neuroscience Environmental Biology Biological Instrumentation and Resources Computer and Information Science and Engineering Directorate Computer and Computation Research Information, Robotics and Intelligent Systems Microelectronic Information Processing Systems Advanced Scientific Computing Networking and Communications Research and Infrastructure Cross-Disciplinary Activities Engineering Directorate Biological and Critical Systems Chemical and Thermal Systems Design and Manufacturing Systems Electrical and Communications Systems Engineering Education and Centers Industrial Innovation Interface Mechanical and Structural Systems Geosciences Directorates Atmospheric Sciences Earth Sciences Ocean Sciences Mathematical and Physical Sciences Directorate Mathematical Sciences Astronomical Sciences Physics Chemistry Materials Research Social, Behavioral and Economic Sciences Directorate Social, Behavioral, and Economic Research International Programs Science Resources Studies . a The Office of Polar Programs was recently transferred from the Geosciences Directorate to the Office of the Director.
|
From page 26... ...
In fact, the importance of program directors begins even before the award stage. Because they are closest to research and researchers, program directors often propose new projects and draft Me substance of project development plans and project solicitations that may result in major awards eventually approved by the NSB.
|
From page 27... ...
Selects appropriate individuals to review proposals as individuals or as members of a panel (within NSF policy guidelines and subject to higher-level approval) Conducts panel meetings and/or site visits Conducts technical reviews and analyses Evaluates external reviews Negotiates proposal budgets and work plans Maintains liaison and coordination with other federal agencies in connection with duplicate proposals or joint funding of proposals Negotiates revised proposal budgets Conducts final review of proposals and evaluations, and recommends acceptance or declination Prepares documentation of review and decision processes Forms proposer about results of review and decision processes Program Management Interacts with the Division of Grants and Awards (DGA)
|
From page 28... ...
Advises prospective awardees and institutional representatives about NSF objectives, policies, and practices or refers them to DGA Serves as primary consultant within NSF on technical matters in his or her area of competence On request, coordinates and advises on aspects of his or her program that involve other facets of society such as national resources, technological assessment, and social and cultural organization Exchanges program information with other agencies and institutions Represents NSF at professional meetings and seminars Administration Formulates plans, supervises program staff, and assigns and reviews work Prepares reports Fulfills internal budget and operating requirements Performs staff work for, and participates in, program review and evaluation activities Recommends new, and improves existing, procedures for program management procedures NOTE: Many of the functions listed are not the final responsibility of the program director but involve section heads, divisions directors, assistant directors, and other parts of NSF.
|
From page 29... ...
Two trends could have a negative impact on the ability of NSF program directors to manage the merit review process for all awards, including major awards. First, the number of proposals has increased much faster than the number of staff.
|
From page 30... ...
Program directors for major awards may handle only a few awards. The large national facilities for astronomy and atmospheric research, for example, which come up for noncompetitive renewal every five years, have several program directors managing their awards.
|
From page 31... ...
The pane! observed that merit review procedures and criteria governing major awards have been adapted from those originally used for awards to individual investigators.
|
From page 32... ...
So far, the success of the merit review system has helped insulate NSF and NSB decisionmaking on major awards from congressional intervention. If confidence in the system is not maintained, the temptation for research institutions to try to have Congress preempt NSF decisionmaking will increase, and to the extent that legislative involvement replaces merit review with political considerations in project selection, the quality of Me nation's research system may be negatively affected.
|
From page 33... ...
Although too much detail in specifying criteria might limit the flexibility to respond to innovative proposals, we concluded that to increase procedural fairness, NSB and NSF should be more precise about Me criteria and review process to be used. In particular, the primary technical criteria as distinct from the other criteria to be considered in the merit review process should be identified in advance in each case.
|
From page 34... ...
Appropriate Roles of Peer Reviewers and Staff The review process must be structured so that the roles of peer reviewers and staff in evaluating and recommending proposals are clearly understood, and trade-offs among technical and other criteria are clearly explained, at each subsequent level of decisionmaking. Currently, the summary rating and ranking of proposals by staff at various decision points does not always distinguish peer review from staff judgments.
|
From page 35... ...
Such memoranda would increase public understanding of major award decisions and therefore enhance public confidence in the system that produces them. More Stringent Setting of Priorities Decisions to solicit proposals for very large major awards should {eke into account their impact on NSF's overall program as well as on the particular research field involved, and they should be contingent on the realization of expected funds and technological progress.
|
From page 36... ...
The project development plan requirement that exists for large-scale facility construction should be extended to all major award projects. NSF has recently augmented its procedures for planning Me review process prior to soliciting proposals (Chapter 41; the next step is to ensure that the key features of the review plan are published explicitly in the proposal solicitation document.
|
From page 37... ...
In following the traditions of peer review of individual investigator research projects, the results of peer reviews and rationale for the award decision have not been made public (Chapter 4~. We believe that such public explanation is justified by the scientific importance, budgetary size, and public visibility of major awards and would increase support for the decisionmaking process.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.