Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
6 Institutional and Administrative Issues Effective management of the expanded competi- tive grants program will require careful attention to and management of a number of institutional and administrative issues. First, the program office must be properly located within the structure of the U.S. Department of Agriculture (USDA). Second, several program transitions must be arranged: program plan- ning and advisory committees must be set up, the peer review process must be managed and its quality en- sured, and the program ' s administrative capacity must be expanded to match the increase in program scope and number of grants. Third, success of the multidis- ciplinary grants must be ensured. Finally, program evaluation and accountability are essential to the program. PROGRAM'S LOCATION IN USDA In fiscal year (FY) 1988, the USDA competitive grants program (funded at $45.4 million) represented less than 5 percent of USDA's Agricultural Research Service (ARS) and Cooperative State Research Ser- vice (CSRS) research expenditures. The program is administered within the CSRS, which is one of three major science and education agencies within USDA's Office of Science and Education; the other two are the ARS and the Extension Service (ES). Within CSRS, the Competitive Research Grants Office (CRGO) is one of five offices reporting to an associate adminis- trator. (CSRS has two programmatic associateadmin- istrators and three deputy administrators responsible for scientific direction and management.) As the competitive grants program reaches $550 million in annual expenditures, its size and scope will clearly warrant its elevation within USDA's Office of 87 Science and Education. Various institutional options are likely to be considered; all of them should be evaluated in terms of the following criteria: · Ensuring the program 's openness to high-quality science and providing it with broad appeal, visibility, and stature within the scientific community. Providing the CRGO program director and chief scientists with direct access to key policymakers within USDA, particularly the assistant secretary for science and education. · Developing strong relations between the com- petitive grants program and the research programs of other federal agencies. · Attracting nationally prominent scientists and managers to positions of program leadership in CRGO and to service on program advisory committees and peer review panels. Three of the more likely options are discussed here. Option I: Major Unit within the Office of Science and Education Under this option, the expanded CRGO would be a fourth major science and education agency within USDA's Office of Science and Education; thus, it would be taken out of CSRS and elevated within the Office of Science and Education. Its administrator would be on an equal footing with the administrators of ARS, CSRS, and ES as the critical policymakers and line managers of USDA's science, education, and . . . . . tralmng actlvltles. This option responds well to the criteria set forth above and has the following advantages:
88 A clear signal would be sent to the scientific and engineering communities that USDA is committed to the competitive grants program. · The leader of the competitive grants program would report directly to the assistant secretary for science and education and would have policy status within the department comparable to the status ac- corded to the heads of comparable research agen- cies for example, the director of the National Insti- tutes of Health ~H) within the U.S. Department of Health and Human Services and the assistant directors of the National Science Foundation (NSF), whoarere- sponsible for each of the major units. The leader of the competitive grants program would, as noted, also have policymaking status com- parable to that of the heads of the other units within USDA's Office of Science and Education (ARS, ES, and CSRS), whose budgets are generally comparable . . In size. · Other USDA agencies, other federal science agencies, and private and public universities would have fair and equitable access to the program. Under this option, probably only straightforward institutional changes would be necessary and proce- dural continuity would be ensured. Option II: Retention within the Cooperative State Research Service The increased expenditures for competitive grants could continue to be administered by CRGO within CSRS. New professional and support staff positions could be established as needed. This approach would · Avoid any need for institutional or legislative changes and ensure procedural continuity. · Minimize the need to consider other organiza- tional and institutional issues. Yet this approach has several disadvantages. It would · Fail to give the competitive "rants program greater visibility and stature and foreclose options in strength- ening it administratively. · Lodge decisionmaking authority and budget advocacy for the competitive grants program at an inappropriately low level within USDA's Office of Science and Education. INVESTING IN RESE - CH Complicate the interactions between CRGO pro- gram scientists and the scientists of other agencies, both within USDA and across the government. · Be less likely to attract top scientists and admin- istrators to positions of program leadership and ser- v~ce. Option III: Creation of a Separate Institute A separate institute within ARS could be similar to those of NIH. Some science policy analysts andpoliti- cal leaders have suggested that NIH could be a model for expanding the scope and improving the quality and responsiveness of USDA's scientific programs. Most NIH research institutes have both intra- and extramu- ral programs. The crux of this option, as it has been suggested, involves the transformation of ARS into the intramural unit of such an institute and removal of the competitive grants program from CSRS. This would lodge it in the institute as the institute's extra- mural arm. In addition, both the intra- and extramural components of ARS's ongoing research programs would be changed in other ways to strengthen the quality and importance of scientific input in setting priorities and adjusting budgets. The agency's pro- gram planning and peer review procedures would also be changed to more closely match those used by NIH. The advantages of this option are that it would · Send a strong signal to the scientific community that a major change is under way in the organization and funding of major USDA-supported federal re- search and development programs, particularly the competitive grants program and ARS. · Provide a mandate to USDA administrators to follow the proven NIH model. Strengthen ARS as an agency by bringing its administrators and scientists into more direct and fre- quent contact with colleagues in the academic com- munity and the private sector. This option has disadvantages, however · It would require substantial legislative change that, in tom, would require a political consensus that would probably prove elusive. · The traditional balance and relationships among ARS, CSRS, and ES would most likely change signifi- cantly, since ARS would be markedly strengthened at the expense of CSRS and at some cost, too, to ES.
INSTITUTIONAL AND ADMINISTRATIVE ISSUES · New responsibility would be placed on the ad- ministrators of ARS. After considering these three options, the Board on Agriculture believes the expanded competitive grants program should be a new, fourth unit in USDA's Office of Science and Education, coequal with ARS, ES, and CSRS. PROGRAM TRANSITIONS No matter which organizational option is selected, the staff of the competitive grants program will need to secure advice and guidance in defining the needs of the program areas and the scientific and technological opportunities and areas to be emphasized. An addi- tional challenge-one that is important in all competi- tive grants programs is to determine the appropriate composition of peer review panels. Administrative questions will also arise as the program's funding increases by $500 million annually. Program Planning and Advisory Committees Advice and guidance will be needed at inception and throughout the program-on a number of key issues, including (1) defining the mission, objectives, and short- and longer-term priorities for each program area; (2) identifying priorities across and among the program areas; (3) ensuring that the areas and quality of science and technology are appropriate; (4) evalu- ating the results of the research in relation to the mission of the program areas; (5) giving special atten- tion to the results from and value of the multidiscipli- nary team and research-strengthening grants in each program area; and (6) evaluating the overall effective- ness of the program. An important mechanism for providing the staff with guidance in defining opportunities is an advisory committee for each of the six program areas. Each advisory committee would be composed of scientists drawn from the range of disciplines critical to advanc- ing science and technology within that particular program area. In addition, individuals from outside the scientific community who have special expertises and perspectives relevant to the program area should also be committee members. Examples of such indi- viduals are producers; processors; leaders from the social, consumer, and environmental sectors; govern- ment leaders and policy experts; and leaders from business and industry. The help of people from 89 outside the scientific community is particularly im- portant in evaluating the relation between the program conducted and the mission of the program area. Ideally, the advisory committees would include both public end private sector scientists. Participation by private sector scientists on such advisory commit- tees is highly desirable and can lead to valuable exchanges of views on the evolving character of practical problems, on the promise of new science and technology, and on ensuring linkages between public and private sector scientists; between science and technology and their further development, innova- tion, and application; and between science opportuni- ties and the needs of the program area. Committee members would also include some scientists with basic research experience, some with applied research experience, and some with multidis- ciplinary research experience. And some members would be experienced in dealing with the market, policy, and institutional forces that shape the relation- ship between science and society. The disciplinary composition of the program plan- ning advisory committees could evolve over time, corresponding to changes in science as well as to changes in economic, social, or regulatory concerns. The six advisory committees would provide the six major program areas with the same kind of overall guidance that the Joint Council for Food and Agricul- tural Sciences and the Users Advisory Board provide to the USDA Of lice of Science and Education. In form and function, the advisory committees could be mod- eled on similar program planning committees used by NIH and NSF (see the box "Program Planning at NIH and NSF'). The Peer Review Process Ensunng the proper composition and functioning of peer review panels is another important ongoing administrative challenge. This issue is especially critical because people with different backgrounds have different useful views on alternative research strategies. Panels must include people who, collec- tively, have the capacity to judge the quality of propos- als and to recognize the most promising opportunities to advance science and technology and solve prob- lems. Drawing panel members from throughout the scientific community is important to ensure thatprob- lems are approached with the most promising and creative strategies, even if they are less proven, and not just with traditional strategies.
go INVESTING IN RESEARCH Program Planning at NIH and NSF Competitive grants programs administered by the NIH, NSF, and USDA all USE program planning advi- sory councils or committees to help in the identification of scientific priorities. (Within all three agencies it is the peer review panels in each scientific area that provide the scientific evaluation for awarding grants.) In each agency, the process of determining the research emphasis has formal and informal components, both of which involve a series of interactions among "bench" scientists, senior program staff, and appointed advisory groups. The informal, or consensus-building, component centers on the peer review process used in evaluating grant proposals. That process provides an ongoing, effective way of monitoring new advances and opportunities in science, because ideas on programs and policy issues often surface from discussions among program staff and scientists during the peer review meetings. For example, the concept of initiating a special effort to map the human genome was raised several times in informal discussions by peer review panels before advisory groups were convened to debate the scientific and policy aspects of instituting such a program. The formal component of determining the research emphasis is special to each institution. At NIH, each institute has an advisory council to review and take action on program and policy. The advisory councils are composed not only of outstanding scientists but also of members of the public with demonstrated interests in the health program areas of the particular institute. (Each institute has a specific mission within the field of human health, with the exception of the National Institute of General Medical Sciences, whose mission is to conduct and support research in the basic medical sciences of significance to two or more institutes, or in research areas that fall outside the general area of responsibility of any other institute.) The knowledge base the advisory councils draw upon when making major decisions about program direction and policy is based on the two levels of review that competitive grants proposals at NIH proceed through. The first level is strictly scientific- a review of the scientific merit of the proposa~and is carried out for all institutes by the Division of Research Grants. The second level of review combines the scientific evaluation from the first-level review with an assessment of the relevance of the proposed research to the mission of the institute. This review is performed by advisory councils and their grant review committees. NSF has a broad and general charge to promote the progress of science, in contrast to other government agencies that support research targeted at more specific missions. Within NSF, changes in programs and funding initiatives are directed more by scientific opportunities and the general need for skilled human resources than by needs arising from any specific public mission. The National Science Board (NSB), whose members represent all areas of science and are from research institutes, universities, and industry, advises the director of NSF on the structuring of programs, budget priorities, and other key initiatives. The NSB is also required to take action on all grant awards that exceed $6 million. In developing annual program announcements, each NSF directorate takes NSB guidance about priori- ties into account. These announcements encourage investigators to submit proposals in certain areas. USDA has also established a competitive grants program advisory comm ttee. Its purpose and activities are still evolving, and USDA is continuing to try to put into place more effective ways of using the insights and skills of the committee's members in identifying and acting upon program priorities. . Efforts must be made to broaden the expertise represented on review panels so that the panels can fully evaluate the quality and relevance of proposed research. In addition to a broad representation of experts from different disciplines, panels should in- clude people from different levels of the research process to help judge the relevance of the proposed research. For example, reviewers of fundamental research proposals should include representatives with backgrounds in applied research, and reviewers of applied research proposals should include individuals with backgrounds in fundamental research. A reliable way of ensuring that peer review panels are not limited in their vision of science and technol- ogy opportunity is to have a varied group of scientists serve on panels on arotatingbasis. When themember- ship of a panel rotates regularly and is made up by individuals from a range of disciplines, from a variety of institutional affiliations, and with a breadth of re- search experiences, there will be greater recognition and support of creative approaches. Each major program area likely will need several peer review panels to review proposals, and more than one panel may need to review some proposals. As a
INSTITUTIONAL AND ADMINISTRATIVE ISSUES / general rule, to justify the cost of convening a panel and yet to avoid imposing excessive work loads on panel members, each panel should review at least $5 million but generally not more than $15 million in requests. Accordingly, if $50 million is appropriated to a given major program area, at least 4 and up to 10 panels mightbeconvened in each major program area. From year to year, the number and composition of panels might warrant adjustment in light of the amount of funding appropriated to each major program area, the types of grants sought by investigators, and the diversity of scientific approaches proposed. Evaluation of multidisciplinary team grant propos- als requires special attention and is discussed in a later section of this chapter. Administrative Changes ii i] The four features of this proposal that require an ncrease in funding from about $50 million to $550 million are the following: · Expanding the number of major program areas from three to six and allotting a minimum of $50 million to each. Offering four types of grants, including two types of multidisciplinary team grants. · Increasing the average annual "rant per principal nvestigator to $100,000. · Extending the duration of grants from 2 years to 3, 4, or 5 years. Although the proposed increase in funds is large, the administrative burdens associated with awarding $550 million would not differ greatly from those associated with the current program. Only the first two features will need to be accompanied by signif~- cant administrative changes. A program advisory committee (or council) will have to be appointed, staffed, organized, and started for each program area (concurrently, a decision will have to be made on the fate of the current CRGO advisory committee). To administer the three new major program areas, the competitive grants office will have to secure addi- tional staff assistance and appoint peer review panels. Procedures and program announcements will have to be expanded to provide grant applicants with guidance on the program areas, the four types of grants and how they will be evaluated, and the special requirements of multidisciplinary team grants. Otherwise, the administrative changes will be minor. In recent years the program has reviewed 2,000 or 91 more proposals in an annual program cycle, awarding '~~ ~~~ ' ' Wholly some mu tO DW grants Averaging about $1~,~0 per grant-$50,000 per year for 2 years). Under a fully funded program consistent with that proposed here, if the success rate for awards were to increase from 22 to 32 percent (approximating the current rate at NIH), the competitive grants office would award just over 1,000 grants each year (see Table 3.7 for the average expected amounts awarded through each type of grant) and would have to review 3,000 proposals, or 1,000 more than it does at present. NEED TO MANAGE FOR MULTIDISCIPLINARY SUCCESS The need for multidisciplinary research-both fundamental and mission-linked is widely recog- nized in the agricultural, food, and environmental community, particularly among producers, proces- sors, and farm organizations and those within other parts of the private sector. Likewise, the difficulty of funding truly multidisciplinary research is widely acknowledged. Accordingly, this initiative has em- phasized the need to provide a significant new source of support for multidisciplinary research (see Chapter 3~. The management of multidisciplinary grants, however, raises both scientific and administrative issues. They include · selecting peer review panels whose membership is suitable for evaluating the proposals, because most members are likely to be experts in the relevant disci- pline, some are likely to be experts in cognate disci- plines that can advance understanding of the proposed research, and a few are likely to be experienced in mul- tidisciplinary research; · avoiding undue disciplinary biases, yet ensuring major scientific strength; · ensuring that the mission-linked proposals relate to major problems, yet also focus on scientific ad- vances and do not have only a practical orientation; · ensuring that the plan of study is appropriate for the proposal's objectives; · evaluating the results and the processes used so that they become a basis for increasing the effective- ness of subsequent multidisciplinary research; · creating and sustaining effective linkages be- tween mission-linked research and the development and applications sectors; and · managing the grant, the research, and the rela- tionships so that the grant's objectives are achieved.
92 Managing the proposed multidisciplinary aspects of the competitive grants program can proceed from, and build on, an already strong base of experience and results: the McKnight grants (a forerunner of the proposed grants), NSF's centers of excellence pro- grams, and NIH's program and training grants. Fur- thermore, the state agricultural experiment stations (SAESs) are, to a large extent, built on the multidisci- plinary model, particularly with respect to strong mission linkages to the development and applications sector (e.g., the Cooperative Extension Service). Multidisciplinary grants programs offer signifi- cant roles for scientists from federal agencies such as ARS and NIH, from universities both within and outside the SAESs, and from the private sector. Fed- eral agency and SAKS scientists, with their long experience in multidisciplinary research, can help identify priorities, evaluate proposals and results, and evaluate the management systems proposed for the research. Cooperative extension staff, Moth SAKS scientists with extension responsibilities and exten- sionspecialistsandadvisers can bedirectlyinvolved in mission-linked multidisciplinary team research. They could serve as research staff, adapt results to site- or region-specific conditions, develop new technolo- gies, adapt existing technologies to new conditions, and disseminateresearch results and information about the applicability of technologies. Nevertheless, USDA should not expect to resolve in 1 or 2 years, in a normal manner, all the scientific and administrative issues that will arise in the context of awarding the two types of multidisciplinary grants. Special efforts will probably also be needed. Extra time and attention may well have to be given to determining how best to advance science through multidisciplinary interactions. In the first few years of grant making, special attempts should be made to assess both the successful and the less successful projects in an effort to determine the evaluation crite- ria and features of proposals that warrant attention in future years. These insights can then become the basis for improving the criteria and the selection and admin- istrative procedures. In addition, institutions and scientists must find a mutually acceptable basis for collaboration, over- come career advancement barriers, and secure suit- able longer-term funding. Partnerships must fonn across disciplines and sometimes between public and private sector scientist~and if money is available for team research, they will. Since real-world problems INVESTING IN RESEARCH am evolving constantly, ongoing research will benefit if several scientists with varied experiences attempt to provide a solution to a problem. Partnerships also will help facilitate the processes of developing and trans- femng information and technology-a key objective if the nation is to capitalize more quickly on science and technology breakthroughs. PROGRAM EVALUATION AND ACCOUNTABILITY A program with an increased investment of the magnitude now being proposed should be systemati- cally assessed to see how well its goals are being met. Another reason to conduct ongoing evaluations is the proposal's several unusual features: a strong empha- sis on multidisciplinary grants, the new type of mis- sion-linked team grant, the research-strengthening grants, and the breadth of the program areas covered by the grants. All of these features make ongoing program evaluation particularly important. Five questions will be central in the evaluations: 1. Are science and technology priorities within the major program areas defined insightfully and do they relate to national needs? 2. Are scientists from across the entire science and technology community seeking grants and submitting high-quality proposals? 3. Are the four types of grants achieving their intended purposes? 4. Is the program effectively linked to, and does it routinely communicate with, other USDA programs, programs of other federal science agencies, state pro- grams and needs, and the private sector? 5. Are grantees achieving important science and t e c h n o 1 o g y b r e ~ h r o u g h s , a n d a r e t h e s e b r e a k t h r o u g h s receiving important and timely application? Some of these questions can and should be raised annually. Others, particularly the last one, should be assessed at longer intervals, after a realistic amount of time has passed and sufficient experience with the program has been gained. At that time it will also be important to assess whether adequate funding is being awarded through the two Apes of multidisciplinary grants and whether adjustments are needed in admin- istrative criteria or procedures to more effectively encourage top-quality multidisciplinary research ac- tivities. .. .,
