NIH Extramural Center Programs: Criteria for Initiation and Evaluation (2004)

It is evident from the description of the National Institues of Health (NIH) planning and budgeting process in Chapter 3 that proposals to establish new initiatives such as a center program are vetted extensively by external advisors as well as professional staff and weighed against other priorities and mechanisms of research support. Clearly there are criteria for adopting center programs—new center programs are not established without considerable effort and on occasion they are revised and even terminated in response to changing circumstances—but the decision criteria are implicit and somewhat variable from institute to institute. There are no official NIH-wide criteria for establishing center programs. The institutes also do not have published criteria for center programs.

The major reason that the definitions of research centers and the criteria for establishing a center program at NIH are not very specific or standardized stems in part from their varied nature. Centers are a means to an end—advancing research in a particular area of science—and the need for a center depends in part on the opportunities and needs in that area. The absence of explicit criteria for establishing a center program is reinforced by the format in which the choice is presented—i.e., as a concept paper for a particular type, number, and size of center. The paper is often written as a draft of the substantive sections of the Program Announcement (PA) or Request for Applications (RFA) that will be issued if the program is approved and funded, rather than as a response to criteria for creating a program.

The benefit of implicit criteria is flexibility. An institute or group of

institutes can establish a new initiative to support projects that function like a center, but call them something else, such as programs, collaborative projects, networks, partnerships, initiatives, teams, or consortia.¹ There is also flexibility in the mechanism of support. Most center programs use center grants or cooperative agreements to support centers, but some use research project grants (RPGs) or research and development contracts.² Whatever the mechanism of support the program uses, the requirements in the solicitation can be tailored to each situation, depending on the desired role of centers in a specific area of research.

The lack of explicit and consistent criteria, however, can be a source of frustration for those on the outside—citizen and professional groups and congressional overseers—who may not be sure why NIH establishes, or opposes establishment of, centers as an appropriate mechanism for productive research. This chapter analyzes the criteria implied in the justifications for center programs published in the invitations for applications (PAs and RFAs). The committee believes that developing and adopting explicit criteria would improve decision making and make it more understandable and acceptable to interested parties outside NIH.

Review criteria for evaluating applications for center awards are published in the PAs and RFAs, but they address the specific purpose of each particular set of centers and, in any case, the criteria for selecting award winners are not the same as those for deciding whether the program of awards is appropriate in the first place. For example, a common criterion for establishing a center program at NIH has been the need for multidisciplinary research on a problem by scientists in different fields who

do not usually collaborate. A criterion, therefore, for selecting an application for a center grant from this program would be whether the applicant shows convincingly that he or she would in fact be able to create the multidisciplinary collaboration the program was established to achieve. The criterion for establishing the center program in the first place would have been that a high-priority research question could only be addressed by multidisciplinary research on a scale that individual laboratories could not achieve working separately.

The only NIH-wide application of criteria for a new center program is the procedure used in reviewing and approving concepts for PAs and RFAs for any new program initiatives. The NIH Manual says that concepts must be reviewed for “relevance, priority, and need” by the institute’s national advisory council or other advisory body (NIH, 1994). If the awards are going to be cooperative agreements, the institute must also justify the rationale and need for substantial scientific and programmatic involvement by NIH staff in the program (NIH, 1993). These criteria apply to new centers because they are always launched, at least initially, by an NIH initiative.

There are also several definitions of centers. According to a glossary of NIH grant terms, “Research Centers” are “Grants that support multidisciplinary, long-term research and development programs at research centers. Research centers usually have a clinical orientation and include all P activities … that are not included in research projects (R); M01 activities; selected U activities (U41, U42, U54); R07; and G12.”³ The reference to letters and numbers refers to activity codes in the codebook for Information for Management, Planning, Analysis, and Coordination (IMPAC), NIH’s management information system (NIH, 2002), which the Office of Extramural Programs uses to track the number and funding of awards. Each code has a short definition, but these were not intended to form the basis for criteria for deciding whether centers are needed. Also, although some institutes post the IMPAC definitions on their website, others have their own definitions. The National Institute of Child Health and Human Development (NICHD), for example, has its own definition of the P50 grant.⁴ The National Cancer Institute (NCI) adopted the IMPAC definition for the P50 specialized center in the list of extramural mechanisms on its website, but in practice it uses the grant selectively to support centers focusing on translational research. NCI uses the P50 code for its Specialized Programs of Research Excellence (SPORE) and has modeled its newer center programs on what it calls the SPORE “blueprint,” such as Transdisciplinary

Tobacco Use Research Centers and Centers of Excellence in Cancer Communications Research (NCI, 2001, 2002).

RFAs and PAs often state the rationale for a center program, usually in a few sentences in the opening paragraph, which states the purpose of the RFA or PA, although the focus is on the criteria for funding centers rather than the criteria for establishing the center program. Sometimes they refer to institute or trans-NIH priorities or strategic goals or to recommendations of an expert group or workshop. They may say that centers are better suited to accomplish program goals than other types of research support, such as traditional individual investigator-initiated grants or program project grants. Some discuss how centers combine with or complement other mechanisms to advance research in a field or on a problem.

The justifications for center programs used in RFAs and PAs can be grouped into the following categories, which are in effect the implied criteria in common use.

1. The scientific opportunities and/or public health needs that the program would address have high priority.

A center program is more expensive, represents a long-term commitment, is more complex to administer and evaluate, and leaves the institute with less budget flexibility than the traditional research grant. The problem or opportunity it would address, therefore, should be especially important, in principle identified or reviewed by outside advisors and made a high program priority by the institute’s leaders and national advisory council or meet a trans-NIH priority. RFAs and PAs for centers typically devote a paragraph or two to the importance of the research the center program is expected to facilitate, usually referring to a priority-setting exercise, such as a strategic plan, institute or NIH initiatives, or advisory group report on research priorities. A program of centers is often intended to be part of a set of activities that complement if not reinforce each other, including individual and program project grants, clinical trial networks, career development and training grants, and access to research resources supported by NIH.

According to the RFA for Centers of Excellence in Complex Biomedical Systems Research, for example, “This program is responsive to the Biomedical Information Science and Technology Initiative and its call for National Programs of Excellence in Biomedical Computing…The NIGMS [National Institute of General Medical Sciences] intends to support Centers of Excellence in Complex Biomedical Systems for research areas that 1) are central to its mission, and 2) focus on developing new computational approaches to biomedical complexity…An example of particular interest to NIGMS has been articulated in the planning document, ‘A Vision for the

Future: A Complete Picture of the Healthy Cell.’”⁵ Additional examples of justifications of center programs based on the importance of the scientific opportunities or public health needs, or both, that they would address, are in Appendix E.

2. The center would provide an organizational environment that would facilitate activities that are most effectively undertaken by teams of investigators working in close proximity.

These activities include:

• multidisciplinary collaborations for problems that require diverse scientific backgrounds

Every PA and RFA examined by the committee states that multidisciplinary or interdisciplinary research was a major reason for creating centers. Some noted that centers provide an environment more conducive to multidisciplinary research on complex problems than individual investigator grants. Some PAs and RFAs stated that centers would make faster progress in multidisciplinary research than individual investigators working separately would.

The RFA for Muscular Dystrophy Cooperative Research Centers, for example, states that “Muscular dystrophy research requires multidisciplinary approaches, based on expertise in muscle biology, genetics, imaging, muscle plasticity, exercise science and physical therapy, nutrition, molecular biology, neuroscience, rehabilitation medicine, epidemiology, clinical trials, bioengineering, electrophysiology, psychology, and behavioral sciences.”⁶ Similar statements in other RFAs are quoted in Appendix E.

• multi-investigator teams capable of a scope of activities not possible with other funding mechanisms

Many centers consist of basic, clinical, and population-based research projects that are supposed to focus on a common theme and interact to promote translation of basic research findings into clinical applications. In some cases, centers are meant to enable assembly of research teams that can tackle research questions too large for other types of research support to handle easily.

The PA for Cooperative Centers for Translational Research on Human Immunology and Biodefense put it this way: “The Cooperative Agreement mechanism (U19)…will be used to support the development of multi-investigator teams with a scope of activities not possible with other funding mechanisms.”⁷ Similar statements from other PAs and RFAs are in Appendix E.

• translating the results of basic research into clinical practice

Many recent centers are expected to engage in translational research, which is the bridge between basic science and better treatments, diagnoses, and prevention efforts. Translational research moves basic research advances into technology development and initial clinical trials while basic researchers benefit from proximity to patients and clinical research. For example, “The major goal of this program [Autoimmunity Centers of Excellence] is to support an integrated basic and clinical research program focused on tolerance induction and immune modulation to prevent or treat autoimmune disease. The close interaction between basic researchers and clinicians will accelerate the translation of basic advances to the clinic and the utilization of patient materials for basic research.”⁸

• complementing existing and stimulating new investigator-initiated applications for research project grants

Core grants, which fund research resources and services, called “cores,” and center administration, but do not fund investigators or research projects, are meant to support researchers funded by other means. Even when the center grant funds research projects, such as a P50 award, the RFA often indicates that the centers are supposed to complement and interact with RPGs and other mechanisms of support. Some larger established center programs, for example, Cancer Centers, are designed to serve as a nationally distributed platform for an institute’s broader portfolio of programs (e.g., individual investigator and other types of grants; training programs; clinical trials; screening and prevention programs; and community education, outreach, and intervention programs). A recent external review of the Cancer Centers program urged NCI to increase its reliance on centers for program implementation and coordination (NCAB, 2003).

Centers for AIDS Research and Alzheimer’s Centers, among others, also carry out this function. The new Regional Centers of Excellence for Biodefense and Emerging Infectious Diseases Research of NIAID will be regional resources, providing biosafety level 3 and 4 containment facilities and backing up first responders in an emergency.

The RFA for Autism Research Centers of Excellence stresses the complementarity of center grants and other forms of research support. “STAART [Studies to Advance Autism Research and Treatment] support is not intended to be a substitute for individual grant support. It is, therefore, expected that project and core leaders will have independent, peer-reviewed research support. Neither should the STAART Center be the primary source of research funding for the investigators associated with the Center. It is desirable for STAART-supported research to complement other funded research related to autism taking place at the applicant institution, including activities supported by R01, P01, P30, P50, and other mechanisms.”⁹ For other RFAs that indicate the center program is supposed to complement the institute’s other mechanisms of research support, see Appendix E.

• training of graduate students, postdoctoral fellows, physician-scientists, nurses, and other health professionals in cross-disciplinary or translational research

In some cases, a center is expected to promote training in the area of research the center addresses, although few center awards fund training activities or support trainees. For example, Regional Centers of Excellence for Biodefense and Emerging Infectious Diseases Research “must include a consistent and significant commitment to career development with the goal of increasing the availability of researchers for biodefense. This may focus on advanced postdoctoral candidates, junior faculty, or established investigators who wish to develop or refocus their careers on biodefense research.”¹⁰

• attracting experienced researchers into a new area of research

The existence of a center is also expected to attract established researchers into the field that is the center’s focus. The purpose of Exploratory Center Grants for Human Embryonic Stem Cell Research, for example, is to “encourage and enable basic biologists with little or no prior

HESC [Human Embryonic Stem Cells] experience to work with HESC and establish the utility of HESC as a model system.”¹¹

• networking with other centers in the program to conduct coordinated research beyond the capacity of any single center, for example, by recruiting larger numbers of patients into common research protocols; pooling patient data and biological specimens on the scale necessary to identify biomarkers for disease risk, disease activity and severity, and clinical outcome; and improving methods and technologies

There is a trend in recent center programs toward considering the set of centers not just on their own, but as a network or consortium of centers with the capacity to collaborate on activities that cannot be accomplished by individual centers. In some cases, the program sets aside funding for collaborative research projects and establishes a steering committee of representatives from the centers and NIH to identify opportunities and set priorities for the funding of collaborative projects. This probably reflects the increasing scale of research in many areas of health research. It also reflects the establishment of center programs addressing relatively rare diseases, where multiple sites are needed to recruit enough patients for translational and clinical research.

In the case of Breast Cancer and the Environment Research Centers, for example, “All study sites will use similar methods to collect data on markers of physiologic changes during the pubertal process, and assessment of environmental stressors of importance to future breast cancer risk, including lifestyle behaviors, nutrition and anthropometric markers, and chemical, physical, and social exposures at home and school. In addition, pooled analysis of genetic polymorphisms of interest will be included to fully explore relevant gene–environment interactions.”¹²

3. The centers would provide critical research resources needed for productive research that are difficult or too expensive to develop in most individual laboratories.

In the case of core grants, usually the majority of funding supports research resources and services that can be provided to investigators on a more cost-effective basis than alternative mechanisms can. Such resources are not unique to core grants. For example, program project (P01) grants

also support cores, but in most institutes they are smaller in budget and number of projects allowed than center awards.

Institutional Center Core Grants to Support Neuroscience Research are an example of a center program established to provide critical research resources. “The purpose of this program is to advance the NINDS [National Institute of Neurological Disorders and Stroke] mission to promote understanding and treatment of neurological disorders by providing core research facilities that are not otherwise available. Each Center Core Grant will support shared resources and facilities used by investigators with research projects funded by NINDS. This support, by providing more accessible resources, is expected to assure a greater productivity than would be possible from the separate projects.”¹³

4. The centers would build the infrastructure to promote the institutional development of a field of research (e.g., nursing, population research) at minority-serving institutions or institutions in regions with little NIH research funding and community education and outreach programs.

Sometimes, a center program is a deliberate strategy to build up a field or area of research. The National Institute of Nursing Research, for example, “has historically supported the development of research infrastructure in schools of nursing by funding Centers,” according to its RFA for nursing research developmental center grants.¹⁴ NICHD started the population research centers program 30 years ago to support researchers when population research was a developing field. Now that the centers are well established, NICHD has changed the program to develop population research infrastructure in new places.

Center programs have been established recently to build research capacity in minority institutions as part of a strategy to address health disparities. These include the National Center for Minority Health and Health Disparities’ Project EXPORT centers; Comprehensive Centers on Health Disparities funded by the National Center for Research Resources (NCRR) and National Institute of Mental Health; NCRR’s Centers of Biomedical Research Excellence; Centers for Population Health and Health Disparities funded by the National Institute of Environmental Health Sciences, NCI, National Institute on Aging, and Office of Behavioral and Social Science Research; and Native American Research Centers for Health co-funded by NIGMS and the Indian Health Service. The RFA for Excellence in Partner-

ships for Community Outreach, Research on Disparities in Health and Training (Project EXPORT) put it this way: “These center grants will provide a mechanism to strengthen the infrastructure for minority health and other health disparities research and training as well as provide resources for the development of innovative partnership models.”¹⁵

NIH has supported research centers to achieve a number of scientific and programmatic objectives. These include encouraging interdisciplinary research and research training, fostering translational research, providing research resources cost-effectively to individual investigators, developing research infrastructure in fields or institutions that have not been research intensive, providing a regional resource for health care providers or first responders, and supporting a network of centers able to recruit adequate numbers of patients for clinical trials or other research protocols.

Finding. The rationale for initiating a center program stated in concept papers or in the PAs and RFAs does not always indicate why a program of centers is a better means for achieving program goals than other mechanisms of research support. The scientific rationale for adding centers to the mix of funding mechanisms in a specific area is not usually made explicit, and the comparative advantage of using centers to accelerate progress is not always shown.

Acceptance and use of explicit and consistent criteria such as these by all interested parties, including Congress, patient advocacy groups, and the scientific community as well as NIH, would contribute to a more informed discussion and improved decision making concerning the appropriate use of centers or other mechanisms of research support in carrying out the NIH mission. At the same time, we believe that the criteria, such as those suggested above, should not be so narrow and detailed that they inhibit creativity and needed flexibility in addressing the wide range of research questions that NIH faces. As with Recommendation 2, the goal is to provide

enough structure to produce transparency, but not so much as to inhibit creativity.

Finding. NIH is occasionally urged to establish centers by Congress or by groups advocating greater federal action on a specific disease or other health issue about which NIH scientists believe the knowledge base or the number of active researchers, or both, are too small to support an effective center program. Even if the process and criteria for reaching this conclusion are made more open and explicit, and involve broad input from the scientific and advocacy communities (see Recommendation 2), differences among stakeholders and scientific experts may still exist. Congressional hearings may not provide the optimal forum for resolving these differences. A need exists for an advisory mechanism to assist Congress and NIH when there is continuing disagreement about the need for centers for a specific disease or other health problem.

The committee does not believe that the ad hoc review process will be used often. Historically, appropriate responses to public health needs have been worked out, sometimes including establishment of a center program. Relatively few center programs have been mandated by Congress. Congress should be the court of last, not first, resort. Congress should first consider whether the process, like the one recommended in the last chapter (Recommendation 2), which would ensure broad input and consideration of the need for centers, has been followed by NIH, and whether criteria like the ones recommended above (Recommendation 3) have been applied, which would ensure that the costs and benefits of the center model have been weighed against other ways of supporting research. If the open process and appropriate criteria have been followed, that is, if all arguments for establishing centers have been heard and the criteria for centers have been applied as part of the NIH program planning and budgeting process, but the responsible officials at NIH—ultimately the NIH and institute director or directors—decide that a center initiative is not an appropriate response to the biomedical problem in question, Congress may be assured that a decision that new centers are not needed is well justified.

NCAB (National Cancer Advisory Board). 2003. Advancing Translational Cancer Research: A Vision of the Cancer Center and SPORE Programs of the Future. Report of the National Cancer Advisory Board Ad Hoc P30/P50 Working Group. [Online]. Available: http://deainfo.nci.nih.gov/ADVISORY/ncab/p30-p50/P30-P50final12feb03.pdf [accessed December 15, 2003].

NCI (National Cancer Institute). 2001. Guidelines for Specialized Programs of Research Excellence (SPOREs). [Online]. Available: http://deainfo.nci.nih.gov/awards/spore.htm [accessed December 15, 2003].

NCI. 2002. Plans & Priorities for Cancer Research: The Nation’s Investment in Cancer Research for Fiscal Year 2004. [Online]. Available: http://plan.cancer.gov/ [accessed December 15, 2003].

NIH (National Institutes of Health). 1993. Implementation of Cooperative Agreements. In: NIH Policy Manual, Chapter 4815. Bethesda, MD: NIH (Available from the Office of Extramural Programs, Office of the Director, NIH).

NIH. 1994. Program announcements and requests for applications. In: NIH Policy Manual, Chapter 4110. Bethesda, MD: NIH (Available from the Office of Extramural Programs, Office of the Director, NIH).

NIH. 2002. Activity Codes, Organization Codes, and Definitions Used in Extramural Programs. [Online]. Available: http://grants.nih.gov/grants/funding/ac.pdf [accessed December 15, 2003].