Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 157
Aids and Behavior: An Integrated Approach 5 The Context of AIDS Programs at NIAAA, NIDA, and NIMH In order to analyze the AIDS research programs of NIAAA, NIDA, and NIMH, one must understand the larger context in which they have been operating. The most significant elements of this context from the point of view of this study are the passage of the ADAMHA Reorganization Act of 1992 (PL 102–321), which separated ADAMHA's research and services entities into two different agencies (NIH and SAMHSA, respectively), and the NIH Revitalization Act of 1993 (PL 103–43), which assigned the NIH Office of AIDS Research (OAR)—housed in the office of the NIH director—new budgetary authority over the AIDS programs of all NIH institutes. These two major legislative events occurred while the committee's assessment of the ADAMHA AIDS programs was under way, and, although they were not intended to be the focus of this report, they are an important part of the overall context in which AIDS research at NIAAA, NIDA, and NIMH was conducted in the recent past and will be conducted in the future. Also, the changes will have considerable impact on the institutes' AIDS programs and their connection to AIDS services programs elsewhere in the Public Health Service (PHS). Because these changes are quite recent, much of their potential impact can only be suggested. There are other contextual factors that also were influenced by the ADAMHA reorganization and the new authority of the NIH OAR, and that in turn affect the nature of the AIDS programs of NIAAA, NIDA, and NIMH. These are the overall and AIDS-specific
OCR for page 158
Aids and Behavior: An Integrated Approach budget process and the grant review process. It is important to understand the budget process because it governs the activities of all federal agencies, by circumscribing the financial constraints under which they operate from year to year. The review process is equally important to understand because it influences the scientific identity of the institutes by determining which specific research projects get funded. THE REORGANIZATION OF ADAMHA BACKGROUND As outlined in Box 5.1, the organization of ADAMHA (and its predecessors) has always been problematic (IOM, 1991b). This is largely because, unlike the rest of the Public Health Service, ADAMHA included research, service, public health, and training activities all in one agency. The debate about the advantages and disadvantages of various options for administering all of these components continued (with periods of greater and lesser intensity) until Congress passed the ADAMHA Reorganization Act in 1992, which for the first time separated services and research into different agencies. On October 1, 1992, NIAAA, NIDA, and NIMH were moved organizationally to NIH, and a new agency—the Substance Abuse and Mental Health Services Administration (SAMHSA)—was created to manage the former ADAMHA service functions. According to individuals both inside and outside of the Department of Health and Human Services (HHS), a number of complex reasons explain why the separation of research and services occurred successfully in 1992 and not earlier. First, the number of new programs and the budgets for these programs (especially those related to substance abuse) increased rapidly during the later part of the 1980s. As the agency grew, it had a more difficult time balancing the competing and sometimes conflicting missions of research and services (including the conflicting demands of constituency groups). Although internally the agency increasingly focused on research (by the late 1980s, the ADAMHA administrator and the directors of all three institutes were research scientists), the external community and HHS were more concerned about service and prevention activities. Under Bernadine Healy, NIH, which previously had fought against incorporating ADAMHA's research institutes, changed its position and actively sought the transfer of NIAAA, NIDA, and NIMH to NIH. The Office of National Drug Control Policy (ONDCP) in the White House also strongly favored the reorganization and believed that the research
OCR for page 159
Aids and Behavior: An Integrated Approach Box 5.1 A Brief History of Research and Services Programs for Mental Health and Substance Abuse The purpose of the following chronology is to describe the chain of events that eventually led to the October 1, 1992 reorganization of ADAMHA. It is based on information from of the IOM Report—Research and Service Programs in the PHS: Challenges in Organization (IOM, 1991b), from a chronology prepared by the SAMHSA Legislative Office, from the Lewin and Associates (1988) study, and from the 1991 hearings on ADAMHA Reauthorization. 1929 Congress authorized a Narcotics Division within the PHS to administer two ''narcotics farms." 1930 The division was renamed the Division of Mental Hygiene and its objectives were expanded to include medical and psychiatric care in federal correctional institutions. 1946 The National Mental Health Act led to the establishment of the National Institute of Mental Health (NIMH), which had three separate missions based on the premise that research, training, and services were inherently interrelated. Following considerable debate, NIMH ultimately became an institute of NIH. It was a unique institute not only because of its explicit commitment to services, but because it was the only NIH institute to strongly support behavioral and social science research. 1950s As the service mission of NIMH grew, the NIH director began to oppose the inclusion of services within any part of the NIH program. 1960 As part of a general PHS reorganization there was an unsuccessful proposal to move the services programs of NIMH (which made up 9 percent of the budget) to other PHS bureaus. This move was defeated by the director of NIMH with congressional support. 1963 The Community Mental Health Center (CMHC) Act led to a major shift in NIMH budgetary priorities—establishing community-based psychiatric treatment and developing separate community-based treatment centers for alcohol and drug abuse. 1966 The National Center for Prevention and Control of Alcoholism (which included research, training, and services programs) and the Center for Studies of Narcotic Addiction and Drug Abuse were both established as part of NIMH. Funding for the services programs had increased to 24 percent.
OCR for page 160
Aids and Behavior: An Integrated Approach 1967 A number of factors contributed to growing controversy around NIMH: the budget for the CMHC programs exceeded the NIMH budget, which caused concern within the research community that services programs had priority over research programs; NIMH targeted more of its research budget toward research into social problems, which was not viewed as an appropriate mission of NIH; NIMH was the largest NIH institute (22 percent of total NIH budget) and its leadership believed that because of its size and unique mission, it should become an independent agency. PHS was reorganized and, despite opposition from the NIH director and the research community, NIMH became an independent agency. It was the only institute to leave NIH, and the research community believed that its research program would suffer. The intramural program remained on the NIH campus and research grants continued to be processed by the Division of Research Grants. 1968 NIMH was moved into the Health Services and Mental Health Administration, a new PHS agency established to coordinate service delivery programs. The Comprehensive Mental Health Centers Act was amended to establish alcohol and treatment facilities and link them with mental health treatment facilities. 1970 The Comprehensive Alcohol Abuse and Alcoholism Prevention, Treatment, and Rehabilitation Act mandated the establishment of NIAAA as a separate institute within NIMH. 1972 The Drug Abuse and Treatment Act mandated the establishment of NIDA within NIMH. Although NIAAA and NIDA were part of NIMH, a 1972 internal management study concluded that all three institutes should become separate institutes working on equal terms under one administrative umbrella. NIMH was not reorganized, however, and the controversy over NIMH's mission resurfaced. 1973 As a result of the Nixon administration's efforts to limit the federal role in the provision of services, PHS was reorganized and NIMH was moved back to NIH to refocus its effort on research. At the time of the transfer, services represented 50 percent of its budget. After the reorganization Congress appropriated additional funds to treatment programs. The research community was concerned that NIMH's research programs were suffering.
OCR for page 161
Aids and Behavior: An Integrated Approach The Mental Health (Gordner) Task Force—established to determine how best to administer the needs for research, services, and training—presented various organizational options for restructuring NIMH activities. The task force favored integrating mental health, drug and alcohol abuse research, training, and services into the larger health care system, but also believed in a need for continued visibility and leadership (especially in the area of alcohol and drug abuse). The task force also concluded that the fields of alcohol and drug abuse should gradually be combined. The Secretary of Health, Education, and Welfare chose that task force's option to create the Alcohol, Drug Abuse, and Mental Health Administration (ADAMHA). This option established three separate and equal institutes—NIAAA, NIDA, and NIMH. The mission of each institute continued to combine research, training, and services. After this reorganization, a greater proportion of ADAMHA's budget was being spent on services rather than research and the research community believed that the research program was suffering because of its placement in ADAMHA. 1981 The categorical and formula grant programs at NIAAA, NIDA, and NIMH were combined into one single Alcohol, Drug Abuse, and Mental Health block grant to the states. Under this block grant system, most of the leadership responsibilities for services fell to the states. As a result, the three institutes began to focus on improving their research programs. The administration proposed eliminating social policy research. 1986 The Anti-Drug Abuse Act mandated the establishment of the Office for Substance Abuse Prevention (OSAP) to award demonstration grants to community agencies for preventing substance abuse among youth and for preventing AIDS among drug abusers. It was organizationally located in the Office of the Administrator because of fears it would not receive proper attention in the increasingly research-oriented institutes. 1987 The Homeless Assistance Act authorized additional demonstration programs and a services block grant for targeting homeless people with mental illness and substance abuse problems. The senior leadership of ADAMHA agreed that the agency should increase its research focus and explore ways of divesting itself of the service programs.
OCR for page 162
Aids and Behavior: An Integrated Approach The placement of both research and services within ADAMHA continued to be an issue, and the Senate requested a position statement from the Department of Health and Human Services (HHS). HHS commissioned Lewin and Associates to investigate the organizational options for ADAMHA, which resulted in the "Lewin Report." 1988 The Lewin Report identified five options for the reorganization of ADAMHA, yet no organizational changes occurred. The Anti-Drug Abuse Act raised OSAP to an institute level in ADAMHA and led to the administrative creation of the Office for Treatment Improvement. Congress began to appropriate increasing amounts of money for ADAMHA prevention and treatment activities. 1991 The Institute of Medicine issued its report on the study of co-administration of services and research programs in PHS. The report indicated that the structure of the organization was less important in determining the administration of programs than the nature of the policies guiding program administration. It, therefore, recommended that agency-level organization not be used as the basis for deterring or encouraging reorganization. In the spring of 1991, the administrator of ADAMHA sent a reorganization proposal to the assistant secretary for health to separate research and services programs into two agencies. NIH submitted a reorganization proposal to incorporate the ADAMHA research institutes into NIH as three separate institutes. In June 1991, the secretary presented his reorganization proposal which called for creating a "new ADAMHA" composed of the services programs and moving the research programs to NIH. Following the secretary's proposal, both Senate and House committees held hearings on the proposed reorganization. Although legislation authorizing the reorganization was approved by the Senate in 1991, the House could not reach agreement on all sections of the legislation and the bill had to be considered again the next year. 1992 The ADAMHA Reorganization Act authorized the separation of services and research by transferring NIAAA, NIDA, and NIMH to NIH and by establishing a new service agency—Substance Abuse and Mental Health Services Administration (SAMHSA). The proposed changes became effective October 1, 1992.
OCR for page 163
Aids and Behavior: An Integrated Approach orientation of ADAMHA made it difficult to respond to immediate policy imperatives related to services—a concern also voiced by Fred Goodwin, administrator of ADAMHA, in his reorganization proposal. In addition, the budgets of the services programs grew enough to be viable as a free-standing agency within the PHS. At the same time, ADAMHA research administrators feared that if budgets were limited, the service programs would be favored at the expense of research activities. In short, a combination of factors fostered a political climate that favored reorganization. STRUCTURAL EFFECTS OF THE ADAMHA REORGANIZATION Overall, the reorganization appears to have had a limited effect on the research programs of the three institutes. Their organizational structure and staffing have been left largely intact, and their review process remains the same as it was at ADAMHA for the period FY 1993 through FY 1996, as mandated by the reorganization legislation. Figure 5.1 shows the structure of ADAMHA prior to the reorganization. NIAAA, NIDA, and NIMH were primarily focused on research, whereas OTI and OSAP focused on services programs. Prior to reorganization in 1992, ADAMHA had 2,186 full-time employees (FTEs). Nearly three-quarters of the ADAMHA FTEs (1,602) were related to the research programs and the remaining one-quarter (585) was related to service activities. The majority of the AIDS funding and staffing was and remains in the research institutes. (See Chapter 6.) Figure 5.2 shows the current organization of SAMHSA. In 1992, there were 656 staff positions, which represented the 585 service positions from ADAMHA plus approximately 70 additional positions added during the reorganization, mainly in the newly created Center for Mental Health Services. The majority of these service positions came from the former OTI, OSAP, and the office of the administrator, but some were originally non-research functions in the three institutes that were subsequently transferred to SAMHSA. Box 5.2 highlights the programs and activities that were transferred from ADAMHA to SAMHSA as a result of the reorganization. Of these 24 programs, only three are directly related to AIDS: AIDS health care worker training (moved from NIMH to CMHS); AIDS health care worker training/AIDS hotline (moved from NIDA to CSAT); and the service delivery demonstrations (also moved from NIDA to CSAT).
OCR for page 164
Aids and Behavior: An Integrated Approach FIGURE 5.1 Organization of the Alcohol, Drug Abuse, and Mental Health Administration (ADAMHA).
OCR for page 165
Aids and Behavior: An Integrated Approach FIGURE 5.2 Organization of the Substance Abuse and Mental Health Services Administration (SAMHSA). Figure 5.3 shows the organization of NIH after the reorganization. The reorganization added approximately 1,600 positions to NIH, all but 31 assigned directly to the three institutes. These 31 positions were research-related positions from the Office of the Administrator at ADAMHA and were assigned to the Office of the Director at NIH. Although the reorganization appears to have had limited direct effect on the AIDS research programs of the three institutes, the Congressional Report language accompanying the Reorganization Act raised concerns about the amount of attention the three institutes devoted to AIDS. The bill required NIDA and NIMH to each create an Office on AIDS, which: … shall be responsible for the coordination of research and determining the direction of the Institute with respect to AIDS research related to: (1) Primary prevention of the spread of HIV, including transmission via drug abuse [sexual behavior in the case of NIMH]; (2) drug abuse services research [mental health services research for NIMH]; and (3) other matters determined appropriate by the Director. The bill made no specific mention of NIAAA.
OCR for page 166
Aids and Behavior: An Integrated Approach Box 5.2 ADAMHA Programs Transferred to SAMHSA as a Result of Reorganization Center for Mental Health Services (CMHS) Transferred From 1. Community Support Program (CSP) NIMH 2. Mental Health Homeless Demonstrations NIMH 3. Alcohol Homeless Demonstrations NIAAA 4. PATH Homeless Formula Grant NIMH 5. Mental Health Protection and Advocacy NIMH 6. Mental Health State Planning NIMH 7. CMHC Construction Monitoring NIMH 8. Mental Health Statistics Improvement Program NIMH 9. National Reporting Program NIMH 10. Mental Health Clinical Training NIMH 11. AIDS Health Care Worker Training NIMH 12. Refugee Mental Health NIMH 13. Emergency Services NIMH 14. Other Programs NIMH 15. SEH Worker's Compensation Office of the Administrator 16. Buildings and Facilities funds Buildings Account 17. Mental Health Services Block Grant OTI Center for Substance Abuse Prevention (CSAP) 1. Workplace Helpline NIDA Center for Substance Abuse Treatment (CSAT) 1. Substance Abuse Clinical Training OSAP 2. AIDS Health Care Worker Training/Hotline NIDA 3. HRSA Service Delivery Demonstrations NIDA 4. Pregnant and Post-Partum Women Program OSAP Office of the Administrator (OA) 1. Quick Response Surveys and Special Projects NIDA 2. Data Collection Programs NIDA 3. Service System Evaluation NIDA
OCR for page 167
Aids and Behavior: An Integrated Approach FIGURE 5.3 Organization of the National Institutes of Health.
OCR for page 175
Aids and Behavior: An Integrated Approach their portions of the proposed ADAMHA budgets for FY 1993 and FY 1994. It was only in September of 1992 (the eve of FY 1993) that the three institutes began to be incorporated into the overall and AIDS-specific NIH budget processes. Obviously, the length of the budget process under normal circumstances, coupled with the type of sudden discontinuities caused by the ADAMHA reorganization, made planning difficult for all institutes and agencies involved. WORKING WITHIN THE NIH PROCESS Given the lead time between budget formulation and execution, most of the institute- and agency-level budget planning for FY 1993 and FY 1994—the first two years that NIAAA, NIDA, and NIMH were organizationally part of NIH—was conducted while the institutes were part of ADAMHA. The institutes first participated in the NIH budget process in September 1992, when they worked with NIH to develop materials for the HHS FY 1994 budget submission to OMB. Thus, while the three institutes had been organizationally part of NIH since October 1, 1992, the FY 1995 budget was the first NIH budget to include NIAAA, NIDA, and NIMH. The institutes joined in the FY 1994 NIH budget process after the major internal HHS decisions about the NIH AIDS budget had been made. However, the former ADAMHA institutes were still affected by those decisions. The PHS budget request sent by the new (Clinton) administration to OMB provided $10.67 billion for NIH, including $1.3 billion for AIDS research. The allocation of the AIDS budget among the various institutes, offices, and centers was made by the director of OAR (and his staff) with the consensus of the director of NIH. The NIH director then made the final overall allocation of the NIH budget within the departmental mark, using the NIH strategic plan and her sense of where limited resources could best be utilized. While the administration proposed a 3.3 percent overall increase for NIH, including a 21 percent increase for AIDS, the allocations for the individual components ranged from negative numbers to significant double digit increases. The FY 1994 budget review process was basically the same as it had been in recent years. Over the past few years, the major difference between the AIDS budget process and the overall process occurred before the preliminary budget was submitted by NIH. In FY 1994, these initial activities occurred before NIAAA,
OCR for page 176
Aids and Behavior: An Integrated Approach NIDA, and NIMH formally became part of NIH. Also in FY 1994, the process was somewhat confused because of the change in administrations. The major difference with the review of the AIDS budget by the NIH office of the director (OD) was the level of detail and the focus on science. Most OD-level budget review focused on mechanisms and appropriate balance, while the AIDS review was conducted on a project-by-project or, at the least, area-by-area basis. The OAR director reviewed the plan and made recommended cuts. The institutes could appeal to him to reconsider specific decisions, or the institutes also could appeal to the NIH director. After this phase, the OAR director took the proposed AIDS budget request to the NIH director and associate directors for final review. At this stage, the review broadened to include issues of program balance, mechanism balance, and magnitude of resources requested (both AIDS and non-AIDS), as well as the question of scientific opportunity. Again, unlike the rest of the NIH budget, the AIDS review did include some consideration of specific projects. According to staff at NIAAA, NIDA, and NIMH, the FY 1994 review process was more top-down and the AIDS and non-AIDS processes were more similar than they had been in the past. NIH was given a departmental mark for AIDS and for its total budget that it then allocated to the institutes based on judgments made by the OAR director for AIDS and by the NIH director for the total budget. Apparently, the decisions in both cases were made based on judgments about where the scientific payoffs would be greatest. In addition to considering issues of overlap and duplication, appropriate use of mechanisms, program balance, and scientific opportunities, the OD also considered the specifics of projects or project areas, the quality of the science, and the likelihood of progress when determining the NIH AIDS budget allocations. In FY 1994, as in all preceding years, once the institute received its budget allocation, the determination of which specific projects to fund was mostly a function of the grant review process, which is described next. THE GRANT REVIEW PROCESS The ADAMHA Reorganization Act of 1992 provided that the ADAMHA peer review systems, advisory councils, and scientific advisory committees utilized remain in effect through FY 1996 (ending September 30, 1996). The report language indicates that,
OCR for page 177
Aids and Behavior: An Integrated Approach while the three institutes should become full members of the NIH research community, they should also retain their independence and integrity. In addition, it recognized that the institutes' current review procedures were developed over time to meet the specific and complex needs of the alcohol, drug abuse, and mental health fields, and they provide optimal specificity for the wide array of neuroscience, behavioral, clinical, and service research responsibilities (Senate Committee Report 102-131). In most ways, the review procedures utilized by the former ADAMHA research institutes are very similar to the procedures at NIH. Both NIH and ADAMHA have used a dual review system that separates technical and scientific assessment of projects from subsequent policy decisions concerning programmatic, scientific areas in which projects will be supported. In addition, the scientific review process is kept separate from funding to ensure that program officials are not involved in making determinations on the scientific merit of research applications. These operational procedures had evolved as part of overall development of PHS policies for extramural research grants—a joint activity of various PHS agencies. Procedurally, the review process at NIH and ADAMHA are also very similar. All ADAMHA and NIH grant applications are mailed to the Division of Research Grants (DRG) in its role as the central receiving point for PHS research grants. In addition, DRG's major management and scientific data systems (IMPAC and CRISP, respectively) have always incorporated information about the ADAMHA research grants. The application form and basic instructions for submitting a research grant are the same for both organizations. The major difference in processing grants is that at the former ADAMHA institutes, all grants are reviewed at institute-specific initial review groups (IRGs), while at the rest of NIH, most grants are reviewed by DRG study sections. At NIH, only the larger and more complicated grants (centers and program projects) are reviewed by institute-established special review groups (SRGs). In both cases, however, all grants receive their second-level review by the advisory council of a specific institute, and funding decisions ultimately are made by institute staff (with the director's approval). The first level of grant review is conducted by technical experts, largely from outside the federal government, and is designed to evaluate competing applications based on scientific and technical merit. The second level of review is conducted by advisory councils to assess the quality of the first level review and to offer
OCR for page 178
Aids and Behavior: An Integrated Approach recommendations based on relevance of the research to the institute's mission. The recommendations of both levels of review are advisory to the federal government and the final funding decisions reside with the institute director. Funding decisions are based not only on scientific merit and policy consideration, but may also consider administration policy, funding availability, and other factors. At the first level of review, each application is considered at a meeting of the IRG (also called a "study section") and is either given a priority score (if it contains "significant and substantial merit") or not recommended for further consideration (NRFC). Priority scores provide a number for perceived quality by the IRG and they range from outstanding (100–150) to acceptable (350–500). To help equalize the diversity of rating styles (hard and soft graders) and grade creep (the increasing proportion of applications receiving scores of 150 or better), percentile scores are calculated using the priority score for each application. Once the first level of review is complete, the reviewers prepare a summary statement (or pink sheet) for each application. These are sent to the National Advisory Council for the second level of review. This ensures that the scientific review was appropriately conducted and generates funding recommendations to the institute. If the council disagrees with the IRG on issues related to scientific and technical merit, it may recommend that the application be referred to the same or a different IRG for further consideration. If the IRG makes the same recommendation following reconsideration, its decision is final. The recommendation by the advisory council completes the formal ADAMHA and/or NIH review of an application. After each council meeting (they usually occur three times each year) the institute makes a funding decision and prepares a funding plan or pay plan, which is based primarily on the scientific merit of the projects (as indicated by their percentile ranking), but also on the availability of funds and balance among research areas. The funding plan is a list of projects recommended for funding by the program office or division and approved by the institute director. Since FY 1992, NIH has used a performance standard, called the "success rate," to measure the quantity and quality of grants funded over time. The success rate is the total number of competing research project grants (RPGs) funded divided by the total number of competing RPG applications received. The success rate varies by institute and year, and it is determined not only by the number
OCR for page 179
Aids and Behavior: An Integrated Approach and quality of applications submitted but also by available funding. The success of AIDS research applications has varied from institute to institute. At NIMH, AIDS grants (traditional investigator-initiated grants—R01s) have had higher success rates than non-AIDS grants; however, those success rates fell from 35.1 in 1989 to 13.5 in 1993 (Figure 5.6). At NIDA, AIDS grants (all RPGs) had higher success rates than non-AIDS grants in 1990, 1991 and 1993, but lower success rates in 1989 and 1992 (Figure 5.7). In general, NIDA RPGs consistently have had relatively high success rates (in the range of 28.4 to 41.1). Since the number of AIDS applications at NIAAA has been relatively small, the success rates may fluctuate dramatically from one year to the next (Figure 5.8). For example, in FY 1990, NIAAA received three competing AIDS applications and funded all of them. During the following year, NIAAA funded 6 of the 19 applications it received. In general, data for all institutes indicate that success rates for all research are declining, which is more likely a result of shrinking budgets and rising per-grant costs than of declining quality among grant proposals submitted. NIH advisory committees include any committee, council, task FIGURE 5.6 NIMH AIDS and Non-AIDS Applications, 1989–1993. Note: Includes R01s. Source: Office of Resource Management, NIMH.
OCR for page 180
Aids and Behavior: An Integrated Approach FIGURE 5.7 NIDA AIDS and Non-AIDS Applications, 1989–1993. Note: Includes R01s, P01s, R29s, R37s, R43/44s, and U01s. Source: Office of Planning and Resource Management, NIDA. FIGURE 5.8 NIAAA AIDS and Non-AIDS Applications, 1989–1993. Note: Includes R01s and R29s. Source: Extramural Project Review Branch, NIAAA.
OCR for page 181
Aids and Behavior: An Integrated Approach force, or group that is established to provide recommendations or advice on policies or other issues related to the missions of the NIH institutes. There are four distinct types of committees: scientific and technical peer review committees for research grant, cooperative agreement, and contract proposals; boards of scientific counselors to review the intramural research programs; program advisory committees to advise on specific research programs and future needs; and national advisory councils to offer advice on policies and programs and to conduct the second level of peer review. Membership on NIH advisory committees is based on demonstrated leadership and excellence in behavioral and biomedical research. Public members are chosen for an active interest in the particular mission of the committee. NIAAA, NIDA, and NIMH each has its own National Advisory Council, Extramural Science Advisory Board, and Board of Scientific Counselors. Each institute also has several review committees (IRGs), including at least one separate review committee for AIDS research. These AIDS-specific review committees were primarily established in response to the mandatory expedited six-month review for AIDS applications (as opposed to the standard nine-month review). NIAAA uses two scientific review committees for all applications: Alcohol Psychosocial Research Review Committee and Alcohol Biomedical Research Review Committee. All AIDS applications are reviewed by a subcommittee of the Biomedical Review Committee, called the Immunology and AIDS Subcommittee, which has twelve committee members with scientific expertise in medicine, microbiology/immunology, pathology, sociology, psychology, public health, behavioral science, cellular biology, and psychoimmunology. NIDA has five research review committees, one of which is the Drug Abuse AIDS Research Review Committee. This committee has two subcommittees: the Biomedical and Clinical Subcommittee (DAAR-1) and the Sociobehavioral Subcommittee (DAAR-2). The twelve members of DAAR-1 represent pediatrics, pharmacology, epidemiology, anthropology, psychiatry, neurology, neuropsychology, and psychology. The thirteen members of DAAR-2 represent the fields of epidemiology, sociology, psychology, psychiatry, other social and behavioral sciences, and addiction medicine. NIMH has more than twenty separate review committees. The Mental Health, AIDS, and Immunology Committee has two subcommittees for reviewing AIDS applications: the Behavioral, Clinical, and Psychosocial Subcommittee and the Psychobiological, Biological,
OCR for page 182
Aids and Behavior: An Integrated Approach and Neuroscience Subcommittee. The ''psychosocial" subcommittee includes five psychologists, six psychiatrists, one epidemiologist, and representatives from the fields of public health, prevention, community research, and family studies. The "neuroscience" subcommittee includes expertise in neurology, psychology, psychiatry, physiology, pharmacology, microbiology, and immunology. Of nearly 200 review groups of the DRG at NIH, there are seven study sections for AIDS and Related Research. All but two of them are focused on biomedical aspects of HIV/AIDS and primarily include representation from the following fields: experimental and clinical immunology, molecular biology and genetics, virology, microbiology, and clinical medicine. One study section is focused exclusively on the neuropsychologic, neuropathologic, and neurophysiologic analysis of HIV-infected individuals and includes expertise in neurology, neurobiology, psychology, pathology, and clinical medicine. Just one study section focuses on behavioral medicine and related disciplines, but it primarily includes representatives from medicine, specifically nursing, general medicine, behavioral medicine, public health, pediatrics, and psychiatry. The second level of review at NIAAA, NIDA, and NIMH is currently conducted by the National Advisory Council of each institute. Although the ADAMHA Reorganization Act mandated that the review processes (both at the first and second levels) be maintained through 1996, it is not yet clear how AIDS research applications at the three institutes will be reviewed when this period ends. (As this report was being written, it was reported that the NIH director had requested that the former ADAMHA institutes enter into negotiations with the NIH Division of Research Grants earlier than 1996 and that the institutes agreed [Federation of Behavioral, Psychological and Cognitive Sciences, 1994].) Given the composition of the NIH-wide AIDS study sections, some have expressed concern that applications related to the biobehavioral and social-behavioral research foci of NIAAA, NIDA, and NIMH—that is, cross-disciplinary focus—will not fare well should the three institutes be subject to the overall NIH review process. Indeed, this was a significant reason for the four-year retention of the institutes' pre-NIH review systems. While it is too soon to determine if this concern is well founded, the committee is aware that it is widespread among the institute program staff and the external research community. Furthermore, given
OCR for page 183
Aids and Behavior: An Integrated Approach this study's finding about the extent of cross-disciplinary research at NIAAA, NIDA, and NIMH, the committee believes that this concern deserves particular consideration. Under current NIH procedures, described above, each member of a review panel assigns to a proposal a score from 1 (best) to 5 (worst). These ratings are averaged, giving equal weighting to each. Proposals are then ranked according to the resulting "priority score" (after multiplying by 100) and characterized by the resulting percentile. Generally speaking, proposals are funded in order, starting with the lowest percentile. In a competitive field, that might mean funding up to the 6th or 8th percentile. As a result, the only proposals likely to be funded are those receiving extremely good ratings from the great majority of panel members. This procedure is inherently prejudicial to innovative and collaborative proposals. Unless some adjustments are made, the agencies will continue to discourage proposals involving new disciplines and multiple disciplines, or cooperation between investigators and community-based organizations. Unless instructed otherwise, responsible panel members likely would regress their scores for innovative proposals toward the mean of the scale. Because it is inherently harder to predict how such research will turn out, it is difficult to have the confidence needed to rate it as extremely good. As a result, a proposal would have to show an extraordinary degree of innovation in order to compensate for this inherent conservatism. Review panel members also are likely to regress the scores that they give to proposals involving disciplines other than their own. They are not in a position to know what "extremely good" is in another discipline. As a result, a single-discipline proposal will be less competitive to the extent that it is judged by specialists from other disciplines. A cross-disciplinary proposal is almost guaranteed to have portions that will be unfamiliar to each reviewer. In cases in which priority scores are compared across programs, single-discipline panels are more likely to have proposals with extremely good priority scores than are mixed-discipline panels. That could be interpreted as evidence that the single-discipline panels receive better proposals. However, it could just be that they have more unitary (or insular or ideological) evaluative standards. The more powerful a discipline is at an agency, the more likely it is to have its own review panel. Politically weaker disciplines are more likely to be combined into mixed-discipline panels. As a result, a naive interpretation of priority scores across panels is likely to create a situation where the rich get richer.
OCR for page 184
Aids and Behavior: An Integrated Approach These problems arise without any deliberate attempt to create barriers to new disciplines and approaches, which might want a share of the resources allocated to a topic such as AIDS. Rather, they provide an honest way of ensuring that such prejudices are part of the evaluation system. Those interested in preserving business as usual can make candid evaluations, then let the system do the discriminating. There may be individual members of review panels who are aware of these threats, and make various efforts to overcome them. However, the extent and impact of those efforts is unknown. These problems may be addressed either by working within the existing procedures or by changing them. Within the system, one possible solution is to weight ratings by the disciplinary competence of the raters. Although that change might reduce the (inadvertent or deliberate) censorship of other disciplines, it would not solve the problems of innovative or cross-disciplinary proposals. Another possibility is to rescale scores so that they are standardized within different categories. More fundamental changes might include convening special review panels, affording votes to ad hoc reviewers, and dedicating funds to unconventional proposals (so that they compete amongst themselves). In implementing these proposals NIH would have to consider the appropriateness of existing review staff for creating the appropriate panels and procedures. Also, such proposals would have to be in accord with the scientific principles of evaluation (often called psychometrics). As this report was being written, the current NIH director established a series of inter-NIH panels to examine the ways in which the peer review process could be streamlined and more innovative projects rewarded. Although this action reportedly was prompted by the "reinventing government" paradigm of the Clinton administration, it is not intended to focus solely on speeding up the process, but rather will take into account some of the issues about study section composition and scoring raised above (Federation of Behavioral, Psychological and Cognitive Sciences, 1994). In deciding whether it is worth the effort to undertake such reforms, the primary concern in the context of this report should be the effects such reforms will have on AIDS research and the AIDS epidemic. However, an important secondary consideration should be the relationship between the agencies and the scientific community. The current evaluation system alienates scientists whose work is treated prejudicially, in particular, those who attempt to cross disciplinary boundaries. Not only will this reduce
OCR for page 185
Aids and Behavior: An Integrated Approach their willingness to work on AIDS-related problems, but it also will damage the reputation of existing research. Over time, that may imperil continuing public support for business as usual, if the federal research establishment is perceived to be unresponsive to the requirements of the AIDS epidemic. CONCLUSION AND RECOMMENDATIONS CONCLUSION The development and management of the AIDS programs at NIAAA, NIDA, and NIMH takes place in a larger context. This context includes legislative and budgetary processes that are often in flux but still wield control over the resources available to manage the institutes' programs. In recent years, the most significant changes in this regard have been the structural reorganization of ADAMHA, which moved the institutes to NIH, and the new budgetary authority of the Office of AIDS Research at NIH, which changed the nature of how dollars will be allocated for AIDS research at the individual institutes. These changes were superimposed on the already complex budget process that governs all federal spending and together produced a climate of uncertainty for the management of the AIDS research programs at NIAAA, NIDA, and NIMH. To maintain some stability, the institutes were allowed to retain their existing grant review procedures for four years (through FY 1996). This was seen as important for ensuring the continued support of behavioral and social research that historically had been better at ADAMHA than at NIH. The fact that this situation will be revisited in FY 1997 raises some concerns about how the cross-disciplinary research so important for advancing knowledge in HIV prevention and intervention will fare in the future. RECOMMENDATIONS RELATED TO THE CONTEXT OF AIDS RESEARCH 5.1 The committee recommends that NIAAA, NIDA, and NIMH develop new programs and grant review procedures to encourage and facilitate innovative, collaborative, and cross-disciplinary proposals. 5.2 The committee recommends that the NIH task force charged with streamlining peer review consider alternative scoring schemes that would favor cross-disciplinary and innovative research proposals.
Representative terms from entire chapter: