Introduction
In the world of philanthropy, there is a growing concern that assessment and evaluation may take a back seat to managing the ongoing programs of the organization. Trustees may have concerns that evaluation of programs is complex, takes time, and can be quite costly. This is especially relevant for smaller funds. On the other hand, evaluation of award programs may generate useful information to guide better decision making by organizations.
In response to a request by the Lucille P. Markey Charitable Trust, the National Research Council (NRC) of the National Academies, through the Board on Higher Education and Workforce (BHEW), is conducting an evaluation of the Markey Trust’s grant programs in the biomedical sciences. During an interval of 15 years, the Markey Trust spent more than $500 million on four programs in the basic biomedical sciences that support the education and research of graduate students, postdoctoral fellows, junior faculty, and senior researchers. This study addresses two questions: (1) Were these funds well spent, and (2) What can others in the biomedical and philanthropic communities learn from the programs of the Markey Trust. To accomplish these goals, the committee overseeing the project
-
Has examined the General Organizational Grants program, intended to catalyze new ways to train Ph.D. and M.D. students in translational research;
-
Convened a conference of Markey Scholars and Visiting Fellows in 2002;
-
Is assessing the Research Programs Grants, which provided funding to institutions to support the work of senior investigators;
-
Conducted a workshop to investigate methods used to evaluate funding of biomedical science by philanthropic donors; and
-
Will evaluate the program for Markey Scholars and Visiting Fellows, which supported young biomedical investigators in their early careers.
The Committee for the Evaluation of the Lucille P. Markey Charitable Trust Programs in Biomedical Science,1 with the assistance of the staff from the BHEW, is evaluating the three major components of the of the Trust’s philanthropy: (1) the General Organizational Grants, (2) the Markey Scholars and Fellows program, and (3) the Research Program Grants.
This report examines the Research Program Grants, which funded research centers or programs addressing fundamental questions in the biomedical sciences. The Trustees awarded 92 Research Program Grants ranging in size from $500,000 to $13 million for a total of $325 million. The awards were made to assist in the establishment, reorganization, or expansion of significant biomedical research centers or programs and to fund established leading investigators with major commitments to the life sciences. NRC staff obtained data and information from the Lucille P. Markey Charitable Trust Records archived at the Rockefeller Archive Center, examined Markey databases, solicited materials from grant recipients, and conducted site visits to a sample of grant recipients. The committee sought to understand whether the grants made to develop centers or programs resulted in program creation and development, program sustainability, research productivity, and faculty development, and positively integrated the funded program with the host institution. Unfortunately, the committee was not able to assess adequately the scientific quality or impact of the Research Program Grants on biomedical research or the impact of the program on the research centers and projects that it funded. This inability stems from one of the Research Program Grants’ strengths, its flexibility in not imposing stringent reporting requirements on grant recipients. As a consequence, information that would be useful
to an evaluation of the impact of the Research Program Grants was not systematically collected.
This is the third in a series of reports that document the activities of the Markey Trust. The previously published, Bridging the Bed-Bench Gap: Contributions of the Markey Trust, examines the General Organizational Grants program, while The Markey Scholars Conference Proceedings summarizes presentations and abstracts from the 2002 Markey Scholars Conference held as part of the National Academies evaluation. Both reports are available through the National Academies Press. Additional reports will assess the Markey Scholars and Visiting Fellows programs and publish the proceedings of a workshop on evaluation practices in philanthropic and public organizations that support biomedical scientists.
Just as each of the Markey programs varied in terms of goals and focus, so did the committee’s approach to assessment and evaluation. The Markey Scholars program was evaluated prospectively and is amenable to greater methodological rigor than this assessment of Research Program Grants or the previously published examination of the General Organizational Grants. This report relies on expert judgments and on the information gathered in site visits. It is organized into several sections and a set of appendixes, beginning with a history of the Markey Trust and the Markey grant programs. It continues with a discussion of the methodological issues related to evaluating these programs as a whole and the Research Program Grants in particular, and it briefly describes each of the 92 Research Program Grants funded by the Markey Trust. It concludes with potential lessons for funding organizations or individual philanthropists with analogous interests in supporting biomedical research. The appendixes summarize the site visits and telephone interviews with principal investigators conducted by the committee, expert consultants, and NRC staff.