The Lessons and The Legacy of the Pew Health Policy Program (1997)

One of the goals of this report is to reveal some of the factors that appear to have made the Pew Health Policy Programs (PHPPs) a success. In so doing, it is hoped that the constraints as well as the lessons learned by the programs can be identified and ultimately that the legacy or legacies can be uncovered. It is hoped that incipient health policy and graduate programs can learn from these experiences and use this narrative to guide them through the process of planning, implementing, and developing innovative and effective programs.

This section focuses on what has been termed the critical success factors of PHPP, as a whole. Critical success factors are a ''limited number of areas in which results, if they are satisfactory, will ensure successful performance" (Rockhart, 1978). These factors, identified by the 25 interviewees, through discussions held at the Pew meetings, and by sifting through archival information, form the foundation of the Pew Health Policy Fellowship Program.⁴ These performance areas represent program activities that are interdependent, that work in unison to generate success, and that should therefore receive attention. They are (in no particular order) recruitment, mentoring, building a community of scholars, an early focus on the dissertation, networking, leadership, and multidisciplinary education and training. As a result of different service concepts, different operating strategies, and other variables, each program emphasizes different sets of success factors at different points in time.

Kate Korman stated that "a variety of needs ... exist in the potential student population: postdoctoral, doctoral (both

residential and nonresidential), and midcareer; and no one program can fulfill all these requirements." With this in mind, the Pew programs pioneered a joint recruitment effort, both implicitly and explicitly, to supplement their individual efforts. Understanding that each program had a unique approach to health policy education and training, those involved in recruitment from all four sites worked together to guide potential fellows toward the program that best met their individual needs and expectations. As a team, Steve Crane (Boston University (BU)/Brandeis University), Kate Korman (RAND/University of California at Los Angeles [UCLA]), Ted Benjamin (University of California at San Francisco [UCSF]), and David Perlman (University of Michigan) searched for potential recruits who would be appropriate and beneficial for the program as a whole and who would also fit well into a particular program site. Steve Crane and Kate Korman explained that students more interested in a traditional academic career were steered toward RAND/UCLA. People more business or problem oriented were directed toward the BU/Brandeis program. For those who could not or did not want to leave their careers or their homes for an extended period of time, the on-job/on-campus Michigan program was recommended. For students looking for postdoctoral education, UCSF was most appropriate, and for professionals without doctorates, seeking additional training in an intensive 1-year midcareer program, RAND/UCLA was suggested. "We all had something to contribute," says Steve Crane, "we were competitive only in that we all wanted to be seen to do well by Pew and to stand in the best light." The four program sites were related in that they all shared the same vision and mission. Their commonality enabled them to work together in team recruitment efforts, and their diversity enabled them to capture the best students from all levels and areas of health care and health policy.

The 1994 Michigan Proposal Narrative states that a program "lives or dies by the quality of [the] student applications." Therefore, Michigan, like the other programs, engaged in major public outreach efforts. Targeted for the Michigan program were highly successful, often high-profile, midcareer professionals who were public sector representatives engaged in health policy formulation as either policy makers, policy analysts, or influential participants in the policy process, such as people in state legislatures and consulting

firms. However, Michigan learned early on that other vitally important aspects beyond professional position and influence are included in an applicant's profile. Leon Wyszewianski commented on the appropriate measures for recruiting fellows for the Michigan program and how this process changed over time:

The BU/Brandeis faculty and administration underscored Michigan's emphasis on the value of public communication and outreach in the recruitment process. Stan Wallack explained that one of the reasons that Brandeis wanted to participate in PHPP was so that the university could use the prestige and stipend to propel it into a more attractive position and recruit students of the highest quality. He conceded that although Brandeis was doing much state-of-the-art work in health policy that was being recognized and that was highly regarded, "students weren't just naturally coming to Brandeis." Stan Wallack stressed the need for active recruitment and the importance of having "program champions," people deeply committed to maintaining a high-quality student body:

Joan DaVanzo stated that having a "point person" functioning inside the program with the fellows was also critical to the success of the fellows and the program. At RAND/UCLA Kate Korman functioned as the program champion. According to DaVanzo:

Joan DaVanzo discussed the difficulty of uprooting one's life to go back to school full-time, and in some cases even switch fields completely. Kate Korman made this potentially rocky midcareer transition smoother for the Pew fellows. Furthermore, DaVanzo indicated that Marion Ein Lewin, director of the Program Office, and her staff were always responsive to and supportive of the Pew fellows. Together, these experiences fostered a defined identity for the fellows and the Pew program.

At BU/Brandeis, the doctoral applicant pool almost always included people who had been thinking about the need for more systematic and rigorous training in health policy analysis as a result of their professional activities (Richardson, 1990). Stuart Altman commented on Brandeis' recruitment goals:

Proven academic ability and independent, organizational skills were major determinants for admission to the BU/Brandeis Pew program because of its accelerated nature. Steve Crane explained that recruiting for fast-track programs is different than recruiting for traditional graduate-level programs, and this difference must be taken into consideration:

Although this type of program is not made for everyone, Stan Wallack stated that for many fellows, the fast-paced curriculum was the primary attraction, and thus an effective recruitment tool:

The postdoctoral PHPP at UCSF emphasized the importance of recruiting the very best among the highly qualified pool of potential applicants. The collective mission of the Pew programs is to train the future leaders of the health policy field; therefore, only the best and most promising applicants are selected. According to Carroll Estes, to achieve this level of applicants, the institution and faculty must also represent the best: ''The excellence of our faculty and their reputations were the single most important attraction in recruitment....You need a sterling faculty and a sterling institution for recruitment."

The UCSF program typically enrolled five postdoctoral fellows each year, drawing on a very high quality national pool of applicants. Each year, there was a mix of medical and nonmedical doctorates, with the physicians drawn from preventive medicine and primary care specialties and the PhDs typically drawn from largely social science backgrounds. Placing fellows on research projects with the faculty is the most important aspect. "Identifying a good match is one of the key things that needs to be done," says Hal Luft. The process of identification begins during recruitment and entails not only matching people to the project with regard to energy levels and similarities of interests but also finding

someone who fits well into the program and who will fit well with the faculty and in ongoing projects. Like UCSF, the RAND/UCLA program incorporated hands-on research projects into the curriculum; thus, the importance of matching student profiles with faculty research projects and interests was an essential part of its recruitment process as well.

Al Williams discussed the interrelatedness of targeting recruitment to the highest-quality students and of creating a thriving, high-quality program, since you cannot do either one well without mastering the other:

However, Al Williams noted that without Pew funds, "skimming the cream" would not have been possible and laments that today RAND is unable to recruit health policy fellows in the same way. Another factor that enabled RAND/UCLA to succeed in attracting high-quality students was the relationship between the two institutions and the availability of cross-registration. He provided the example that although the RAND Graduate School did not teach epidemiology, fellows had access to multitude epidemiology courses at UCLA. Thus, many students were going back and forth between RAND and UCLA, an opportunity that allowed for the development of a rich and comprehensive program. The joining of forces between BU and Brandeis also allowed for these cross-registration and program enrichment opportunities.

Steve Crane expressed the commonly held belief that one of the great strengths of the program as a whole was the individual faculty members who were committed to Pew's mission and who were accessible to the fellows:

Marion Ein Lewin has the unique perspective of having been the overseer of PHPP as a whole and having been closely involved in the evolution of each program. From her perspective, there have been tremendous improvements in terms of the recruitment process. Over time, the targeted market changed from highly intelligent people vying for a predoctoral, postdoctoral, or midcareer program to highly intelligent people with the same wants in terms of getting a degree or higher-level training, but who also had unique motivation, discipline, and organizational skills. The task imposed upon the Pew fellows was exceedingly demanding not only because of the complexity and unstable health care environment (the basis of the curriculum) but also because of the fast-paced nature of the program. In her words:

Jonathan Howland agrees with Lewin's conclusion that the recruitment process needs to include screening for motivation, but he adds the dimension of screening for completion. He stated that with regard to the doctoral programs, it should be made clear from the start that there is an expectation that comes along with being accepted into the program and that is, that the fellow will complete the program. This topic will be revisited later in the report.

Hal Luft discussed some unique recruitment issues that come up in postdoctoral-level programs. People applying for postdoctoral fellowships are often applying for tenure-track positions simultaneously. Thus, if the postdoctoral application process ends before most institutions notify their new faculty recruits, the fellowship may lose some of the very best because they will be holding out to hear about faculty positions. This may seem like a simple logistical matter; however, competing external factors like this could have long-term effects on the quality of the student body and must therefore be addressed.

This section has elaborated mostly on the comments of the program directors, those with firsthand knowledge of the recruitment process. However, Dan Rubin noted that it would be interesting to look at how the fellows actually came to choose a particular program or even how they came to be interested in seeking the kinds of degrees and programs offered by PHPP. Although this was not one of the questions specifically asked of the alumni who were interviewed, the information gathered showed a clear consensus that going back for more rigorous training in health policy was the best way that the fellows imagined that they could either empower themselves in their current positions or have greater influence on policy making and change in the future.

During an advisory meeting in 1993, Phil Lee stated, "A good fellow-mentor relationship is one of the single most important determinants of a successful health policy fellowship." The importance of mentoring was clearly understood by all the programs, as evidenced by their emphasis on supplementing the classroom experience with various research projects that stimulated faculty and fellow interaction and that taught students how to apply theories and techniques to answer real-life problems.

The Institute of Health Policy, a leading health care and research center at Brandeis University, expanded its mission to include educational training. This enabled the Pew fellows to have access to key faculty and research staff. All research professors from the institute were held responsible for teaching and mentoring students. The Heller School and the institute firmly believed that a program in health policy would fall short if it relied only on traditional academics to teach policy research and analysis, so research practitioners were integrated into the core faculty (Raskin et al., 1992).

The fellows from the BU/Brandeis program spoke highly of the mentorship offered by faculty at both BU and Brandeis. Many stated that because of their rich experiences and strong relationships with the faculty, they were not only inspired to provide but also felt appropriately prepared to provide similar mentoring to other colleagues and to their students. Fostering these kinds of relationships between faculty and fellows and between fellows and other fellows, where people

are comfortable and even eager to ask questions, give answers, probe issues, and challenge each other, is exactly what the Pew program was hoping to accomplish. These people trained by PHPP to respect, listen to, and learn from peers and colleagues will inform the debate and stimulate effective change. Linda Simoni-Wastila reflected on her experiences in the BU/Brandeis program:

Although the seminars and course work are vital parts of the UCSF Pew Health Policy Program, for many fellows the most important reason for participating in the program was the opportunity to participate in multidisciplinary health policy and health services research with the faculty of the institutes. This environment is further enriched by faculty-fellow interaction and the mentoring that faculty provide (Lessons Learned, UCSF, 1992–1993). The issues of faculty-fellow interaction and the mentoring and research training overlap to a great extent. Fellows receive detailed orientation to the faculty and their projects. Intensive, individualized programs of counseling, mentoring, and career development have been implemented. Carroll Estes cites the colleagueship and mentoring stimulated through large-scale projects as one of the most innovative methods of program implementation at UCSF. Hal Luft underscored his colleague's emphasis on the value of mentoring:

Likewise, says Carroll Estes, "The fact that the two institutes at UCSF together have about $20 million in funded research every year and probably 40 faculty investigators who are well known in their area has really contributed to implementing very significant training opportunities."

The postdoctoral fellows at UCSF had close contact with their assigned advisers and met with them regularly to review progress in research, field placements, publications, course work, and career planning. Faculty monitor and evaluate each fellow's progress, adjust his or her individualized program according to emerging needs, and provide assistance with specific problems. Regular meetings provide an opportunity for informal discussions and for forming the kind of trainee-mentor relationship that is so vital for career development (Lessons Learned, UCSF, 1992–1993).

In an effort to improve faculty-fellow interaction, all UCSF Pew fellows received guidelines listing the expected goals of the Pew fellow-mentor arrangements. This helped them to know what was expected of them and what they could expect from their faculty preceptor. Fellows and faculty were also provided with general guidelines for establishing a productive and enjoyable relationship with a mentor. These simple guidelines helped to clarify the confusion that often surrounds the roles and responsibilities expected of fellows and faculty in academic programs. (See Appendix C for guidelines.)

Pew funding permitted students to offer free assistance to a faculty member in exchange for ongoing mentoring and inclusion in a research project that would meet the individual fellow's training goals. Faculty usually found that the benefits of working with a fellow far exceeded the time of mentoring, provided that the match was a good one in terms of skills, content, timing, and expectations. It was made clear that fellows were "free agents" in their dealings with faculty mentors, and if things were not working out, they were free to seek out another faculty member and research setting. In the experience at UCSF, changing mentors because of irreconcilable conflicts over the research approach was rare, but conflicts over authorship occasionally occurred. This led to encouraging faculty and fellows to discuss issues of authorship order at the beginning of a writing effort.

When asked how the UCSF postdoctoral fellowship approach differed from a traditional postdoctoral fellowship, Carroll Estes responded:

Estes attributes the success of the mentorship program to the general support provided by the Pew program for faculty who would not have support for any teaching or mentorship concerns otherwise.

Research collaboration between fellows and faculty does not end at the conclusion of the fellowship period. Fellows who assume academic positions, in particular, are often involved in completing research and publications initiated during the fellowship as well as developing research proposals out of earlier projects. The extent to which postdoctoral fellows who have graduated continue to collaborate with faculty from their programs may be one way to measure the success of the mentorship available during the training period. For example, Joan DaVanzo, who went through a combined doctoral program at RAND and UCLA, emphasized the value of the work projects in fostering close mentoring relationships. In addition to getting on-the-job, real-world experience, the work projects allowed for close contact with the faculty and project directors.

The University at Michigan program is designed to provide active support for doctoral fellows for a 3-year period. There are 2 years of course work and additional support for the completion of the dissertation. The demands of course work and dissertation projects are substantial, competing with full-time professional activities. To deal with this obstacle, faculty began to closely monitor and provide increased academic and psychological support to the fellows. According to Richardson (1990), this approach has been extremely helpful in assisting students in completing of their course work and dissertations.

The fellowship is a transition period, with marked uncertainties about the future and a groping for identity as health policy researchers (Lessons Learned, UCSF, 1992–1993). In addition, fellows are often at similar stages in their nonprofessional lives, and the sharing of experiences is important. Therefore, specific steps were taken at all sites to foster a feeling of community among the Pew fellows through traditional and formal methods such as seminar-style classes, lunchtime colloquia, and conferences. However, PHPP helped to create a sense of community through other, less

formal routes. These include various social activities, the establishment of research project teams, and the logistical issue of placing the fellows in offices close to each other and the faculty because placing fellows in close proximity to one another and to the hub of communication is an important component of creating a community of scholars.

During the early years of PHPP, however, several of the sites did not have the space to place fellows in such strategic locations. Fellows were strewn about, hindering the vital interaction needed to create a community of scholars. Recognizing the dangers of haphazard placement, most of the programs resolved their space-related problems early on. For instance, the RAND/UCLA and the UCSF programs took the issue of proximity very seriously and worked hard to get fellows space together and close to the faculty members' offices. This enabled the fellows to informally discuss research projects, classes, and shared experiences. This exposure allows for the development of crucial critiquing skills because it provides the fellows with the opportunity to receive and provide feedback to their peers.

UCSF (Lessons Learned, UCSF, 1992–1993) points out the fact that too much space or space in some out-of-the-way location can be detrimental to student evolution. Fellows should be clustered near the power center of the organization. Important business is often transacted during chance encounters while picking up mail or getting coffee. These meetings are particularly important in getting fellows to meet and interact with faculty members with whom they are not primarily attached (Lessons Learned, UCSF, 1992–1993).

Carroll Estes attributes much of the success of the Pew programs to the commitment to fostering a community of scholars, in large part due to the general support for faculty provided by the Pew Charitable Trusts. Pew funding for faculty time spent mentoring and interacting with fellows on research teams fostered a secure and close community of scholars. Several program directors cited faculty commitment as one reason why the Pew Health Policy Programs are in a league of their own, and this is due in large part to the general support of the Pew Charitable Trusts. Carroll Estes speculates about how the outcomes have differed because of this unique fellowship training approach:

Richardson (1990) found that fellows greatly valued scholarly interaction with their peers. Fellows considered interaction within the student community and with faculty and other health professionals as being vital to their successful evolution as health policy professionals. The fellows also cited Pew-sponsored conferences as contributing to the learning experience and a great opportunity to make additional contacts in the field. The fellows viewed the annual meeting and the meetings in Washington, D.C. very favorably (Richardson, 1990).

Due to the nonresidential nature of the Michigan program, issues related to physical space and working area proximity are not relevant in terms of establishing a community of scholars. Yet, according to Bill Weissert, the intensity of interaction during the weekend sessions create a scholarly atmosphere where fellows can draw upon each other's strengths and experiences:

The high-tech computer conferencing, discussed earlier by Leon Wyszewianski as one of the great strengths of the Michigan program, also allowed for the development of a sense of community among the fellows. Pat Butler and the other Michigan fellows who came from all over the country explained that the computer conferencing minimized the potential of feeling of isolated.

The diversity and maturity of the PHPP student body created a situation ripe for community learning. Steve Crane discussed how at BU the weekly seminars would bring together faculty and fellows from various disciplines to work together to solve problems from many different perspectives:

Those involved with the other programs had similar experiences, and most of those interviewed discussed the intensity of the Pew learning community. This environment, besides creating stimulating learning experiences, also reflected the real world of health policy, where problems cannot be solved by people sitting in closed offices. Real-life solutions need communities working together toward common goals.

Most students in the BU/Brandeis program were concerned about the accelerated 2-year doctoral program and their chances of completing their dissertations within that time frame. Raskin and colleagues (1992) found that although all students managed to complete their courses and exams within a year and a half, only a few focused fellows finished the dissertation within the 2-year time frame. For those fellows who did manage to finish the dissertation within 2 or 3 years, most had their dissertation topic (and some even had data) before they started the program. The fast-paced nature of the program and the fact that the fellows did not have to collect their own data but could use an existing data set helped them complete the degree quickly. Jonathan How-land, a BU fellow who went on to direct the program for some time, was one of the few fellows who was able to complete the courses and the dissertation quickly. Howland explained how he went about the process:

However, Jonathan's case was fairly unique. Most students needed slightly more direction. To assist the students with identifying and getting started with a dissertation topic early on, the training directors in the BU/Brandeis program developed a special dissertation seminar for the Pew students. Initially, the dissertation seminar was led by Sol Levine and Stan Wallack, then Steve Crane and Mary Henderson took over, and for the last 5 years of funding (and beyond) the seminar has been taught by Jon Chilingerian. This dissertation seminar has been crucial to the success of the program. The weekly seminars in the first year primarily focus on research methods and the interplay between data

and conceptual thinking at various stages of the research process. The emphasis is on managing the throughput and how one draws inferences from social science data. The seminar offers methodological perspectives for the study of health policy to open the way for fellows to begin developing their own areas of professional concentration. By the end of the first-year seminar, fellows are requested to turn their research interests and ideas into viable research topics. Stan Wallack and Jeff Prottas (a Heller School research professor and political scientist) led the seminar in the second year. The second-year seminar was initiated in 1995 and concentrates on lending support and providing peer critiques of topics regarding data sets, problems, and analysis processes relating to the student's individual projects. At each session one student is selected to lead a discussion of a recently written position paper related directly to that student's proposed dissertation research.

Linda Simoni-Wastila believes that the dissertation seminars at BU/Brandeis were one of the most innovative and useful aspects of her Pew doctoral program. She explained that the weekly seminars were attended by second-and third-year students so that all were exposed to different processes and different levels of the process:

Bill Weissert explained that early on the Michigan program did not have anybody whose job it was to make sure that students were equipped to do a dissertation. This concept of having one or more faculty members, outside of advisers, help maintain the dissertation momentum was learned over time. Richardson's evaluation (1990) also found that the University of Michigan greatly refined, over time, its orientation to and preparation of students for dissertation work with an early seminar and faculty mentoring. As part of the strategies developed to help students make timely and satisfactory

progress toward their dissertations, the Michigan program established as part of the curriculum a formal dissertation seminar. Its purpose is to provide systematically and concisely information on what is known about effective ways to structure and manage the dissertation (Michigan Proposal Narrative, 1994). The seminar serves as a forum for the discussion of general and specific problems related to the dissertation and their solutions, as well as for students to present their progress to fellow classmates for feedback. This has proven to be a very successful exercise (Michigan Proposal Narrative, 1994). During the final portion of the program, each student must sign up with a faculty member of the student's choosing for an independent study course focusing on the dissertation. The usefulness of the dissertation seminar was cited by all four alumni interviewed, and as explained by Dan Rubin, the dissertation process took on new meaning:

To sustain the momentum to completing the dissertation that was begun during the first and second years of the fellowship, the fellows are brought back to Ann Arbor on three separate occasions during the third year. At each of these sessions, students are expected to meet with their dissertation advisers. They also meet as a group at the dissertation seminar, where they report to the group and professor and receive feedback on the progress they have made, the problems they have encountered, and the insights they have gained about the dissertation progress (Michigan Proposal Narrative, 1994).

Unlike the two other doctoral programs, RAND/UCLA emphasized an early focus on the dissertation from the outset. Kate Korman and Al Williams explained that at RAND/UCLA, the research projects discussed earlier and called on-the-job training were held with the same importance as the course work, because it was the project work that was supposed to lead to a dissertation. Doctoral and midcareer fellows were encouraged from the very beginning to get involved in research projects that had the potential to

lead to a dissertation or publication, respectively. Fellows spent about half of their time on projects and were evaluated on the basis of progress and performance. Approximately half of the fellows worked on projects that led directly to dissertations. Kate Korman explains the value of the work projects to the early dissertation focus:

According to Altman (1995), one of the strongest aspects of the Pew program has been the networking among the fellows in each of the programs, between the fellows in the different programs, and between the fellows and those engaged in the policy process in Washington, D.C. All the fellows interviewed were extremely appreciative for the commitment made by the Pew Charitable Trusts and the Institute of Medicine (IOM) in establishing and maintaining a network that has extended far beyond the training programs. To many, the network has proved invaluable for their careers by providing ready access to a wide array of highly trained professionals working in all sectors of the health policy field. The network fosters collaborative efforts and enhanced communication among members of academia, government, foundations, the service sector, and the corporate world.

PHPP explicitly adopted the objective of implementing a networking system for fellows that included academicians, health professionals, and policy makers. For example, as the UCSF Pew program strengthened its ties with a number of governmental and nongovernmental organizations at the local, state, and national levels, it introduced postdoctoral and midcareer fellows to an extended network of health policy professionals, including those in the Physician Payment Review Commission, the National Academy of Social Insurance, the Centers for Disease Control and Prevention/U.S. Public Health Service, and the San Francisco Department of Public Health.

The PHPPs were designed with the understanding that to promote effective change, health policy professionals must

be trained in the art of communicating with colleagues from all areas and sectors of the health care arena. The network of Pew fellows, alumni, and faculty reflected this underlying premise. Indeed, it is the multidisciplinary nature of the network that Lisa Bero from UCSF most appreciates:

Steve Crane explained that another great thing about the Pew program was the multitude of opportunities that the fellows had to meet and network with leaders in the field, academia, and the public and private sectors. Establishing these relationships gave the program "weight of influence in the system" and gave the students access to the people who could use their research more quickly. Steve Crane reiterated the value of the multidisciplinary, multisector, real-world Pew network:

In terms of the BU Corporate Fellows Program, Crane believes that the effects of the Pew network is still reverberating throughout the business community. Likewise, the success of Bruce Spitz's Community Program at Brandeis, he says, was dependent on networking:

The overwhelming consensus and unbridled enthusiasm that the alumni used to describe both the career value and the social value of the Pew network was remarkable. Pamela Paul-Shaheen from the University of Michigan stated that she draws upon the extensive Pew network regularly through her work. Furthermore, Pamela finds the network to be extremely broad in scope:

Linda Simoni-Wastila, a graduate of the BU/Brandeis program, stated that the emphasis on building and fostering a far-reaching network tied into the policy scene in Washington, D.C., was one of the Pew program's greatest contributions to the field. She explained how the process of network development began early on and how it functioned as a motivating force from the outset:

Mark Legnini, John McDonough, Sarita Bhalotra, Terry Hammons, and Kathleen Eyre all discussed how becoming part of the network was just as important and valuable as the other aspects of the program to their successes as health policy professionals. Bhalotra explained that being part of the Pew network meant interacting with some of the leading minds in the world of health policy. John McDonough explained that the Pew program was about acquiring skills and making unique relationships:

Terry Hammons, an alumnus of the RAND midcareer program, found networking with people around the country who were engaged in policy-relevant health services research very valuable. He stated that as a result of relationships established during his year at RAND, he continues to talk regularly with academics from RAND, Brandeis, and Harvard and policy analysts in the Congressional Budget Office and the U.S. Department of Health and Human Services.

Kathleen Eyre found the network to be a huge advantage not only for the alumni influencing policy, but also for the field itself. The Pew network has linked the many different sectors of the health care arena, thereby increasing the possibility of effecting change.

Joan DaVanzo, also from RAND, gave a personal anecdote to describe the value and accessibility of the Pew network once she had completed her doctoral degree and was looking for a job out in the field. She also cited the value of the continued effort on the part of IOM to produce the directory, a means of getting in touch with alumni:

Although it was through Marion Ein Lewin's and IOM's nurturing that the extensive Pew network was formed, it would not have been nearly as successful without the commitment of the faculty and program administrators at the individual program sites. PHPP faculty considered networking to be a vital part of the curriculum. Therefore, they were committed to networking activities, such as the annual or semiannual Pew conferences. Carroll Estes stated that networking is essential to the growth of health policy professionals:

Marion Ein Lewin expressed some concern about the future of the Pew network. Although she stated that the foundation of the network is securely in place and that nearly 300 Pew alumni and many faculty are already tied in with the Washington, D.C. health policy scene, there will no longer be that concerted effort by an organizing body (like IOM) to monitor and foster the networking activities. She hopes that the programs that continue beyond the last funding cycle will continue to recognize the importance of linking fellows from across the country:

Integrating health policy fellows is vital to the success of any program, if success is measured by policy influence and communication of health care issues between the sectors. Even if future programs are not financially or academically linked, there would be great value in organizing some networking activities. Such activities not only broaden the scope and extend the reach but also magnify the ultimate products of advanced graduate programs. Pew may have only 306 fellows and alumni penetrating the health policy field, but with the relationships they have developed among themselves and with non-Pew professionals, their impact and their ability to get things done are enormous.

Altman (1995) discussed the changing needs of the U.S. health care system and stated that what was lacking was a sufficient supply of well-trained individuals who could understand and synthesize the clinical, practical, political, and economic implications of the policy process. He stated that the future of the U.S. health care system is dependent on policy decisions that reflect the problem-solving strategies of different disciplines and perspectives. Altman (1995) cites the Robert Wood Johnson Foundation's Health Policy Fellowships and the Pew Charitable Trust's Health Policy Programs as the driving forces behind educating and training health care professionals about policy making and "bringing the synergy of an interdisciplinary approach to problem solving." He described the Pew Health Policy Program as "an effort to bridge the gap between health economists and other disciplines." Furthermore, Altman (1995) stated that PHPP "fosters a cross-fertilization of different theoretical perspectives, as well as different methodological approaches in solving concrete policy problems.''

Hamilton's 1995 evaluation states that PHPP "has had from its inception a single stated goal, which was expressed

in the RFP: to stimulate the development of multidisciplinary health policy education programs that will equip a cadre of leaders with the required skills to deal effectively with the nation's complex current and future health policy issues." The interdisciplinary strength of the curriculum was cited as a primary success indicator by all three evaluations (Hamilton, 1995; Richardson 1985, 1990) and was underscored by the more recently conducted interviews.

Steve Crane stated that the Pew program was all about change. He explained that the programs were not intended to produce policy makers or academic researchers per se but, rather, change makers. Being an agent of change is a subtle and important role, says Dan Rubin, and one that is difficult to observe and measure. Rubin commented that in his career he makes many contributions to change that are invisible outside his organization. Change, says Steve Crane, requires interdisciplinary people who can talk and understand the complex and interdisciplinary nature of the current health care system:

Many Pew fellows agreed that their multidisciplinary training enabled them to understand the mechanisms of change and work toward solving problems using a variety of disciplinary approaches and perspectives. However, as the following quote from Lisa Bero so poignantly illustrates, most graduate programs do not realize the value of multidisciplinary training:

Patricia Butler explained that because the Michigan program was housed in the School of Public Health, the curriculum was able to incorporate a fundamental set of disciplines whose influence is often lacking in the national health policy debate:

Butler discussed another aspect of a multidisciplinary program: the diversity of the students. It takes more than a multidisciplinary curriculum and faculty to foster among the fellows an understanding of the convergence and interdependence of health policy disciplines. Bringing together fellows from different sectors with different expertise created the necessary depth and breadth. Patricia Butler described her class at Michigan:

The concept of multidisciplinary education or training can take on several different meanings, depending on where one sits. Steve Crane explained that the PHPP approach to being multidisciplinary certainly did not mean having people from many different disciplines sitting in the same room or housed in offices next door to one another. Rather, the PHPP idea of what constituted multidisciplinary training was that each of the disciplines had to be represented in each student. This gets back to the reasoning that problems in the health care system are not strictly discipline oriented. Many problems have economic, sociological, psychological, and political components. Furthermore, health care issues span many sectors and involve many different actors. Effectively trained change agents have acquired the knowledge and the skills to understand and apply the many different perspectives required to analyze complex health care system problems. Furthermore, the problems in health care are not institutional in nature, and they are not even mostly state oriented, explained Steve Crane. The problems in health care

are nationally oriented, cutting across all political boundaries. There continues to be a dire need in the health policy community for people who can see the big picture and apply that vision in an effective way, the way that the Pew fellows were trained to do:

Stan Wallack discussed the importance of training students to look through the various "lenses" of health policy. He explained how the Brandeis curriculum was multidimensional in terms of political science, economics, and sociology and how a specialization in health was built in using the lenses of the economist, the political scientist, and the sociologist. Stan stated that it is essential for these different paradigms and other perspectives to be incorporated in any program teaching health policy.

Despite the importance of multidisciplinary education and training for policy research, policy making, and change, the integration of different disciplines and experiences into advanced graduate training is still quite rare. Nonetheless, Dan Rubin from Michigan hopes that the prestige of the schools and the success of the programs will ultimately work to disseminate the interdisciplinary training approach at the doctoral and postdoctoral levels. Kathleen Eyre stated that it was this aspect of the curriculum—the integration of the different social sciences into a health policy framework—that distinguished the Pew approach training from the traditional training approaches.

Hal Luft explained that the innovative training approach used by PHPP enabled the fellows, once they entered the field, to continue interacting and communicating with professionals from other disciplines in their own or other sectors. Communication of this nature is fostered during the training years when the academically focused and nonacademically focused fellows merge their expertise to work on projects and solve problems. As a result of this training approach, explains Hal Luft, fellows who go on to

become policy analysts in the government can call upon their colleagues in academia for advice in ways that would not have been possible had they not gone through joint training.

Al Williams underscored the value in exposing fellows to projects where they would have to work together with professionals with expertise in various areas. This kind of preparation significantly affects the future success of the fellows. The general consensus was that the health policy field needed broadly trained health policy professionals to effect change. PHPP prepared its fellows for this challenge. In reference to the doctoral program in particular, Dennis Beatrice stated that through this innovative multidisciplinary training approach, ''Pew educated a new kind of PhD, one better prepared and more relevant for today's world."

It is interesting and noteworthy that IOM had an interest in the Pew Health Policy Program from the start. Marion Ein Lewin remembers that when the Pew Charitable Trusts began looking for an organization to direct the program, IOM had responded but had not won the proposal. The Pew Health Policy Program was brought to IOM from the American Enterprise Institute in 1987; however, it was not officially called the Program Office until 1988. It was on the recommendation of Bill Richardson, who conducted the program evaluations, that IOM was named the official Program Office.

Marion Ein Lewin describes the role of the IOM office as one that develops the joint activities of all the programs, plans annual meetings and meetings for new fellows, develops the directory and semiannual newsletters, monitors the budgets, and makes sure that all the programs are performing according to the requirements of the program. Perhaps most importantly, IOM is responsible for the network and the "family" of Pew fellows over and above their connection to the individual programs and institutions. The success of this aspect of the IOM Program Office's responsibility can perhaps be best described by a quote from Jonathan How-land, a BU/Brandeis Pew alumnus who also had the unique experience of directing the Pew program at BU for a time.

The definition of "success" varies across graduate programs. Traditionally, successful doctoral or postdoctoral programs would seed the academic field with their graduates, and successful midcareer programs would seed the management field. PHPP however, aimed to seed the whole field of health policy, encompassing the academic world, the health care management world, the various levels of government, the corporate world, and the public and private sectors. How can one program, however, produce successfully trained health policy professionals to meet the needs of so many different areas of the health care field? One way is for each program to establish a strong multidisciplinary core, a sense of urgency, and a national identity that rises above all of the programs and the fellows' individual aspirations and career paths.

The leadership of PHPP was able to foster this sense of belonging through an empowering Pew identity, but it was not easy. IOM and the program administration went to great lengths to instill in the Pew fellows a sense that they were all, regardless of their backgrounds and expertise, working toward a common goal, that is, to be more effective change agents in the health policy arena. Marion Ein Lewin reflected on the painstaking but enormously rewarding process. She stated that in the beginning none of the students thought of themselves as "Pew fellows"; rather, they looked at themselves as individuals all getting degrees or going through training programs at individual sites across the country and that Pew was only a financier. During the first year or two, the fellows thought that the source of program funding was the only common bond between their programs. As Marion Ein Lewin stated:

In an effort to counteract this early hostility, the program administration incorporated controversial presentations into the Houston national meeting in 1989, and tried to teach the fellows how to communicate effectively with people who have varying perspectives. Marion Ein Lewin describes what happened:

Marion Ein Lewin explained that this was a real turning point for the program because it was the first time that there was a discussion about what it meant to be a Pew fellow, that it was something above and beyond just financing an education. The fellows began to learn that the program had a purpose and that it wanted to create a cadre of similarly trained people who could work effectively in health policy at all levels and in all sectors.

Many others claimed that the program office helped the individual academic programs to achieve their goals. The goals of program leadership extend beyond integration and the fostering of community. They included motivating fellows to complete their challenging and difficult programs. Joan DaVanzo discussed how the dedicated leadership of the program sites and IOM set the PHPPs apart from other doctoral programs:

Likewise, Linda Simoni-Wastila discussed how the leadership motivated her to finish her doctoral program at Brandeis:

Following along the same idea articulated by both DaVanzo and Simoni-Wastila, Jonathan Howland discussed how the program leadership distinguished PHPP from other traditional doctoral programs:

Howland explained that there are two sides to this leadership approach, one positive and one an area for concern. On the positive side, the fellows had access to a "very user-friendly faculty" who believed that there was value in appropriately training fellows and allowing them expeditious entry into the field. On the less positive side, this approach allowed some fellows to be irresponsible in terms of course selection, the amount of work that they did and what learning they did. Nonetheless, overall, the fellows were mature enough to take on the challenge of a nontraditional, accelerated program, and the supportive environment succeeded in producing effective health policy professionals focused on real-world issues.

The Pew Charitable Trusts were also commended for their strong commitment to leading and administering the Pew Health Policy Program. According to Marion Ein Lewin, the Trusts' faith in the program never wavered. She explains that the directors at the Trusts had an uphill battle because foundations do not inherently like to fund programs for the long term. From the beginning, the willingness on the part of the Trusts was crucial to the success of the program. Steve Crane reiterated the uniqueness of Pew's commitment:

The Pew Charitable Trusts' decision to discontinue financial support for the health policy programs with the final grant in 1991 was based on the premise that over the 15 years of funding, three very distinct and highly successful models had been developed, and these models should become institutionalized and adopted by other institutions (Pew Charitable Trusts, Rimel and Asbury, 1995). The Trusts' claim that the last grant was, in part, a transitional effort to enable the sites to develop "how to" materials and to explore alternative approaches to financing their programs in the future. The Trusts, however, recognize that with potential reductions in health services and policy research funding these models may be difficult to sustain in the future.

In 1993, the Michigan Pew Program Advisory Committee assessed the feasibility of continuing the Michigan program if and when the Pew funding was discontinued. They determined that the university would probably not be able to retain the many faculty members supported by Pew money. They were also concerned that only those students who could afford to self-fund their education would apply to and enroll in the program. They hypothesized that such a situation might lead to a more homogeneous cohort of students, a loss of diversity in backgrounds and perspectives, a more regional student body, and a loss of national focus. The advisory committee feared that the university would be faced with the decision to retain a program of greatly reduced quality and scope or to end the program completely. This was of grave concern because the Michigan program is still the only nonresidential (on job/on campus) doctoral program in the United States. Ending the program would clearly leave a devastating void (Site Visit, 1993).

The Pew doctoral program at the University of Michigan selects midcareer professionals who are active in policy formulation and upgrades their knowledge and skills, thus enabling them to participate more effectively in the policy process. Approximately two-thirds of these individuals receive salaries that make the costs of tuition, fees, travel, and books for the program unaffordable without assistance. Patricia Butler commented on the problems of sustaining a midcareer nonresidential program like the Michigan program without funding and the changes that reduced funding would likely bring:

Pew funds have also enabled the University of Michigan to support visiting faculty. The program coordinators have found that students benefit greatly from lectures by specialists on important topics not necessarily on the research agenda of the University of Michigan faculty. The experience has also been enriched by faculty brought in from other institutions to develop courses or to fill in for local faculty who are unavailable to teach a necessary course (Michigan Proposal Narrative, August 1994). This flexibility in program offerings is very important in maintaining the quality of the Pew experience; however, it is in jeopardy as funding ceases.

On a more positive note, the faculty and program directors at the University of Michigan are committed to sustaining the nonresidential doctoral program on their campus. Their primary concern, as highlighted previously, is maintaining the rich diversity that makes their program so valuable to the health policy field. Bill Weissert commented:

The 1990 Hamilton evaluation states, "In all cases, the acid test of legitimization at the grantee institution itself is

whether the activities survive after the PCT [Pew Charitable Trusts] umbilical is cut.'' Indeed, all the Pew program sites have passed the test. At UCSF, the support for infrastructure and related training and research resources earlier in the history of the program were instrumental in acquiring permanent university funding for the Institute for Health and Aging. Hal Luft reflected on the structures now in place at UCSF as a result of the Pew program and the indirect benefits that PHPP has brought to UCSF.

The Health Policy Management Program at UCSF that was funded during the first cycle by Pew was continued on a permanent basis with funds supplied by the UCSF hospitals and clinics to support one to two fellows a year (Richardson, 1990). Other efforts to institutionalize the Pew program have continued. At an advisory board meeting in 1993, Phil Lee and Carroll Estes reported that the seminars that were developed for the Pew program have been institutionalized at the Institute for Health Policy Studies.

A major objective of the UCSF Pew postdoctoral program has been to develop the capacity to provide full financial support for core faculty and staff. However, this has been extremely difficult. Prolonged economic recession in California continues, as does an even more prolonged crisis in funding the state of California budget. Due to substantial cuts in state funding, the University of California as a whole, including UCSF, is undertaking major reductions in its workforce and salaries. In this climate of spending reductions, the university is loathe to take on additional responsibilities for new administrative or faculty expenses. Given the structural economic and government funding problems, there is little prospect in the foreseeable future for a substantial reversal in UCSF policy.

At the same time, none of the other fellowship funding agencies fully pays for the real expenses incurred for core fellowship program faculty and staff. Most of these agencies appear to assume that they are involved with a normal university disciplinary department, with faculty paid for through

the university's annual budget, rather than two research institutes that depend almost exclusively on external sources of funding. The institutes do not have the financial resources to engage in cross-subsidization of the fellowship program activities for any significant period of time, since virtually all reserve comes from research contracts and grants, and discretionary funds are very limited.

Thus, UCSF finds itself in the paradoxical position of training an increasing number of exceptionally qualified fellows while receiving a declining amount of support for core faculty and staff. Although the institutes will look to other foundations for financial support for training-related costs, it is not clear whether these foundations will be willing to pay for nonstipend fellowship costs. Without a solution to this problem, UCSF might be unable to avoid a substantial reduction in the size of what may be the most important postdoctoral and midcareer fellowship training program in health policy and health services research in the United States (UCSF Annual Report 1992-1993).

UCSF continues to take great strides in making PHPP self-sufficient. UCSF faculty have implemented major steps toward the institutionalization of training. A series of new formal courses was developed and the curriculum has been enriched. Several courses have reached the stage of having structured syllabi with regular annual and biannual scheduling, and they attract a wide range of students and fellows from UCSF, the University of California at Berkeley, and other campuses. During the third cycle of Pew funding (1992 to 1995), a major part of the UCSF program's mission was to expand the support bases by attracting new sources of funding and midcareer fellows who were able to bring stipend support as well as pay the program for the cost of training.

The Pew doctoral program at Brandeis has enabled the Institute for Health Policy to broaden the health policy specialization in the Heller School curriculum. As of 1993, at least four courses developed through Pew funding were institutionalized at Brandeis (Advisory Board meeting, February 18, 1993). Because of the excellence of the Heller School's PhD program and the national reputation of the Pew doctoral program, many more students who are interested in careers in health policy research are becoming attracted to the program (Raskin et al., 1992). However, since the Pew

program funded only six students per year, many of these applicants have accepted the Heller School's offer to matriculate in the doctoral program and specialize in health policy.

Because of this expanded interest in health policy, the institute has developed the Institute for Health Policy Fellows Program to help support some of these strong doctoral candidates. Selected students are required to follow the health policy core curriculum and participate in the weekly Seminar in Health Policy Research. In addition to the academic requirements, Institute fellows are required to work 1 day a week at the institute in exchange for a $7,500 annual stipend (Raskin et al., 1992).

In 1994, the Institute for Health Policy adopted the strategy of seeking health services research (as opposed to health policy research) training grants from the federal government to complete the transition to alternative funding. The Heller School was awarded training grants from the Agency for Health Care Policy and Research (AHCPR), and in 1995 the Heller School enrolled its first fellows in this program. The health services research training program's curriculum is very similar to that of the health policy research training program's curriculum, and the Heller School faculty believe that their success with the Pew programs greatly assisted them in receiving the AHCPR grants.

The Pew program for corporate midcareer people that was housed at BU has been maintained; however, with the retirement of its director in 1996, its future is questionable. This was the only program of the joint BU-Brandeis program that was maintained at the BU campus, but this is less a result of discontinued funding and more a conscious decision by the university.

According to Al Williams, the experience of the RAND/UCLA Pew program represents the real test of whether or not the Pew programs can be maintained after funding ends. The RAND/UCLA programs were defunded after the second funding cycle for the reasons discussed earlier. Although the schools were unable to maintain the midcareer program without support for the fellows, the Pew doctoral program in health policy studies became part of the regular doctoral program at the RAND Graduate School. RAND continues to train health policy fellows by using a multidisciplinary approach and with new funding streams. Furthermore, the success of the RAND/UCLA program has spawned new

health policy graduate training programs. Joe Newhouse, who was on the faculty at RAND, built upon the RAND Pew experience in his new PhD program at Harvard University. The University of Minnesota has also developed a stronger policy focus over time, and it may have been influenced by the Pew programs as well. Leighton Ku stated that the Johns Hopkins University's health policy program, a well-established program, has grown tremendously since the beginning of the Pew programs. Although he stated that he had no evidence that the Pew programs specifically influenced that growth, Ku speculated that some of the newer programs have likely been modeled on the Pew experiences. He also discussed the fact that the University of California at Berkeley may have been influenced somewhat by the Pew programs. Two former Pew fellows, James Robinson and Helen Schauffler, are teaching at UC-Berkeley and have carried over some of their graduate training experiences.

Al Williams stated, "Imitation is a form of endorsement," and Dan Rubin agrees. Rubin asserts that, as a result of the theory of innovation diffusion, the prestige of the schools involved with the Pew programs and their subsequent successes with the innovative, nontraditional approaches to health policy training may have encouraged other universities to emulate their programs.