Part IV.
Summarizing the Legacy: Some Conclusions and Thousand for the Future

The best investment around is the professional school.

—Peter Drucker

Bottom line: there is still a real need. I do not think the job is done.

—Hal Luft

The greatest thing about the Pew program is that it got people excited.... Wow, we can change the world!

—Linda Simoni-Wastila

This fourth and final section of the report attempts to draw the many loose threads together. Based on archival work, the external evaluations, the 25 interviews and narrative accounts, the roundtable discussions and focus groups, and the authors' observations, the Pew Health Policy Program (PHPP) evolved from a unique, ambiguous idea to a highly successful, highly sought after, and well-respected academic program. The demographics are mesmerizing: hundreds of graduates have been placed into a wide variety of health policy jobs and they are now university professors, researchers, consultants, federal and state policy makers, and health care managers.

The Pew Health Policy Program evolved from a unique, ambiguous idea to a highly successful, highly sought after, and well-respected academic program.

Some questions remain. From a strategic standpoint, what are the elements of successful programs? How are the collective accomplishments summarized? What lessons have been learned about niche educational programs in health policy? Where do we go from here?

Part four of this report addresses these questions in four sections. The first section develops (with the aid of an analytic framework) strategic guidelines to help foundations and



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program Part IV. Summarizing the Legacy: Some Conclusions and Thousand for the Future The best investment around is the professional school. —Peter Drucker Bottom line: there is still a real need. I do not think the job is done. —Hal Luft The greatest thing about the Pew program is that it got people excited.... Wow, we can change the world! —Linda Simoni-Wastila This fourth and final section of the report attempts to draw the many loose threads together. Based on archival work, the external evaluations, the 25 interviews and narrative accounts, the roundtable discussions and focus groups, and the authors' observations, the Pew Health Policy Program (PHPP) evolved from a unique, ambiguous idea to a highly successful, highly sought after, and well-respected academic program. The demographics are mesmerizing: hundreds of graduates have been placed into a wide variety of health policy jobs and they are now university professors, researchers, consultants, federal and state policy makers, and health care managers. The Pew Health Policy Program evolved from a unique, ambiguous idea to a highly successful, highly sought after, and well-respected academic program. Some questions remain. From a strategic standpoint, what are the elements of successful programs? How are the collective accomplishments summarized? What lessons have been learned about niche educational programs in health policy? Where do we go from here? Part four of this report addresses these questions in four sections. The first section develops (with the aid of an analytic framework) strategic guidelines to help foundations and

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program academic institutions to implement niche educational programs. The second section highlights the lessons for other academic institutions learned from the Pew experience. The third section summarizes (in a nonscientific way) impressions of the collective accomplishments and program impacts. To accomplish these three final tasks, the focus is shifted away from the voice of the participants to a more conceptual plane that makes sense of the Pew legacy as a ''lived experience." As Hal Luft, Stuart Altman, Marion Ein Lewin, and others have said, we have learned how to do this, and this work needs to continue. Therefore, it is appropriate for this section to consider the future not only in terms of knowledge creation and vision but also the degree of future participation in policy making and policy training. It is unlikely that market forces will mobilize the loosely coupled community of Pew scholars into an active network. There is a need for a few leaders in the group to create a strong organization. The fourth, and final section concludes by offering the Pew fellows thoughts about a path for the future. A STRATEGIC FRAMEWORK FOR ANALYZING THE PHPP EXPERIENCE: BASIC AND INTEGRATIVE ELEMENTS This section analyzes the PHPP from a strategic standpoint by employing a analytic framework developed by Heskett.1 According to Heskett (1986) there are four basic elements for success in implementing any service. They are targeting markets, well-defined service concepts, focused operating strategies, and well-designed service delivery systems. Heskett also argues that the four basic elements mentioned above are mediated by three integrative elements: positioning, leveraging, and integrating the operating strategy with the service delivery system. In fact, each of the PHPP sites spent the last dozen or so years honing health policy programs around these basic and integrative elements. Each of the sites assembled its programs around the four basic elements for strategic success (Heskett, 1986). Sites targeted internal (faculty) and external (fellows) "market" segments and focused on understanding their needs. Each site also carefully crafted a distinct educational service concept in terms of the results that they could produce for fellows and faculty. A third basic element during implementation was developing a focused operations strategy, and the fourth element was designing a system (pedagogy and methodology) for providing educational services. 1   This section is based on the framework developed by James Heskett (1986) in his book Managing in the Service Economy.

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program Each program eventually developed a distinct market niche in health policy by positioning itself to serve faculty, students, and the health policy world. Irrespective of how programs position their educational concept, they must leverage their activities so that the education is valued by the fellows. This was especially important for the midcareer and on-job/on-campus students. Finally, all of the programs in varying degrees, had to integrate and coordinate their operating strategies with their other educational programs (delivery systems) to insure a high quality education at reasonable costs, an engaged faculty, and internal consistency (Heskett, 1986). These ideas are portrayed in Figure 2. Figure 2. Developing Niche Education Programs: Basic and Integrative Elements Adapted from Heskett, 1986 DEVELOPING STRATEGIC PROGRAM VISIONS: BASIC ELEMENTS Recruiting Faculty and Fellows: Examples of internal and External Targeting To launch an educational program, there is a need to do internal faculty recruitment and external fellow recruitment. There is a synergy between the two types of recruitment. To have a successful program, a strong and committed core faculty had to be enlisted and deployed. To attract a strong faculty, excellent students are needed. Involving faculty members in new activities is difficult, requiring resources and resourcefulness. Two examples illustrate the idea. At the University of Michigan program, they found

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program that running an off-site weekend program required obtaining an "up-front" commitment of faculty willing to teach adult learners on Saturdays and Sundays for 5 months of the year. Traditional ways of hiring faculty did not work, because a new breed of faculty member was needed. According to Richardson (1990), "the faculty selected to teach this group of adult learners" had to be carefully screened and oriented because many Pew fellows were experts in their work domains. Over time Michigan learned how to develop and deploy an outstanding faculty. At the University of California at San Francisco (UCSF), for example, the problem of engaging the faculty required "selling them" on the benefits of working with postdoctoral fellows. When the program began attracting exceptional Pew fellows, a committed faculty quickly surfaced. RAND/University of California at Los Angeles (UCLA) was able to build on the strengths of the faculty at each school to create a singularly rich training program. The faculty selected to teach this group of adult learners had to be carefully screened and oriented because many Pew follows were experts in their work domains. PHPP averaged students with 12 to 14 years of work experience before they came to the Pew program. Targeting excellent fellows to keep the faculty interested and involved was not unique to UCSF but occurred at every site. There is strong evidence that all of the program sites managed to attract high-quality fellows from a strong applicant pool. Throughout the program, the outside evaluators found the quality of the fellows to be "very strong"; moreover, there were noticeable improvements in the quality of fellows from 1982 to 1985, 1985 to 1989, and 1989 to 1994. PHPP averaged students with 12 to 14 years of work experience before they came to the Pew program. The postdoctoral fellows were PhDs, and clinicians. The predoctoral fellows often had Master's degrees or professional graduate degrees from excellent schools. In doctoral and postdoctoral programs, the attrition rates are surprisingly high (Bowen and Rudenstine, 1992). PHPP hypothesized that attrition may be related to the ability to recruit high-quality students. Therefore, the Pew program focused on "quality students" along two dimensions: (1) finding the best and brightest candidates and (2) finding people motivated to complete these challenging programs. Recruitment of high-quality, highly motivated fellows was not the only recruitment characteristic. Each program also had to discover ways to single out groups of applicants (i.e., student segments) with common characteristics whose educational needs could best be met by a particular program in terms of the results they could deliver. So, in recruiting fellows, each program had to learn how to screen for fellows motivated and capable of completing the degree and who fit well with the educational services that the program could deliver.

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program Discovering ways to single out segments of applicants was not a trivial problem for a niche program in health policy. The Michigan program, for example, was a nonresidential doctoral program in health policy for working people. This meant locating prospective fellows who wanted to pursue a full-time doctoral program without giving up their full-time professional lives. To increase the chances of locating highly qualified fellows, schools need to select from a large applicant pool. Recruiting from a national (as opposed to a local) market yields a larger pool. Michigan learned that its program had great appeal to high-profile, midcareer professionals from government and health care management, which offered a large, national segment to draw from. Recruiting Michigan fellows from this pool, however, was tricky because Michigan fellows had to be good at both work and school. To complete a doctoral degree, Michigan had to identify fellows who could master the craft of being a good student. By and large, great leaders are not great students; moreover, older students do not do well on standardized tests. To insure that fellows who were selected would complete the program, Michigan used college grades, past work accomplishments, and Graduate Record Examination (GRE) scores as criteria for final selection. To increase the chances of locating highly qualified fellows, schools need to select from a large applicant pool. Recruiting from a national (as opposed to a local) market yields a larger pool. Each of the programs found different ways to group applicants into segments with common characteristics. At first, Boston University (BU)/Brandeis sought people who had had some governmental or health care management experience and who wanted advanced training in health policy. At BU, the Center for Industry and Health Care aimed at increasing the involvement of the business community in health policy by establishing corporate coalitions, so they targeted students who were planning careers in business and public service. Over time, Brandeis singled out people who were interested in issues of social justice for vulnerable populations-violence, people with AIDS, chronic disease populations, veterans, and so on. The UCSF program targeted a high-quality national pool, consisting of a mix of applicants who wanted advanced training in research: MDs with specialties in primary care and preventive medicine and PhDs (in social science and other areas) from academia. An Evolving Educational Service Concept To recruit bright and motivated fellows who are willing to leave their jobs and careers to go back to school and to

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program attract a dedicated faculty willing to allocate their time and attention, an educational service concept must be defined. This educational concept must be defined not as offering a degree or a high quality educational experience but in terms of opportunities and potential advantages for the fellows and the faculty. At the outset of a program, it is rare to have clarity in an educational concept; concepts evolve. The program had a commitment to excellence and ability to take corrective action when the external program evaluations uncovered weaknesses. In 1981, the staff at the Pew Charitable Trusts believed that there was a lack of depth and breadth in health policy research, analysis, and management, and they believed that multidisciplinary educational concepts might address this problem. Each of the sites offered variations on the traditional educational concept: providing a high-quality educational experience for students. One key result area, implicitly promised by the Pew Program was in an area in which graduate education fails to achieve, academic survival and success. Over time, the programs sharpened their definitions of these concepts. In part, this occurred as a result of the programs, commitment to excellence and ability to take corrective action when the external program evaluations uncovered weaknesses. Defining the concept became an evolving process, in which programs learned the results that can be achieved and they adapted the current concept in that direction. UCSF initially offered three types of 2-year fellowships: predoctoral, postdoctoral, and management. The predoctoral and management programs, although highly successful, were discontinued so that resources could be concentrated on developing a strong postdoctoral program. The educational concept was a multidisciplinary postdoctoral health policy program in which fellows worked with faculty on research projects. The faculty would commit to an active involvement with fellows as mentors and colleagues. Faculty would participate in annual fellow reviews and mentorship meetings. The BU/Brandeis program initially contained two elements: a Pew scholars program and a Pew fellows program. The Scholars Program was an on-campus, post-master's, multidisciplinary accelerated doctoral program which aimed at reducing the time, cost, and incompletion rates to produce educated individuals effective in the health policy system. The Pew fellows program aimed at mid-level and upper-level managers and trained them in health care management and cost-containment strategies. Michigan offered a unique, nonresidential 3-year doctoral program in health policy. The central educational con-

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program cept was that fellows could get their doctorates while keeping their jobs. The OJ/OC doctoral program provided 20 4-day sessions and two 4-week sessions. Upon completion of the program, fellows received a doctor of public health degree. Each of the program developed a strong core curriculum that taught students to appreciate and synthesize the clinical, sociological, political, economic, and behavioral implications of the policy process. In 1982, a joint venture between RAND/UCLA offered a 3-year interdisciplinary program in which fellows could obtain a PhD in policy analysis at RAND or a PhD in several different areas, such as public health and epidemiology at UCLA. The program also had a 10-session lecture discussion series to introduce medical students to policy issues and a 1-year nondegree policy career development program.2 A Focused Operating Strategy Heskett (1986) argues that that to deliver on the promises implicit in the educational concept (while achieving the internal operating goals) a focused operating strategy is needed. This especially makes sense for an educational program since faculty and other resources are so scarce. So, to develop into a high quality educational services program, the PHPP leadership had to concentrate its attention in a few strategic areas, which included: developing a strong curriculum with a well-integrated learning sequence, the deployment of faculty, the creation of a small "service-oriented" organization, and the control of costs. Developing a Strong Curriculum That Ensures Academic Success and Survival The multidisciplinary nature of the curriculum was mentioned throughout the interviews and evaluations as one of the greatest strengths of PHPP. In each of the programs, the social sciences, economics and clinical "disciplines" began to be seen as "tools" to help policy makers solve problems. Each of the programs developed a strong core curriculum that taught students to appreciate and synthesize the clinical, sociological, political, economic, and behavioral implications of the policy process. (The actual curriculums are in Appendix B.) For example, one of the major strengths of the Brandeis program was its broad multidisciplinary approach. The doctoral program required course work in the basic social sci- 2    By 1991, it was decided that three educational concepts should be maintained: postdoctoral education in a policy-rich environment, an accelerated doctoral program, and a doctoral program for people with full time jobs.

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program ences, statistics, research methods, and policy analysis and advanced course work in health policy (Issues in Health Policy; Health Care Organization and Politics; Social, Ethical, and Legal Issues; and Health Economics). At least one course in special populations was also required. However, to ensure academic survival and success in postdoctoral work, midcareer programs, and the accelerated doctoral program, new mechanisms were needed. The dissertation seminars at BU/Brandeis and Michigan became a pedagogical mechanism not only to help students structure and manage the dissertation process early on but also to provide the psychological support to build and connect the fellows into a committed group of mature scholars. An innovative mechanism was needed to get students to start thinking about a dissertation on the first day of the program. The dissertation seminar also became an opportunity to integrate the multidisciplinary curriculum. The dissertation seminars at BU/Brandeis and Michigan became a pedagogical mechanism not only to help students structure and manage the dissertation process early on but also to provide the psychological support to build and connect the fellows into a committed group of mature scholars. The promise of survival and success in a postdoctoral program also required a strong core curriculum. In addition to developing a strong health policy curriculum, UCSF developed a weekly health policy seminar, writing seminar, and journal club to sharpen the fellows' communication skills. These seminars became mechanisms for bringing fellows closer to researchers and policy makers, while providing psychological support and assisting in their "socialization" in health policy. To help ensure academic survival, RAND/UCLA employed an innovative strategy that included an early dissertation focus with an research apprenticeship model. Not only did this innovation get students to experience real research and to develop professional relationships with faculty; these projects often turned into dissertations. Faculty Deployment To meet the fellows' expectations for an exceptional education, there was a need to provide adequate contact with faculty. At UCSF the operating strategy focused on a highly structured learning sequence and a model of mentoring. The mentoring at UCSF worked well. The model required each student to work with two faculty members in an apprenticeship model. Pew, through its habit of systematic program evaluations "prodded," the programs to develop more structure. All of the Pew programs compensated the faculty for their contribution to the program in one of two ways:

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program (1) remuneration for time spent or (2) the opportunity to be a mentor for and collaborate with some excellent fellows. Although money alone does not motivate faculty, asking faculty to teach additional courses in health policy without pay ''can be difficult." At UCSF, the model was very faculty intensive, with one or two faculty mentors assigned to a fellow. Hal Luft said that getting fellows involved with faculty research did work. Faculty liked working with Pew fellows, who were seen as good colleagues. Although faculty were not compensated with salary, the faculty were motivated by intrinsic rewards, such as the value of working with a good research assistant. Changing Behavior Patterns via Service-Oriented Culture The organization of most graduate schools offering predoctoral and postdoctoral education is a hierarchical, loosely coupled set of rituals and routine activities that more or less accomplish the task of enrollment management. The premise of most graduate programs is that given enough time, students will find their way around. They will link up with faculty, select the right courses, and connect with interesting and important research. The process is haphazard, and not student centered. Faculty liked working with Pew follows, who were seen as good colleagues. For the PHPP to work, people had to go out of their way to help students connect with faculty and offer greater flexibility when problems arose. Not every program succeeded in creating a service oriented culture; in fact, fellows complained about the need for more attention. But every program did create a small, effective staff usually involving one faculty member as program director, one or two key faculty, and one or two support staff to serve faculty and fellow needs. Program champions also emerged at each site. Cost-Effectiveness Although every program had to control costs, because of its unique service concept and national targets, Michigan had to focus on controlling costs. Since Michigan chose to target fellows from a national (not local) pool, the program managers had to control (1) travel costs, (2) communication costs, and (3) the massive investment in regular and adjunct faculty, all within the regular tuition structure. Therefore, the Michigan program not only managed costs but also became cost effective.

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program Developing an Educational Program into a Well-Designed Delivery System To offer an educational program, facilities must be designed to work well with the operating strategy. Programs must consider the important characteristics of the educational program as a service process. In fact, the development of an educational service delivery system was an important feature of each program's evolution. Explicit consideration had to be given to the role of key people, physical layout, and changing procedures. Role of Key People To deliver on the strategy of a service-oriented culture, each of the programs needed an incredibly dedicated program staff to serve faculty, students and program leadership. At RAND/UCLA, Kate Korman was cited as "an indispensable linchpin with respect to tying together diverse interests and activities." Throughout the interviews, Marion Ein Lewin, Steve Crane, Ted Benjamin, and David Perlman were all mentioned as being extraordinarily supportive. Physical Layout Every program created a small, effective staff usually involving one faculty member as program director, one or two key faculty, and one or two support staff to serve faculty and fellow needs. Program champions also emerged at each site. Placement of people in offices (or the lack of office space) affects people in a number of ways. Physical layouts are an indicator of social distance and membership (Schein, 1985). The allocation of space (crowding students into rooms, size of the office, and quality of the furniture) symbolizes the rank of the people and affects their feelings of inclusion. Finally, organizational research has shown that the probability of weekly interactions drops to zero when people are more than 40 meters apart. Being on different floors (or buildings) is like being in different cities. Each of the programs was challenged by the physical locations of the fellows, especially in relation to faculty. Some programs, such as Brandeis, were never able to offer their fellows office space, which undermined communication. To compensate, Brandeis provided weekly dissertation seminars and biweekly colloquia which were opportunities for concerted group action. In a bid to promote communication among off-site participants, Michigan used electronic communication. UCSF found that to the extent that fellows were located off-site, interaction with faculty and other fellows was hindered. The situation improved when office space became avail-

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program able. They found that when they brought the fellows into office space with the faculty, mentoring relationships improved. Changing Rules and Procedures Although there was a clear need for many more people broadly trained in health policy, each program had to focus its attention on what it could do differently from other graduate programs in health services. At Michigan, they discovered that the OJ/OC weekend mode resulted in the accumulation of many incompletes because of the working students' inability to complete term papers. Over time, these incompletes became obstacles to completing the dissertations. The remedy was the creation of new procedures aimed at discouraging major papers due at the end of a course. The procedures encouraged the faculty to assign deliverables over the terms rather than a major paper at the end of the term. As a result of those procedures, the rate of incompletes dropped. Incompletes were only given if a serious personal problem (such as a death in the family) had occurred. The procedures helped to reduce the risk of low completion rates. To decrease feelings of elitism and to increase integration among Pew and non-Pew students, BU/Brandeis changed the rules to allow any student into the dissertation seminars. To help fellows complete the program in 2 years, Brandeis changed its procedures to allow Pew fellows to take their qualifying examinations in the first year. At UCSF, formal procedures were developed to improve faculty-fellow interactions and mentoring. Incoming fellows received a detailed orientation to the faculty. Fellows had scheduled meetings with their primary research advisers. Finally, guidelines listing goals and expectations of the faculty mentors and fellows were developed. DEVELOPING STRATEGIC PROGRAM VISIONS: INTEGRATIVE ELEMENTS Positioning Each Program Positioning in education means identifying academic needs that no one is serving. Before each site could position itself it needed a deeper understanding of: (1) what other health services research doctoral programs were offering, (2) the needs of potential employers in the policy world and (3) how to serve the educational needs of the students. Although there was a clear need for many more people broadly trained in health policy, each program had to focus its attention on what it could do differently from other graduate programs in health services. Most programs focused on teaching policy analysis and advanced research skills from a single perspec-

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program fessional positions." In the words of one fellow, the program helped people to break "through the glass ceiling for non-PhDs." In the words of another fellow, the program "empowers individuals in their workplace.'' Success Requires Building on Core Competencies The success of each program was a result of adopting a core competence perspective that required building on the strengths of each school. For the competency approach to work, the program leadership had to perform three basic tasks: (1) identify the existing distinct competencies, (2) improve and develop the competencies, and (3) deploy the competency in novel ways. During the first year of the program every fellow went to Washington, D.C. The fellows were exposed to the world outside of their particular program. They discovered that there were hot issues and policy wonks, gurus, and experts attached to those issues. The creation of new educational niches in health policy research requires the development and deployment of at least three competencies: (1) expertise in multidisciplinary policy research, (2) flexible program development, and (3) "faculty focused on students" as a service concept. Turning a new pre-or postdoctoral curriculum into an effective educational program demands an ability to understand the area of study and the educational issues, anticipate student needs, and direct resources to meeting those needs. The first competency is having a strong knowledge base in health services research and real-world expertise in the health policy sciences. The key indicators of knowledge and experiences are faculty publications, faculty involvement in major policy issues and initiatives, and a strong faculty record in basic and applied funded research. The second competency for niche creation is "flexible program development." When compared with the lengthy time periods for completion of traditional pre-and postdoctoral programs, niche programs require shorter time periods, higher completion rates, and an ability to accommodate diverse student interests against diverse faculty research interests. By and large, professional educational programs are built around degree concepts rather than the core competencies of the school. To be successful, the faculty must be focused on the needs of students. Faculty must know what students want, the faculty must accept that student expectations are reasonable, and performance standards must be set. A strong message must go out in the form of a service guarantee everyone (faculty and staff) at the program, without exception, will have a service-oriented attitude (Heskett, 1986).

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program Institutionalization Takes Time and It's Paradoxical Funded programs have a life cycle, and death is an inevitable feature. In the case of PHPP, death may not be inevitable. The longer a program survives in a school, the more institutionalized the program becomes—that is, the greater the likelihood that it will survive in the future. Institutionalization means acquiring both stability and status within a school. The degree to which an educational program has achieved institutionalization is defined partly by its age and depends on the degree to which flexibility, autonomy, and coherence have been achieved. Fellows felt that they had acquired the knowledge and skills to analyze, make, and implement decisions affecting the policy system. According to Kimberly (1981), "institutionalization is that process whereby new norms, values, and structures become incorporated within the framework of existing patterns of norms, values, and structures" (p. 31). So institutionalization takes time. In the case of the PHPP by the tenth year the programs were just beginning to understand how to make a niche educational strategy in health policy a success. However, innovation and institutionalization often work at cross purposes. In creating new educational programs, success can be paradoxical (Kimberly, 1981). The problems of getting started and the problems of institutionalization lead to very different attention structures. Being new and different creates short-run opportunities for niche programs because commitment leads to experimentation and tolerance for mistakes (Kimberly, 1981). Eventually, institutionalization leads to formalization and stability, but "diminished innovativeness" (Kimberly, 1981). The Best Lessons Are Often Unanticipated As is often the case, program evaluation measures the success or failure based on accomplishing predetermined goals. What's often lost in this type of analysis is an appreciation of the unintended consequences of programs. By observing and analyzing these unanticipated consequences one can also learn some powerful lessons. At Michigan, for example, the faculty's experience with the off-campus education concept had several positive, although unanticipated consequences for faculty job satisfaction. The faculty found it very rewarding to have these part-time doctoral students coming to the campus to learn and then go back to their full-time jobs to apply that knowledge. Another unexpected consequence was the spill-over effect of the Pew experience to other corners of the school. The

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program financial support from the Pew program made the faculty's policy research and teaching materials available to other health policy students in the school. HIGHLIGHTING COLLECTIVE ACCOMPLISHMENTS AND PROGRAM IMPACTS For many reasons, understanding the collective accomplishments of any new educational program are difficult. Program consequences or impacts are often difficult to link (causally) to specific educational activities, because outcomes are ''delayed, confounded, and negotiated" (Weick, 1995). Often people can only understand what they are doing many years after they have done something (Weick, 1995). If one were to fast-forward 10 or even 20 years, what would the Pew legacy be? One simple, but inadequate, way to assess accomplishment is to count the final outputs produced or services provided. The program produced hundreds of people who have contributed to knowledge creation by publishing hundreds of publications and who will contribute to future health policy visions. Consequences will continue for many years in the future. Institutionalization means acquiring both stability and status within a school. The degree to which an educational program has achieved institutionalization is defined partly by its age and depends on the degree to which flexibility, autonomy, and coherence have been achieved. However, innovation and institutionalization often work at cross purposes. Another measure of accomplishment is to track job changes over time. Table 6 lists various health policy fields and Pew fellows' migration paths before and after attending the program. Although the entire program has helped to place many people into a variety of health policy positions, each of the program sites has had different impacts. The BU/Brandeis and Michigan programs saw significant shifts into academic positions (from 21 percent to 37 percent and from 8 percent to 28 percent, respectively). BU/Brandeis, University of Michigan, and RAND/UCLA had greater shifts into research jobs (from 0 percent to 10 percent, 4 percent to 8 percent, and 9 percent to 20 percent). UCSF had some shifts to consulting (15 percent to 8 percent) and government (12 percent to 17 percent). UCSF and Michigan had small shifts into health care management (from 13 percent to 17 percent and 22 percent to 26 percent, respectively). When one considers the range of positions occupied by Pew fellows today, one finds that there are few problems in the domain of health policy issues that lie out of some Pew fellows' reach. In one sense the program has fostered an invisible health policy college with a network of potential allies. It is presumed that Pew fellows will continue to improve the formulation and implementation of health policy. Whether this

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program Table 6. Migration Paths of Pew fellows by Program Site Before and After Attending the Program   BU/Brandeis UCSF Michigan RAND/UCLA Total (all programs) Field Before After Before After Before After Before After Before After Academia 14/67 (21%) 25/67 (37%) 34/83 (14%) 38/83 (8%) 4/50 (8%) 14/50 (28%) 23/69 (33%) 23/69 (33%) 74/269 (28%) 98/269 (37%) Consulting 4/67 (6%) 9/67 (13%) 1/83 (1%) 7/83 (8%) 4/50 (8%) 3/50 (6%) 3/69 (4%) 3/69 (4%) 12/269 (5%) 22/269 (8%) Research 0/67 (0%) 7/67 (10%) 2/83 (2%) 3/83 (4%) 2/50 (4%) 4/50 (8%) 6/69 (9%) 14/69 (20%) 10/269 (4%) 28/269 (10%) Government 19/67 (28%) 5/67 (7%) 10/83 (12%) 14/83 (17%) 16/50 (32%) 8/50 (16%) 10/69 (14%) 8/69 (12%) 55/269 (20%) 35/269 (14%) Health Care Management 17/67 (25%) 7/67 (10%) 11/83 (13%) 14/83 (17%) 11/50 (22%) 13/50 (26%) 14/69 (20%) 11/69 (16%) 53/269 (16%) 45/269 (17%) Professional Associations 1/67 (2%) 3/67 (4%) 0/83 (0%) 1/83 (1%) 4/50 (8%) 3/50 ((6%) 1/69 (1%) 1/69 (1%) 6/269 (2%) 8/269 (3%) Other 7/67 (10%) 1/67 (2%) 15/83 (18%) 3/83 (4%) 6/50 (12%) 3/50 (6%) 4/69 (6%) 4/69 (6%) 32/269 (12%) 10/269 (4%) Data Missing 5/67 (7%) 10/67 (15%) 10/83 (12%) 3/83 (4%) 3/50 (4%) 2/50 (4%) 8/69 (12%) 5/69 (7%) 26/269 (9%) 18/269 (7%) NOTES: The data for classes entering 1983 to 1993 are included. All information retrieved from Pew directories corresponding with year of entry The "Other" category, described in greater detail elsewhere, includes fields such as: clinical medicine, communications, and business.

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program potential network is used to help in major health reform or restructuring efforts in the future remains to be seen. It was found that the program affected the program sites, the fellows, the faculty, IOM, the Pew Charitable Trusts, and the health policy world. Each of these will be discussed in turn. Program Sites If one were to fast-forward 10 or even 20 years, what would the Pew legacy be? According to leadership at each of the program sites, the schools benefited in a number of ways. Each program site learned how to organize a curriculum and a faculty that satisfied program faculty, students and external evaluators. Each program site also developed a methodology for the multidisciplinary training of predoctoral and postdoctoral students, and they were able to leverage these programs into larger educational domains. Today UCSF has a health policy specialty that would not have existed without Pew support; it now has seminars and courses that would not have existed otherwise. For example, at UCSF a writing seminar and a health policy seminar that met every week built on the experience of the Pew program to develop an Agency for Health Policy Research training program, and interdisciplinary scholarly work came as a result of Pew. The Institute for Health and Aging would not have existed without the Pew program. The Pew program led to the development of a successful joint University of California at Berkeley/UCSF proposal to the Robert Wood Johnson Foundation for a program in health policy. Brandeis was also able to extend the Pew experience into a successful new predoctoral program in health services research funded by Agency for Health Policy Research. RAND/UCLA found ways to sustain a commitment to the Pew approach to training and educating doctoral students. Finally, Michigan learned that the OJ/OC concept could work in doctoral education. Fellows Each of the programs contributed to the fellows' professional lives in several ways. Many fellows spoke favorably about the education they received. The education helped many fellows to think about health policy in a rigorous way, expanding their knowledge of policy and their research skills. Fellows also claimed that they learned about the role of science and knowledge in political institutions.

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program Some fellows felt that they had obtained the credentials to be more legitimate and convincing advocates. The program also introduced fellows to multidisciplinary education. As a result, fellows claimed to have become better "policy colleagues" by developing a greater awareness, appreciation of, and respect for what their colleagues in other disciplines do. Faculty Pew fellows will continue to improve the formulation and implementation of health policy. Whether this potential network is used to help in major health reform or restructuring efforts in the future remains to be seen. The program also had important impacts on the faculty in every program. UCSF faculty spoke of the Pew experience as "broadening, offering faculty a chance to enrich their work-loads and their professional careers." This enrichment occurred in several ways. Faculty claimed that the Pew courses were fun to teach, often exposing the faculty to challenging and stimulating sessions. Faculty, especially in the postdoctoral programs, wrote many papers with Pew fellows. When faculty worked with Pew fellows, they found that they were affected by them. For example, Pew faculty also learned to be better "policy colleagues" by working with fellows from disciplines other than economics—law, medicine, management, political science, and sociology. IOM IOM also benefited from the Pew fellows, who continue to add a fresh voice to the policy work of the program. IOM has hired Pew fellows and commissioned papers and reports from Pew fellows. The rigorous training received by Pew fellows will continue to serve IOM in the years to come as fellows are called upon to help it deal with contemporary issues. Pew Charitable Trusts For all the reasons cited above, it appears that the Pew Charitable Trusts have also benefited from the program. This program was one of the Trusts' first efforts that was national in scope. The program has been considered a tremendous success by the external evaluators and the health policy world. During the last decade, the effort took on all the characteristics of a "signature program" for the Trusts. Health Policy World One can speculate endlessly on the presumed impacts on the policy world. But the most significant impact of the

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program program has been the placement of highly educated people in strategic health policy positions in virtually every segment: academia, research and consulting, government and public health, professional associations, and health care delivery. Each individual has a unique set of special skills, yet each fellow is trained to understand the multidisciplinary nature of health policy. Each one is trained to translate research findings into policy-relevant and managerially relevant language. Each one is given an awareness of how politics, markets, and organizations works in health policy. Each one has an appreciation of the significance of knowledge in making better policy. Each one is committed to lifelong learning. Between 1991 and 1996 Pew fellows and alumni published about 650 scholarly papers and technical reports covering a wide variety of health policy subjects. Years Estimated Number of Publications4 1991–1992 106 1992–1993 127 1993–1994 159 1994–1995 165 1995–1996 92 Total 649 As one considers the contributions to knowledge made by these publications, one is reminded of Peter Drucker's insight into the central role of the individual: "Knowledge does not reside in a book, a databank, a software program; they contain only information. Knowledge is always embodied in a person; carried out by a person; created, augmented, and improved by a person; applied by a person; taught and passed on by a person; used or misused by a person" (Drucker, 1993, p. 210). According to Drucker, knowledge production is enriching, but the advancement of knowledge requires human action, and that requires defining a role for people. Envisioning a role for Pew fellows in the future is the subject of the final section. ENVISIONING A FUTURE POLICY ROLE FOR PEW FELLOWS In the coming years, health policy will continue to be a high-drama, high-stakes, high-social-purpose field. With a growing underclass of vulnerable subpopulations, and with managed care and competition causing strategic re-orientations, 4    These estimates are based on self-reported figures given to the Program Office at IOM. It is likely that the actual number of publications is 20 to 40 percent higher.

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program alliances, and changes in finance and payment, the world of health policy becomes ever more complicated. Health policy makers will be central characters in this drama. Today, health services research and health care management have emerged as new disciplines capable of making important contributions to health policy (Ginzberg, 1991). A recent study found that the demand for highly qualified health policy researchers would continue to exceed the supply (Field, et al., 1995). If the health care system needs more health policy scholars, doctoral education and postdoctoral programs focused health policy will remain important social investments. More well-educated people will be needed. The most significant impact of the program has been the placement of highly educated people in strategic health policy positions in virtually every segment: academia, research and consulting, government and public health, professional associations, and health care delivery. It is hoped that the Pew fellows will become the self-appointed "trustees" of this new health policy world.5 Amidst the growing complexity, neither government nor the delivery system can afford to shoot from the hip. The health care delivery system of the future will be assigned responsibility for vulnerable populations. These organizations must develop an unprecedented capacity to learn—which includes understanding the issues and framing them as policy questions, discovering new theories, developing and deploying them, and evaluating the outcomes on the populations. To paraphrase a quote made by a hospital chief executive officer in the midst of this sea change: "Today the familiar paths no longer seem to work. This time we will have to think our way out of this situation; because we can no longer simply buy our way out." —Personal communication with a Boston-area hospital CEO [emphasis added is the authors'] That quote suggests that innovations in health policy will require abandoning old ways. Thought needs to be given to the knowledge base for policy makers, the need for better-educated policy makers, and the need to get off the treadmill to begin generating new ideas. As one reflects on these lessons, one can see the need for health reform to be greater than ever. However, the future world of health policy does not need a brilliant leader with a vision. Health reform can be characterized as a massive construction project in need of academically oriented, meta-policy architects:6 Policy architects who are willing to advocate the social values that underlie desirable consumer and producer behaviors (responsibility, social justice, caring, and so on). Academically-oriented architects who are continually sharing their deep understanding of the evolving needs 5    Borrowed from former Ambassador to Great Britain, Kingman Brewster, who asked, "Who are the Trustees of the future." 6    The role of meta-architects was first described by Charles Handy (1990).

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program of patient populations and the nature and capacity of the delivery system. Practice-oriented policy architects who are producing knowledge and vision for the health policy world. For several reasons, there is an opportunity for a collective rather than an individual role for Pew fellows. First, the difficulty of concentrating attention on more than a few policy issues at a time is a severe limitation for the health policy world. If, for example, one were to rely on the media to frame the health issues, the end result would be a faddish and transient agenda rather than an intellectual agenda (Simon, 1983). Without a group whose mission is to systematically identify and solve policy problems, attention drifts away from issue A to issue B to issue C. Innovations in health policy will require abandoning old ways. Thought needs to be given to the knowledge base for policy makers, the need for better-educated policy makers, and the need to get off the treadmill to begin generating new ideas. Second, since most complex health policy issues are dynamically complex, single experts can have damaging effects on health policy. Rather than contribute to the pool of knowledge, experts and gurus often become symbols representing positions taken (pro and con) for any given solution. Knowledge can take generations to assimilate, but experts become "creatures of media machines" (Rieff, 1972). We prefer groups of people working together in teams because groups (when compared with individuals) have more information to share, have greater breadth and depth of experiences, and have the capacity to use multiple lenses. Therefore, by enlarging the pool of ideas, groups have the potential to produce higher quality and more effective policy decisions. Third, Herbert Simon continually reminds us that each individual only sees the part of the world in which he or she lives and tends to aggrandize the importance of that part (Simon, 1983). Since Pew fellows are located throughout the policy world, no individual can see the whole world. Each fellow sees that part of the world that he or she knows. There is a need to share information. Fourth, from a health policy standpoint, the delivery of health care depends on expectations about the future and the market's reactions to those expectations, and these are difficult to study. So, to understand the alternatives and their consequences, policy research and development and demonstrations in health care need to continue. For all these reasons there is a need for sustained attention from Pew fellows as a collective group. Today there is a living, breathing network of more than 300 Pew fellows with a significant opportunity to influence health policy development. To be effective in health policy,

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program there will be a continued need for Pew fellows to bridge three cultures: (1) academia and intellectual research, (2) polity, and (3) health care delivery systems. The intellectual community must continue to focus on language and ideas, the polity must focus on the realpolitik allocation of values, and the delivery system must focus on people and services. The emotional and intellectual energies of those fellows will continue to make a tremendous difference if actions such as the following are taken:7 Today there is a living, breathing network of more than 300 Pew fellows with a significant opportunity to influence health policy development. Translate and frame academic knowledge into policy-relevant and managerially relevant language. Develop mechanisms to enlist new health policy fellows by: (a) establishing a new professional association or (b) establishing a virtual or invisible college, perhaps at the original program sites. Launch a "virtual Pew program" in which Pew fellows will continue to share their collective knowledge, ideas, and other resources through distance learning technology, electronic networking, formally organized conferences and colloquia, and possibly, new policy publication outlets. Maintain current information about Pew fellows and their locations in academia, federal and state government agencies, corporate organizations and the delivery system via a home page on the World Wide Web, newsletters, and phone directories. Continue to hold annual reunions aimed at developing commitments to an intellectual agenda. Continue to read, synthesize, contribute to, and disseminate the literature. Continue to validate and refresh the policy-making inputs—the data, information, knowledge, and theories. Track fellows global presence in health policy by developing a capacity to disseminate local knowledge through the Pew network and infrastructure. For these actions to happen, there will be a need for a leadership group to emerge, with leaders willing to find common values and shared expectations and instill a deep respect for those areas where differences exist. These leaders who will not merely project the current system incrementally into the future, but will play with new ideas, discover new zones of study, and look upon change with what Karl Weick calls "disciplined imaginations." This report has tried to capture some of the feelings and some of the spirit of the Pew Health Policy Program in its attempt to stimulate multidisciplinary education in health policy. By writing about this experience, it is hoped 7    Some of these ideas come from Hamel and Prahalad (1994) and Handy (1990).

OCR for page 95
The Lessons and The Legacy of the Pew Health Policy Program that something has been done to encourage people to think about the future. Health policy remains a vast subject, with an ability to affect human lives in extraordinary ways. The only possible conclusion that can be reached is that health policy has just begun to be studied. We believe that Pew fellows will continue to play a critical role in leading the effort.