11
Implementation and Dissemination of Prevention Programs

Part II illustrates the substantial progress made in prevention science since 1994. It describes numerous efficacious or effective prevention programs (Chapters 6 and 7), as well as the cost-effectiveness of many of these programs (Chapter 9). It also demonstrates numerous methodological advances that increase confidence in the reliability of evidence that provides a strong basis for believing that the mental, emotional, and behavioral health of the nation’s young people could be significantly improved if evidence-based programs and policies were widely used (Chapter 10). Thus far, however, preventive interventions have generally not been widely implemented in schools and communities (Ennet, Ringwalt, et al., 2003; Gottfredson and Gottfredson, 2002; Hallfors and Godette, 2002; Wandersman and Florin, 2003) and have done little to reduce behavioral health problems in American communities (Chinman, Hannah, et al., 2005; Sandler, Ostrom, et al., 2005).

While sustained, high-quality implementation by communities is essential to achieving greater public health impact from the available tested and effective preventive interventions (Elliott and Mihalic, 2004; Glasgow, Klesges, et al., 2004; Spoth and Greenberg, 2005), implementation of existing programs alone is unlikely to be sufficient. Implementation must also include development and evaluation of research-based adaptations of programs to new cultural, linguistic, and socioeconomic groups; evaluation of approaches that have broad community endorsement; and implementation of policies and principles that support healthy development.

This chapter begins with a discussion of alternative implementation approaches. It goes on to review examples of experience with implementa-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 297
11 Implementation and Dissemination of Prevention Programs P art II illustrates the substantial progress made in prevention science since 1994. It describes numerous efficacious or effective preven- tion programs (Chapters 6 and 7), as well as the cost-effectiveness of many of these programs (Chapter 9). It also demonstrates numerous methodological advances that increase confidence in the reliability of evi- dence that provides a strong basis for believing that the mental, emotional, and behavioral health of the nation’s young people could be significantly improved if evidence-based programs and policies were widely used (Chapter 10). Thus far, however, preventive interventions have generally not been widely implemented in schools and communities (Ennet, Ringwalt, et al., 2003; Gottfredson and Gottfredson, 2002; Hallfors and Godette, 2002; Wandersman and Florin, 2003) and have done little to reduce behavioral health problems in American communities (Chinman, Hannah, et al., 2005; Sandler, Ostrom, et al., 2005). While sustained, high-quality implementation by communities is essen- tial to achieving greater public health impact from the available tested and effective preventive interventions (Elliott and Mihalic, 2004; Glasgow, Klesges, et al., 2004; Spoth and Greenberg, 2005), implementation of exist- ing programs alone is unlikely to be sufficient. Implementation must also include development and evaluation of research-based adaptations of pro- grams to new cultural, linguistic, and socioeconomic groups; evaluation of approaches that have broad community endorsement; and implementation of policies and principles that support healthy development. This chapter begins with a discussion of alternative implementation approaches. It goes on to review examples of experience with implementa- 27

OCR for page 297
28 PREVENTING MENTAL, EMOTIONAL, AND BEHAVIORAL DISORDERS tion of existing prevention programs, as well as a number of challenges to implementation. The chapter then describes strategies that can complement the implementation of evidence-based interventions. Next is a discussion of research needed to increase understanding of and support successful imple- mentation. The final section presents conclusions and recommendations for moving implementation forward. IMPLEMENTATION APPROACHES A major implementation issue is the balance between delivering an evidence-based program as developed and adapting a program to meet the specific needs of the community. This section describes three alternative implementation approaches: (1) direct adoption of a specific evidence- based prevention program, (2) adaptation of an evidence-based interven- tion to community needs, and (3) community-driven implementation. Table 11-1 summarizes the advantages and disadvantages of each. These three approaches are not mutually exclusive or exhaustive of all potential approaches. Each requires an active partnership among community leaders, organizations and institutions, and researchers and must address issues of trust, power, priority, and action. The appropriate approach in a given com- munity will depend on its characteristics and priorities and the availability of an existing evidence-based program that matches its needs. Ideally, evaluation is a component of all three approaches to shed light on why a specific approach works in a particular community or how to generalize knowledge about successful implementation to other programs, communi- ties, or institutional settings. Adoption of an Existing Evidence-Based Program A community’s adoption of a specific prevention program involves deliv- ering the program with high fidelity, increasing the likelihood that its impact will be similar to that found in the original studies. Typically, programs have met a specific standard of evidence, often articulated by federal, state, or other external funding sources (Halfors, Pankratz, and Hartman, 2007). Standardized curricula, teaching manuals, or taped media help deliver the program in a manner similar to that used by the original researchers. Gener- ally, there is limited adaptation of the program to the cultural or historical characteristics or the particular interests of the community. Sites typically need sufficient local capacity and resources and technical assistance from the program developers or other certified trainers to ensure fidelity, monitoring, supervision, and sustainability (Elliott and Mihalic, 2004). Both the Nurse-Family Partnership Program and Life Skills Train- ing, considered strong evidence-based programs backed by research findings

OCR for page 297
2 IMPLEMENTATION AND DISSEMINATION TABLE 11-1 Comparison of Three Implementation Approaches Model Advantages Disadvantages Implementation High program fidelity Program may not fit community of an existing needs, strengths, or capacities evidence-based Relatively high likelihood of program achieving intended impact Real-world implementation may differ dramatically from the way Known resources and originally tested requirements for effective implementation Lack of ownership in the program Likely continued funding under Few evidence-based programs have federal and state supported the capacity to provide technical evidence-based prevention assistance and training An evidence-based program may not target outcomes relevant to community Adaptation of an Ownership and high support Key program components may be existing program from community and potentially modified, thereby reducing to meet high adoption outcomes community needs Program more relevant to ethnic, Essential program components not racial, or linguistic characteristics always evident of community Reasonably likely to achieve impact Community- Can develop high community Lengthy period to develop driven acceptance and ownership community awareness, common implementation vision, and program Potential for broader implementation across different Potential for ineffectiveness or organizations and institutions iatrogenic effects within the community Challenges in obtaining funding for Opportunity to empirically sustaining a unique program evaluate the outcomes of programs accepted by the community and use quality improvement methods to enhance outcomes over time

OCR for page 297
00 PREVENTING MENTAL, EMOTIONAL, AND BEHAVIORAL DISORDERS from multiple randomized trials in different types of communities, are being implemented in specific communities using this approach. There is some evidence that they are flexible enough to provide benefit across communities with diverse ethnic backgrounds (Botvin, Griffin, et al., 2001). However, it often takes decades of longitudinal follow-up for a pro- gram to be designated as evidence-based, and the original program may not address the current needs or priorities of communities. Research-based programs rarely can meet the triple challenges of maintaining an active research program, a successful marketing strategy, and a qualified technical assistance and training program. In addition, it may be difficult to repro- duce in the community the level of expertise of staff used to deliver the intervention in the original study. Finally, importing a program may result in a lack of ownership in the community, negatively affecting the ability to sustain the program over time. Given increasing evidence of the importance of community engagement and technical assistance, several models have been developed to help com- munities build the infrastructure needed to identify and implement specific evidence-based programs (see Box 11-1). For example, the Communities That Care (CTC) model leads a community through an assessment process to select specific evidence-based programs. The CTC model strongly discourages BOX 11-1 Models for Community Implementation of Evidence-Based Programs Communities That Care Communities That Care (CTC), a prevention system designed to reduce ado- lescent delinquency and substance use, was built as part of the Center for Sub- stance Abuse Prevention approach to effective implementation (see http://ncadi. samhsa.gov/features/ctc/resources.aspx). It provides a process for communities, through a community prevention board, to identify their prevention priorities and develop a profile of community risk and protective factors. The CTC logic model involves community-level training and technical assistance for three steps: (1) community adoption of a science-based prevention framework, (2) creation of a plan for changing outcomes through a menu of evidence-based programs that tar- get risk and protective factors identified by the community, and (3) implementation and evaluation of these programs using both process and outcome evaluations. Currently, there are 56 available programs that meet CTC’s required standard of evidence.

OCR for page 297
0 IMPLEMENTATION AND DISSEMINATION BOX 11-1 Continued CTC’s theory of change hypothesizes that it takes two to five years to observe changes in prioritized risk factors and five or more years to observe effects on delin- quency or substance use. CTC’s data driven process is being evaluated in multiple steps. The first step, a five-year nonexperimental study with 40 incorporated towns, assessed the degree to which they reported using tested and effective programs. In the next phase, 24 of these communities who had not reported already using such programs agreed to be part of a large randomized community-level trial to test the CTC model (Hawkins, 2006). Early findings from these communities indicate that CTC has positive effects on targeted risk factors and delinquent behavior (Hawkins, Brown, et al., 2008) as well as alcohol use and binge drinking (Hawkins, Oesterle, et al., in press). Longer term follow-up is under way. PROmoting School-community-university Partnerships to Enhance Resilience Model The PROmoting School-community-university Partnerships to Enhance Resil- ience (PROSPER) model (Spoth, Greenberg, et al., 2004; Greenberg, Feinberg, et al., 2007) has devised a system aimed at broad implementation of evidence- based programs designed to support positive youth development and reduce early substance use delivered to rural areas with supports at the local, regional, and state levels. Underlying this system is the building of an infrastructure that supports local ownership and capacity building as well as leadership and institu- tional support (Spoth, Greenberg, et al., 2004). Three groups are involved in the PROSPER partnership model: (1) faculty from land grant universities and affiliated cooperative extension staff, (2) the elementary and secondary school systems, and (3) community agency providers of services for children and families, along with other community stakeholders. The partnership benefited from the existing training and technical assistance infrastructure provided by the Extension System and the U.S. Department of Edu- cation’s Safe and Drug-Free Schools (SDFS) Program. Because the prevention programs in PROSPER are delivered by local practitioners, it focuses on building strong support of the school–local community team, which chooses interventions and is responsible for their implementation. At the state level, researchers work with regional Extension Service prevention coordinators and coordinators from the SDFS Program. These regional coordinators then provide support to local teams of extension agents, elementary and secondary school faculties and staffs, and community interagency coalition members. The long-term goal is to provide infrastructure support as well as direct assistance to sustain effective, empirically based programs in communities. This implementation model has national implications, as the Extension Service has more than 9,600 local agents working in 3,150 counties across the United States. The Department of Education has multiple technical assistance centers that support efforts to adopt empirically supported programs that can reduce sub- stance abuse, violence, and other conduct problems in the schools. Furthermore, the SDFS Program currently has coordinators in many schools to facilitate the implementation of such research-based programs.

OCR for page 297
02 PREVENTING MENTAL, EMOTIONAL, AND BEHAVIORAL DISORDERS program adaptation, based on evidence that delivery of evidence-based pro- grams as designed is likely to lead to the most successful prevention efforts. Adaptation of an Existing Program to the Community Adaptation of programs focuses on concerns about community or cultural relevance. A community identifies an evidence-based program that matches its needs, values, and resources and modifies or adopts elements of the program to maximize community acceptance, implementation, and sustainability. Researchers often work in close collaboration with commu- nity leaders to find ways to integrate components of prevention programs in ways that are acceptable and meaningful to the community and to evaluate results. There is long-standing consensus that health promotion and preven- tion programs should be culturally sensitive, along with concerns about whether a given prevention intervention is generic enough to be efficacious and effective with diverse cultures (Resnicow, Baranowski, et al., 1999; Seto, 2001; Woods, Montgomery, and Herring, 2004; Weeks, Schensul, et al., 1995; Hutchinson and Cooney, 1998). Prevention programs must also be mindful of developmental processes, reinforcements of risk behavior, relevant contextual factors, and a population’s unique risk profile (Brown, DiClemente, and Park, 1992). A few studies have shown that making adaptations to different cultural groups while maintaining core elements of programs implemented with fidelity can produce strong results across different cultural groups (Botvin, Schinke, et al., 1994; Botvin, Baker, et al., 1995; Botvin, Schinke, et al., 1995; Reid, Webster-Stratton, and Beauchaine, 2001). However, there is currently no consensus and limited scientific evidence on the key elements that determine the necessary balance between program adaptation and program fidelity. Bell, Bhana, and colleagues (2008) point out that, for an intervention to be culturally sensitive, it must have content that is welcoming to the target culture, contain issues of relevance to the culture, not be offensive, and be familiar to and endorsed by the culture. If a given intervention embodies generic principles of health behavior change, such as aspects that create social fabric, generate connectedness, help develop social skills, build self- esteem, facilitate some social monitoring, and help minimize trauma (Bell, Flay, and Paikoff, 2002), it can usually be adapted to have an appropriate level of cultural sensitivity (Bhana, Petersen, et al., 2004; Peterson, 2004; LaFromboise and Lewis, 2008; LaFromboise, 1995). For example, if going on a spirit quest builds self-esteem in American Indian culture, efforts to build self-esteem in American Indians might best be served by a spirit quest exercise instead of formation of a soccer team (Bell, 2005; see also

OCR for page 297
0 IMPLEMENTATION AND DISSEMINATION BOX 11-2 A Program Adaptation for an American Indian Population An American Indian tribe in the Southwest worked in collaboration with aca- demic researchers to create the American Indian Life Skills (AILS) intervention for the purpose of reducing the factors associated with suicidal behavior (LaFromboise and Lewis, 2008). AILS was found to have a positive impact on American Indian high school students’ feelings of hopelessness, suicidal ideation, and ability to intervene in a peer suicidal crisis situation (LaFromboise and Howard-Pitney, 1993). When used as a comprehensive suicide prevention approach, the interven- tion demonstrated a substantial drop in suicidal gestures and attempts. Although suicide deaths neither declined significantly nor increased, the total number of self-destructive acts declined by 73 percent (May, Sena, et al., 2005). Extensive input was solicited from members of the tribe initiating AILS in order to fit its cultural norms. Key aspects of giving instruction, problem solv- ing, and helping others in that culture were examined. Focus groups members were selected by community leaders to give guidance on intervention content, implementation issues, and intervention refinement. It was believed that suicidal behavior could be attributed to direct modeling influences (e.g., peer or extended family member’s suicidal behavior) in conjunction with environmental influences (e.g., geographic isolation) and individual characteristics (e.g., hopelessness, drug use) that mediate decisions related to self-destructive behavior. Life Skills Training was used throughout the intervention to complement traditional ways of shaping behavior. Each skill-building activity was selected from research supporting best practices for social emotional regulation and cog- nitive skills development, including methods of group cognitive and behavioral treatment. Needed modifications were made to strategies identified. For example, in lessons on recognizing and overcoming depression, the Pleasant Events Schedule (Lewinsohn, Munoz, et al., 1986) was modified to reflect American Indian adoles- cent socialization in the reservation context, renamed “Depression Busters,” and used as the basis for both an intervention activity and a homework assignment. Items such as “talking on the telephone” or “playing a musical instrument” were retained, while new items, such as “doing heavy outdoor work (e.g., cutting or chopping wood, clearing land)” or “being at weddings and other ceremonies,” were added. In lessons addressing stress management, the eight ways of coping advanced by Folkman and Lazarus were shared in the focus groups to better determine cultural coping preferences and coping styles (Folkman, Lazarus, et al., 1986). The coping strategies most highly endorsed by participants in these groups were emphasized throughout the intervention. This “hybrid-like” approach (Castro, Barrera, and Martinez, 2004) encouraged the inclusion of traditional and contemporary tribal world views in the intervention without compromising its core psychological components. After several formative evaluations with diverse tribal groups, AILS has been refined to address the needs of both traditional and pan-tribal adolescents (LaFromboise, 1995; LaFromboise, Coleman, and Hernandez, 1991). It has been continued

OCR for page 297
04 PREVENTING MENTAL, EMOTIONAL, AND BEHAVIORAL DISORDERS BOX 11-2 Continued implemented by interventionists (including teachers) for work with urban and res- ervation youth during in-school, after-school, and community-based programs for American Indian youth. AILS is thought to be broad enough to address concerns across diverse American Indian tribal groups yet respectful of distinctive and heterogeneous cultural beliefs and practices. The program received support in 2007 from three suicide prevention projects, funded by the Substance Abuse and Mental Health Services Administration, to train American Indian interventionists on a wide-scale basis, to complete an early adolescent version of the intervention, and to create an implementation guide. Efforts to evaluate AILS in an urban Indian education program are currently under way. Box 11-2). Bernal, Bonilla, and Bellido (1995) provide a framework for developing culturally sensitive interventions that calls for consideration of language, persons, metaphors, content, concepts, goals, methods, and context. On the other hand, research has indicated that, although cultural or other adaptations made by practitioners that reduce dosage or eliminate critical core content can increase retention by up to 40 percent, they also reduce positive outcomes (Kumpfer, Alvarado, et al., 2002). For example, efforts to create and disseminate best-practice components of the Nurse- Family Partnership Program failed to produce the same results as the controlled trial replications (Alper, 2002; Olds, 2002). While research on dissemination of tested and effective prevention programs appears war- ranted, more research to identify the active ingredients of those programs is required before adaptation and dissemination of best practices distilled from these programs are warranted. In general, there has been a dearth of research on cultural, racial, and ethnic issues involved in interventions aimed at preventing mental, emo- tional, and behavioral (MEB) disorders (U.S. Public Health Service, 2001a) and even less research on the effectiveness of specific prevention strategies when implemented in a population other than that originally targeted by a trial. However, several models are being used to examine the extent to which program adaptation can be used to address the unique cultural needs of communities. Castro, Barrera, and Martinez (2004), for example, describe a hybrid approach to modifying the content and delivery of an existing prevention program. This area needs more research, as few empiri- cal studies have examined alternative strategies. One method of enhancing cultural sensitivity and cultural relevance is

OCR for page 297
05 IMPLEMENTATION AND DISSEMINATION to involve the community in every aspect of a prevention trial (LaFromboise and Lewis, 2008; LaFromboise, 1996; Madison, McKay, et al., 2000; McCormick, McKay, et al., 2000; Baptiste, Blachman, et al., 2007; Bell, Bhana, et al., 2007; McKay, Hibbert, et al., 2007; Pinto, McKay, et al., 2007). However, developing and maintaining community involvement throughout all stages of program implementation present considerable challenges, as discussed below. Community-Driven Implementation Community-driven implementation builds heavily on the decision mak- ing of community leaders, often in partnership with researchers, with a focus on improving the community relevance and sustainability of a program. Implementation is guided by a community-driven agenda and staged imple- mentation of a prevention program, in some cases including development, implementation, and testing of a locally developed intervention. Evidence- based programs or principles are often introduced by research partners relatively late in the process. Built on the community-based participatory research approach, an agenda for community action is developed through a cooperative process with community members and multiple community constituencies. The involvement of researchers in identifying priorities may be quite limited or very involved (Minkler, 2004), but it always focuses on community leadership and establishment of an organizational structure for building and sustaining one or more interventions (Baptiste, Blachman, et al., 2007). In many minority communities, there is a history of mistrust of outsiders, government agencies, or researchers in particular (Thomas and Quinn, 1991), which influences the degree to which researchers are involved in decision making (McKay, Hibbert, et al., 2007). The traditions of research, including reliance on planned research designs, multiple assessments, and legal consent documents, are often viewed negatively by communities. Thus, researchers may begin as outside advisers who listen to the goals and needs of communities, with a part- nership in the decision-making processes evolving over time. The wealth of practical experience and wisdom in community-based organizations may offer opportunities for communities to establish an empirical basis for interventions with strong community support through community– research partnerships. Collaborations with key community constituents can (1) enhance the relevance of research questions, (2) help develop research procedures that are acceptable to potential community participants from diverse cultures, (3) address challenges to conducting community-based research, (4) maximize the usefulness of research findings, and (5) fos- ter the development of community-based resources to sustain prevention funding beyond grant funding (Israel, Schulz, et al., 1998; Institute of

OCR for page 297
06 PREVENTING MENTAL, EMOTIONAL, AND BEHAVIORAL DISORDERS Medicine, 1998; Schensul, 1999; Jensen, Hoagwood, and Trickett, 1999; Wandersman, 2003). Efforts to move from efficacy and effectiveness to full-scale implementation can and often do begin early by establishing such partnerships (Fixsen, Naoom, et al., 2005). A number of prevention specialists have called for the scientific study of community–research partnerships (Chinman, Hannah, et al., 2005; Spoth and Greenberg, 2005; Trickett and Espino, 2004; Wandersman, 2003). The principles that guide such partnerships are clear and involve researchers developing win-win relationships with communities in their efforts to foster trust and mutual respect (see Madison, McKay, et al., 2000; Israel, Schulz, et al., 2003; Trickett and Espino, 2004; Bell, Bhana, et al., 2007; McKay, Hibbert, et al., 2007; Pinto, McKay, et al., 2007). Researchers and com- munity collaborators should attempt to develop shared vision and mission, BOX 11-3 CHAMP: Collaborative HIV Adolescent Mental Health Program The Community Collaborative Board for the CHAMP project builds on the framework for an academic–community collaborative approach to HIV/AIDS risk reduction with urban adolescents (McKay, Hibbert, et al., 2007). The mission was “if the community likes the program, the research staff will help the community find ways to continue the program on its own” (Madison, McKay, et al., 2000). The CHAMP Community Collaborative Board structure is characterized by moderate- to high-intensity collaboration (Hatch, Moss, et al., 1993). All of the CHAMP Family Programs use community representatives as liaisons between youth and families in need and prevention programs, as suggested by research (Koroloff, Elliott, et al., 1994; McKay and Paikoff, 2007). Community parent facili- tators, who had participated in the program themselves, are trained to reach out to their neighbors and invite them to learn more about the program. In addition to providing factual program information, they are also able to share personal, firsthand program experience. Community members also play a role in delivering the intervention, helping to address issues of cultural sensitivity and addressing research concerns of efficacy and effectiveness while preparing the community for dissemination. All of the intervention sessions are cofacilitated by a mental health intern/parent facilitator team. The team receives weekly joint training in program content; skills related to facilitation of child, parent, and multiple family groups; and issues related to prevention research and protection of human subjects, including confidentiality and mandated safety issues. Grant funding to enhance leadership development among community board members was secured to help pave the way for the community to take over the

OCR for page 297
07 IMPLEMENTATION AND DISSEMINATION consensus on strategies, and synergy in execution and implementation (Senge, 1994). Resources should be openly discussed with community mem- bers, who should benefit from the resources as much as do the research- ers. Thus resources, both tangible (e.g., researchers employing community members and partners providing facilities for programs) and intangible (e.g., partners’ knowledge of participants and researchers’ knowledge of research methods), should be shared (Suarez-Balcazar, Davis, et al., 2003). Collaboration must also involve team training in which researchers learn community issues and community partners learn research issues. Early involvement of communities, power sharing, mutual respect, community benefit, and cultural sensitivity (Sullivan, Kone, et al., 2001) are needed to meet these challenges. Box 11-3 describes a program aimed at HIV/AIDS risk reduction that is built on such a collaborative model. intervention from the research team (Madison, McKay, and the CHAMP Col- laborative Board, 1998). Community support was hypothesized to facilitate wider dissemination of prevention messages and strategies (Galbraith, Ricardo, et al., 1996; Schenshul, 1999; Stevenson and White, 1994). The team believed that given the business skills necessary to run such programs, large community-based agen- cies might be more able than academic research teams to retain proven programs within their infrastructure, enhancing the likelihood that a specific program would be sustained over time (Galbraith, Ricardo, et al., 1996; Goark and McCall, 1996). Community leaders were also responsible for the day-to-day research operation (with consultation from university researchers) (McKay, Chasse, et al., 2004). CHAMP-Chicago and New York were also funded to study how to transfer an academic research project (based at the University of Illinois in Chicago and Mt. Sinai School of Medicine in New York), with both efficacy and effective- ness components. For example, in Chicago the program was transferred to a community-based organization (Habilitative Systems Inc., a social service agency in urban Chicago). Key elements of the framework for this 13-year transfer are (1) ensuring a good academic-agency fit, (2) early planning for sustainability, (3) building in continuous quality improvement, and (4) balancing program adaptation with fidelity (Baptiste, Blachman, et al., 2007). The experience implementing the CHAMP Program in Chicago and New York helped inform the 2001 CHAMP-South Africa research project. Based on its success, in 2007, the South African HIV prevention intervention obtained private foundation funding to serve 500 families, many of whom were in the control condi- tion during the study.

OCR for page 297
26 PREVENTING MENTAL, EMOTIONAL, AND BEHAVIORAL DISORDERS Dissemination and Moving to Implementation Studies Scale Sustainability Adoption Effectiveness Studies Efficacy Studies Preintervention Fig11-1.eps FIGURE 11-1 Stages of research in prevention research cycle. implementation to achieve greater public health impact. They suggest three interrelated sets of research requirements and findings to accomplish popu- lation-based prevention: “(1) rigorously demonstrating intervention effec- tiveness; (2) attaining sufficient levels of intervention utilization in diverse general populations, requiring study of recruitment/retention strategies, cul- tural sensitivity, and economic viability; and (3) achieving implementation quality, involving investigation of adherence and dosage effects, along with theory-driven, intervention quality improvement” (p. x). To accomplish this, it may be useful to view implementation as having three phases: adoption, sustainability, and moving to scale (see Figure 11-1). Prevention scientists, government organizations, state and community organizations, and com- munity leaders have major roles throughout this process. Ideally, the results of these phases will feed back to earlier areas of research. Specific research questions related to each of these phases warrant additional attention: • Research questions related to the adoption of a prevention program into a service system, which routinely involves the formation of partnerships and the development of an infrastructure to support the technical, financial, administrative, monitoring, evaluative, and logistical needs related to the program.

OCR for page 297
27 IMPLEMENTATION AND DISSEMINATION • Research questions pertaining to sustaining the program once it is introduced in a service system. The ability to sustain a program relates to the organizational structures, practices, data monitoring, leadership, and related characteristics in place in the home institu- tion for the program. • Research questions involving moving to scale, or understand- ing which steps facilitate the structures and funding necessary to expand the program to other sites. There are major challenges of introducing and taking effective pro- grams to scale, particularly in poor and underserved communities (Madon, Hofman, et al., 2007; Sanders and Haines, 2006), and clearly the current body of generalizable knowledge is inadequate to provide robust strategies for effective implementation across different populations, systems, and programs. Nevertheless, there is reason for optimism. First, the dearth of generalizable knowledge is a product of the lack of significant investment in scientific studies of the implementation and dis- semination process. Second, while the specific factors regarding successful program implementation may vary from case to case, there are many com- monalities in why organizations have difficulties adopting and sustaining prevention programs. For example, poor communities, minority popula- tions, and developing countries often lack professionally trained staff to deliver a program as originally designed, so successful implementation may need to identify unique program delivery agents using existing resources (Sanders and Haines, 2006). Also, there is general agreement in the field about shared dimensions of organizational change that are relevant across widely different interventions; these include system readiness for change, culture, and the role of leaders (Chambers, 2008). Third, the increasing use of more rigorous designs, such as randomized trials that test differ- ent implementation strategies (see Chapter 10), social network analysis (Chambers, 2008), and the combined use of qualitative and quantitative data, is likely to lead to more precise implementation inferences around shared research questions. This information can be used as markers to guide the development of successful implementation efforts across diverse fields and settings. Implementation research turns the traditional efficacy/effectiveness research questions of prevention science into experimental questions about the process of implementation itself. To date the leading model for exam- ining implementation of prevention programs has been to focus on a sin- gle region, such as an urban school district or an entire state. With this approach, one can examine how process factors affect the adoption of a program over time, even if there are unique or novel factors operating in that particular system (Biglan, 2004). In addition, some randomized trial

OCR for page 297
28 PREVENTING MENTAL, EMOTIONAL, AND BEHAVIORAL DISORDERS designs are beginning to be used to study the process from effectiveness to large-scale implementation. For example, the community epidemiology model of Kellam and colleagues can examine questions related to effec- tiveness, sustainability, and moving to scale, along with randomization (Kellam, Koretz, and Moscicki, 1999; Kellam, 2000). This approach exam- ines variation in the community through the use of random assignment of the intervention conditions to different contexts and across time. To test effectiveness of a classroom-based intervention, for example, classes in a school can be randomized to intervention, and the impact for interven- tion and control classrooms can be compared in the first cohort (Brown, Wang, et al., 2008). This can be followed by examination of intervention sustainability by measuring the level of program fidelity that intervention teachers deliver in the second year with new students. Teachers remain in the same intervention condition, but the support structure in the schools for monitoring, supervision, and resource allocation changes from the first year to reflect the way such a program is likely to be delivered over time. Finally, scalability can be examined in a third cohort in which all the teach- ers implement the full intervention. Again, with resources allocated as one would anticipate in a scaled-up program, this third cohort can be used to compare the level of program fidelity as well as child outcomes with those of previous cohorts subjected to different levels of infrastructure support. This model, however, allows limited testing of the components of a higher level implementation strategy involving the full school district, since there is only one such district studied at a time (Kellam, 2000). Implementation Trials A valuable approach that would increase knowledge of successful imple- mentation strategies is to test alternative strategies using a randomized trial design (implementation trials). This would necessarily require multiple loca- tion and multilevel analyses to fully examine impact. One such implementa- tion trial is comparing the CTC model (see Box 11-1) to an implementation plan with passive assistance (Hawkins, 2006). In Project Adapt, Smith, Swisher, and colleagues (2004) tested two types of implementation of Life Skills Training (LST) using group-based random assignment. They compared a standard implementation model in which the LST curriculum stood apart from the day-to-day teaching activities with an infusion model that integrated the curriculum into traditional courses. Three rural school districts were randomly assigned to the traditional LST condi- tion, the infused LST condition, and a control condition. There was some suggestion of beneficial impact of both intervention conditions against con- trol for girls in the first year of the study, although these findings generally disappeared by the second year and did not show at all for boys. There were

OCR for page 297
2 IMPLEMENTATION AND DISSEMINATION few differences between the two intervention conditions, although this could have been due to the low statistical power for this school-based design. An ongoing randomized trial of two different methods of implementing an evidence-based program for foster care in California counties may shed new light on implementation approaches. The trial was driven by a Cali- fornia mandate to use evidence-based practices and interest in identifying ways to facilitate statewide implementation. Although this particular trial involves multidimensional treatment foster care (Chamberlain, Saldana, et al., in press), an evidence-based program that targets high-need children who are in state custody, it can also be enlightening for the implementa- tion of evidence-based prevention. Training had earlier been offered to all California counties, but only about 10 percent of the counties became early adopters, not unlike that of most novel interventions (Rogers, 1995; Valente, 1996). All the remaining 40 eligible counties were randomly assigned to one of two methods for implementation: a standard model and a community development team model, which used cross-county peer-to-peer support to address the administrative, financial, and logistical challenges in implement- ing the program. The evaluation is assessing whether the rate and length of time for adoption and sustainability is reduced by the team model, taking into account the dependence between team members. Research on Increasing Rates of Intervention Adoption and Participation The rate of adoption of a particular program across different commu- nities and the rate of participation in a community are major issues that affect levels of program implementation. A variety of potential approaches to increase these rates could be evaluated in future research. Encouragement Designs The general class of randomized “encouragement designs” are ones that randomize individuals to different modalities of recruitment, incen- tives, or persuasion messages to influence their choice to participate in one or another intervention condition. Such incentives as cash or child care dis- counts have been used to encourage participation. An important advantage of these designs is that they allow one to take into account self-selection factors in examining impact (Yau and Little, 2001; Frangakis and Rubin, 1999; Barnard, Frangakis, et al., 2002). They also address whether targeted efforts to increase participation reach those most at risk (Brown and Liao, 1999). Randomized encouragement trials have been used to evaluate early versus late enrollment in Early Head Start (Administration for Children and Families, 2005), whether antiviral medications for HIV should adhere to a rigid regimen or be more flexible (HIV SMART AntiRetroviral Trial),

OCR for page 297
0 PREVENTING MENTAL, EMOTIONAL, AND BEHAVIORAL DISORDERS BOX 11-4 The Internet as a Potential Tool for Wide-Scale Dissemination of Preventive Interventions The enormity of need for mental health services often produces a type of paralysis: since it is not feasible to train enough providers to treat all individuals with mental, emotional, and behavioral disorders, how can preventive interven- tions be provided to those at risk? This dilemma is caused in part by the exclusive reliance on consumable interventions, such as face-to-face services, and the use of medications. Once a prevention or therapy session is over, no other individual can benefit from that hour of contact. Once a medication is consumed, no one else can benefit from its therapeutic effect. The development and implementa- tion of interventions delivered via the Internet offers the promise of an approach to make interventions available on a continual basis to a wide range of young people at minimal cost while addressing several dissemination and implementa- tion challenges. Fidelity: The fidelity of Internet interventions is inherent as the material on the computer screen remains the same, no matter how many times it is used. The content of the intervention can be shared widely exactly as tested in randomized control trials. Scalability: An Internet intervention can be shared with literally thousands of users beyond the locality in which it was created, while remaining accessible to the original locality. The site of a proven Internet intervention can be immediately opened to use by anyone with web access, which also allows effectiveness evalu- ation on a wide scale. Sustainability: The cost of maintaining a website hosting an evidence-based preventive intervention is relatively modest, especially if the site is an automated, self-help intervention. and whether strategic, structural engagement of adolescents increases completion of family therapy more than traditional engagement methods (Szapocznik, Perez-Vidal, and Brickman, 1988). Such designs may have value in exploring ways to increase the reach of prevention programs. In the years since publication of Reducing Risks for Mental Disorders: Frontiers for Preventive Intervention Research (Institute of Medicine, 1994), a modest number of experimental tests have aimed at increasing individual- or family-level participation rates for a preventive intervention. For example, motivational interviewing techniques have been used in trials in an attempt to engage parents around problems or issues that they can relate to their own children (Dishion, Kavanagh, et al., 2002).

OCR for page 297
 IMPLEMENTATION AND DISSEMINATION Accessibility: Internet interventions can simultaneously serve users across a community, a state, the nation, or the world, at any time of the day or night, includ- ing holidays and weekends. Stigma: The availability of Internet interventions that are used in the privacy of one’s own home, educational or work setting, or using a public access computer makes these interventions more likely to be used by people who would not come to a mental health–oriented program. Reaching multicultural, multilingual communities: Internet interventions can be implemented relatively easily in multiple languages. Similarly, advances in technology now make it possible to create Internet interventions that require a minimum level of reading or writing. The use of video, graphics, and audio allow the creation of Internet interventions that can be used by individuals at any edu- cation level. Internet interventions also have limitations. One of the most troublesome is the lack of access to the web by many low-income, low-education groups. However, Internet access is increasingly available via mobile devices, such as cell phones. Many developing countries have skipped the stage of land-line phones and moved directly to cell phones. As is the case for other venues, Internet interventions will not be effective in preventing all types of MEB disorders. It is useful to think in terms of “market segmentation,” in which specific means of reaching populations at risk will need to be developed and evaluated to see which is most effective for which population. Nevertheless, to help make prevention feasible, one must think beyond traditional interventions and harness the power of advanced communica- tion media, such as the Internet. Use of Current Technologies The advent of the Internet and modern use of technology presents new opportunities for both dissemination and research. Broadcasting the avail- ability of accessible web-based or CD-ROM programs or making imple- mentation resources (e.g., training, manuals) available could potentially increase the use of prevention programs. Implementation of interventions via the web has the potential to address several implementation barriers (see Box 11-4). Because online interventions can occur anonymously, these technologies also have the potential to be less stigmatizing, a significant potential barrier to participation.

OCR for page 297
2 PREVENTING MENTAL, EMOTIONAL, AND BEHAVIORAL DISORDERS Identification of Early Adopters Rogers (1995) identified general factors that affect or influence the diffusion of innovations. This early work on program diffusion was based on a synthesis of careful observation from case studies. One major finding is that early adopters share traits that can be readily measured or inferred from behaviors or attitudes. While this earlier work was observational in nature and did not attempt to influence adoption itself, assessments can be used to identify communities, organizations, institutions, families, as well as individuals who are most likely to be early adopters of such programs. Thus identification of those likely to be early adopters and targeting preven- tion efforts to these groups represent a potential strategy to affect program adoption. Use of Opinion Leaders More recently, the same principles underlying research on diffusion of interventions and social influence have been used proactively to increase the adoption of prevention programs and test adoption strategies in group-based randomized trials. One approach used early in HIV prevention is to target opinion leaders in a community who would themselves deliver peer-to-peer messages to promote increased program adoption. Kelly, St. Lawrence, and colleagues (1991), for example, successfully identified and then trained gay opinion leaders in rural communities to encourage safe sexual practices. These leaders were able to modify HIV risk behaviors in their communi- ties. Also, media campaigns for HIV prevention in developing countries are using soap operas in which leading actors talk openly about the use of condoms and getting tested for HIV (Valente, 1996). A similar approach is now being used in approaches to youth suicide prevention; teenage leaders are trained to deliver messages to both peers and adults in their community aimed at increasing help seeking among sui- cidal youth. Suicidal youth are often much less likely to talk to adults than are nonsuicidal youth (Wyman, Brown, et al., 2008), yet the vast majority of youth tell a friend before committing suicide. A general strategy for reducing suicide is to increase willingness to talk to a trusted adult by both suicidal youth and their friends. One such program (Sources of Strength) is designed to change peer norms about secrecy and disclosure surrounding distressed youth. A first implementation step is to identify peer leaders from diverse social networks. The program then modifies norms by having each of the peer leaders identify trusted adults in their own lives to whom they would turn at times of stress.

OCR for page 297
 IMPLEMENTATION AND DISSEMINATION Market Research Many evidence-based prevention programs are delivered to small por- tions of the population. A small number of state agencies, schools, com- munities, or families select programs with the highest levels of evidence, opting instead for programs that have less evidence, or no program at all. One promising approach to improve program reach to individual families is to integrate business models into prevention to address consumer needs from the beginning (Rotheram-Borus and Duan, 2003). By following a prevention service development model that integrates consumer preferences from the beginning (Sandler, Ostrom, et al., 2005), the research team can aim for effectiveness and large-scale implementation from the start of the product development cycle. Similarly, there is a need for greater consideration of the most effective metrics to report outcomes to the public. Although effect size may be the most appropriate metric for studies of indicated interventions in which all participants begin with a substantial rate of symptoms, it may be a poor metric for universal interventions. In universal interventions, it is usually the case that a large percentage of the population begins with low levels of symptoms, and thus it is unlikely (at least in the short term) that much of this population will benefit from the intervention. In most cases it is only in the higher symptom group of the population that larger effect sizes will be obtained (Wilson and Lipsey, 2007). Thus, for universal interventions, alternative methods are needed to convey the practical and social policy sig- nificance (Davis, MacKinnon, et al., 2003; McCartney and Rosenthal, 2000). Cost-effectiveness is one such metric, as universal interventions may achieve more benefit in relation to their cost given their large reach. Naturalistic Large-Scale Public Health Research Although their internal validity makes them valuable science, random- ized control trials do not always have good external validity. Furthermore, much academic research is rarely applied to the day-to-day world. Science can often benefit from the experience of everyday clinical observations. For example, in 1982 when clinical observations in a community mental health setting found an extraordinary number of children exposed to violence, a plethora of scientific research projects confirmed this observation, culminat- ing in several large-scale strategies to prevent these children from develop- ing mental health sequelae (Jenkins and Bell, 1997; Bell, 2004). In addition, communities often implement programs because they are based on extensive clinical wisdom and have widespread community sup- port. Research designed to empirically test programs being implemented in naturalistic environments could identify approaches that are readily imple-

OCR for page 297
4 PREVENTING MENTAL, EMOTIONAL, AND BEHAVIORAL DISORDERS mentable by other communities. Gibbons, Hur, and colleagues (2007) have developed statistical methodology that provides some evidence in support of such interventions. Conversely, randomized control prevention trials can also inform public health practice. For example, a violence prevention trial, Aban Aya (Flay, Graumlich, et al., 2004) informed a Chicago public school violence prevention initiative with teenage mothers (Bell, Gamm, et al., 2001), which demonstrated significant reductions in pregnant teenage dropout rates. In addition, most teens had only one child despite becoming a mother at very young ages (Lamberg, 2003). CONCLUSIONS AND RECOMMENDATIONS There have been clear advances in implementing effective programs since the publication of Reducing Risks for Mental Disorders: Frontiers for Preventive Intervention Research (Institute of Medicine, 1994). Indeed, the knowledge base on effective prevention programs at that time was very thin. However, the levels of effective implementation are much lower than the availability of tested interventions suggests. Conclusion: Implementation of effective preventive interventions is hampered by lack of ongoing resources and competing priorities of the service systems or communities that could implement them. One of several contributors to the relative lack of implementation is lack of empirical evidence regarding how to effectively approach imple- mentation. A critical next phase of research needs to examine methods for enhancing the implementation of effective programs. The prevention research cycle proposed in the 1994 IOM report assumes a “hierarchical scientist-as-expert perspective and portrays scientists as separate agents conducting research on ‘subjects’ and ‘groups’” (Dumka, Mauricio, and Gonzales, 2007). Although the stages of research in the model require the cooperation of individuals and organizations, the model did not specifically address the relationships and collaborative processes that are critical to accomplishing each stage (Dumka, Mauricio, and Gonzales, 2007). For implementation to be successful, there needs to be strategic input from science, policy, and practice perspectives that builds on the scien- tific knowledge base. Evidence is needed on how to make implementa- tion occur in communities, the policy directives that promote or enforce the use of evidence-based programs and data systems, and the effective adoption and sustainability of programs in practice (Greenberg, 2004). Important progress has been made, and there are now new opportunities to make partnerships between scientists, policy makers, and practitioner

OCR for page 297
5 IMPLEMENTATION AND DISSEMINATION communities to transport effective prevention programs into community settings. Additional research is needed to identify core components shared across programs. Major implementation challenges suggest new avenues of research. Conclusion: Knowledge about effective strategies for implementing or adopting evidence-based prevention interventions is limited. New approaches to implementation represent the frontier of prevention research. Recommendation 11-1: Research funders should support experimen- tal research and evaluation on (1) dissemination strategies designed to identify effective approaches to implementation of evidence-based programs, (2) the effectiveness of programs when implemented by communities, and (3) identification of core elements of evidence-based programs, dissemination, and institutionalization strategies that might facilitate implementation. Knowledge gained from evaluation of implementation approaches will be more generalizable if it is conducted in multiple settings. A number of evidence-based interventions are viable candidates for implementation. Evaluations that involve partnerships between states or communities ready to implement interventions and researchers could yield valuable results. Recommendation 11-2: Research funders should fund research on state- or community-wide implementation of interventions to promote mental, emotional, or behavioral health or prevent MEB disorders that meet established scientific standards of effectiveness. Although there are many evidence-based models, it is not clear how gen- eralizable they are to groups other than the ones with which they were tested. Interest in an intervention is likely to be greater if it is culturally relevant and embraced by the community. Lack of relevance may contribute to interven- tions being implemented with limited fidelity and resultant limited outcomes. Addressing this may include replication with new populations as well as examining versions that strengthen the cultural competency of interventions. Conclusion: Despite multiple dissemination venues, evidence-based interventions have not been implemented on a wide-scale basis. Where interventions have been implemented, they are often not implemented with fidelity, with cultural sensitivity, or in settings that have the capacity to sustain the effort.

OCR for page 297
6 PREVENTING MENTAL, EMOTIONAL, AND BEHAVIORAL DISORDERS Conclusion: Little research has addressed the question of how trans- portable evidence-based interventions developed for one ethnic group are to a range of ethnic and cultural groups. Recommendation 11-3: Research funders should prioritize the evalua- tion and implementation of programs to promote mental, emotional, or behavioral health or prevent MEB disorders in ethnic minority com- munities. Priorities should include the testing of culturally appropriate adaptations of evidence-based interventions developed in one culture to determine if they work in other cultures and encouragement of their adoption when they do. Finally, multiple opportunities for naturalistic research could enrich the prevention portfolio and convincing evidence that collaborations between researchers and communities can increase the relevance and sustainability of interventions, including through efforts to adapt existing evidence-based interventions. Recommendation 11-4: Researchers and community organizations should form partnerships to develop evaluations of (1) adaptation of existing interventions in response to community-specific cultural characteristics; (2) preventive interventions designed based on research principles in response to community concerns; and (3) preventive interventions that have been developed in that community, have dem- onstrated feasibility of implementation and acceptability in the com- munity, but lack experimental evidence of effectiveness. On a practical level, for tested preventive interventions to become widespread, the available research suggests that successful interventions should include at least the availability of published material, such as hand- books, curriculum, and manuals describing the intervention and prescribing actions to be taken; certification of trainers or an electronic training sys- tem; high-quality, data-driven technical assistance; implementation fidelity measures; dissemination efforts that are organized around marketing and delivery; an information management system; and community demand for systems that work. In addition to development and implementation of effective programs, the nation needs to support implementation of policies and broad preven- tion principles in order to create a comprehensive, sustained approach to prevention. Policies that support low-income families and promote healthy development are needed as the basic foundation for such an approach.