CHAPTER 6
Lessons from Experimental Evaluations

Findings from nonexperimental and small scale quasi-experimental studies of community programs for youth provided the foundation for the descriptions in Chapter 5 of programs considered to be effective by participants, evaluators, observers, and qualitative researchers. In this chapter we summarize the committee’s review of findings from experimental and large-scale quasi-experimental evaluations of community programs for youth. The committee examined several meta-analyses and review articles that summarize the findings from many different studies of community programs for youth and considered in more detail three specific program evaluations.

This examination had four primary objectives. First, we were interested in whether there is evidence from the most rigorous evaluations available that community programs for youth make a difference in the lives of their participants. Second, we wanted to know whether the programs that had received these types of rigorous evaluations included either the program features outlined in Chapter 4 as potentially important components of community programs for youth or the personal and social assets outlined in Chapter 3 as potentially important targets for these pro-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 147
Community Programs to Promote Youth Development CHAPTER 6 Lessons from Experimental Evaluations Findings from nonexperimental and small scale quasi-experimental studies of community programs for youth provided the foundation for the descriptions in Chapter 5 of programs considered to be effective by participants, evaluators, observers, and qualitative researchers. In this chapter we summarize the committee’s review of findings from experimental and large-scale quasi-experimental evaluations of community programs for youth. The committee examined several meta-analyses and review articles that summarize the findings from many different studies of community programs for youth and considered in more detail three specific program evaluations. This examination had four primary objectives. First, we were interested in whether there is evidence from the most rigorous evaluations available that community programs for youth make a difference in the lives of their participants. Second, we wanted to know whether the programs that had received these types of rigorous evaluations included either the program features outlined in Chapter 4 as potentially important components of community programs for youth or the personal and social assets outlined in Chapter 3 as potentially important targets for these pro-

OCR for page 147
Community Programs to Promote Youth Development grains. Third, when this information was available, we wanted to know whether the program features hypothesized to be important in Chapter 4 were indeed important in both facilitating positive youth development and preventing the emergence of serious problems. Finally, as a lead-in to the next chapter, we also wanted to assess the quality of these evaluations. We include in this chapter comments on whether the evaluations selected for inclusion in this chapter meet the most rigorous methods of evaluation—described in Chapter 7—that can be used with community programs for youth. REVIEWS AND META-ANALYSES OF EVALUATIONS The committee reviewed seven reputable reviews and meta-analyses of prevention and promotion programs for youth from the fields of mental health, violence prevention, teenage pregnancy prevention, and youth development.1 We considered reviews that included both programs for youth with a primary focus on prevention and programs explicitly focused on a youth development framework since, as explained previously, the committee found the distinction between prevention and positive youth development not very clear in practice. Programs with prevention goals, for example, often included components common to programs with youth development goals. Details of the programs included in the meta-analyses reviewed are summarized in Table 6–1. As noted, the primary objective of reviewing these meta-analyses was to provide our best assessment of the extent to which community programs have beneficial effects for youth. Our secondary objective was to determine the extent to which these programs incorporate the features of positive developmental settings developed in Part I and whether there is evidence of links between these program features and positive youth outcomes. Since none of the programs were designed with these specific assets and features in mind, we often had to make assumptions about likely features from the descriptions of the programs included in the various reviews. Through this review, we also identified the strengths and limitations of these evaluations. 1   We included reviews and meta-analyses published between 1997 and 2000 that appeared in either published, professionally reviewed journals, or selected government documents. In the latter category, only government documents using rigorous methods of review were included in order to reduce the potential biases associated with reviews conducted by groups with high stakes in positive or negative conclusions.

OCR for page 147
Community Programs to Promote Youth Development We begin with meta-analyses focused nominally on mental health promotion and mental illness/problem behavior prevention programs because these have the most rigorous evaluations. There are several reasons for this: first, there is a long tradition of both mental health promotion and mental illness/problem behavior prevention work; consequently, these programs have had time to develop and to have comprehensive rigorous evaluations completed. Second, many of these programs are being conducted in school buildings during the official school day. These two characteristics make it much easier to conduct a successful randomized trial design evaluation because the participants are drawn from a “captive” population. Conducting such evaluations in community programs is much more difficult precisely because participation is voluntary. We say more about this later in this chapter and in Chapter 7. We then move onto the meta-analyses and summative reviews of largely out-of-school time community programs. Mental Health Programs Durlak and Wells Durlak and Wells (1997, 1998) conducted two meta-analytical reviews of primary and secondary prevention mental health programs operating prior to 1992 for youth under the age of 19. (Primary prevention programs intervene with normal populations to prevent problems from developing. Secondary prevention programs target individuals already at risk or exhibiting problems). Both meta-analyses included only programs with a control group of some type (e.g., a group that is not participating in the program or a group that is on a waiting list to participate in the program). Randomized designs were used in 61 percent of the primary prevention program evaluations and 71 percent of the secondary prevention program evaluations. The primary prevention review (Durlak and Wells, 1997) included 177 programs; the secondary prevention review (Durlak and Wells, 1998) included 130 evaluations. Finally, most of these programs took place in schools (a likely reason that these studies had such low attrition rates of young people leaving the program before completion and final testing). In general, the results suggest that preventive mental health programs can be effective across a variety of psychological outcome measures for periods of up to two years following the intervention exposure. They are particularly effective at increasing such personal competencies as

OCR for page 147
Community Programs to Promote Youth Development TABLE 6–1 Community Programs to Promote Adolescent Development Program Name/Age, Grade Evaluated Program Description Outcomes Personal and Social Assets Features of Positive Developmental Settings Across Agesa,b Grade: 6th Location: Family, school, community Sessions: Mentoring (2 hrs/wk), community service (1 hr every other week), social problems solving (26 1 hr sessions) over school year Content: mentoring by older adults, classroom-based life skills training, community service activities, workshops for parents ↑ Positive attitudes toward school, the future, older people, knowledge of elders, and community service ↓ School absence Intellectual psychological emotional social Emotional and instrumental support; belonging; positive social norms; opportunity for efficacy; opportunities for skill building; integration with family, schools, and community Adolescent Transitions Projecta,d Age: 10–14 Grade: 6th–8th Location: Family, school, community Sessions: 12 over 18 hrs Content: Youth self-regulation skills training (teen focus group), parent management skills training (parent focus group), consultant to improve parent-youth communication (teen/parent focus group) ↑ Social learning ↓ Negative engagement with family, conflict, negative family events, youth aggression (increased school behavior problems for teen focus group at one year follow-up) Psychological emotional social Emotional and instrumental support; belonging; positive social norms; opportunities for skill building; integration with family, schools, and community

OCR for page 147
Community Programs to Promote Youth Development Bicultural Competence Skillsa Age: 11–12 Location: Community Sessions: 10 Content: Skills training to promote competence and positive identity based on bicultural fluency ↑ Self-control, assertiveness, healthy coping, substance abuse knowledge ↓ Alcohol, tobacco, and other drug use Psychological emotional physical Belonging; positive social norms; opportunities for skill building Big Brothers Big Sistersa,b,c,d Age: 10–16 Location: Community Sessions: 9–12 hrs/mon for one year Content: Activities with mentor ↑ Grade point average, parental trust ↓ Hitting behavior, likelihood of initiating alcohol and drug use, skipping school, lying to parents Intellectual psychological emotional social physical Emotional and instrumental support; belonging; positive social norms; physical and psychological safety; opportunity for efficacy; opportunities for skill building Brainpower Program f Age: 10–12 (indicated) Location: School Sessions: 12 lessons (60–90 minutes each) Content: Social competency: focus on improving the accuracy of children’s perceptions and interpretations of others’ actions ↓ Aggressive behavior following the intervention Social Belonging; opportunities for skill building

OCR for page 147
Community Programs to Promote Youth Development Program Name/Age, Grade Evaluated Program Description Outcomes Personal and Social Assets Features of Positive Developmental Settings Bullying Prevention Programc,d Grade: 4th–7th Norway; 5th–8th U.S. equivalent (universal) Location: School Sessions: 9–12 hrs/mon for one year Content: 32-page booklet included information on the scope, cause, and effects of school bullying and detailed suggestions for reducing and preventing it. Abbreviated bullying info to families with school-age children. A 25-minute video with vignettes of bullying situations. Students completed a brief questionnaire related to bullying to increase awareness and promote discussion of the problem ↓ 50 percent or more in bully/victim problems for boys and girls across all grades (4–9), with more marked effects after 2 years than after 1 year (sleeper effects) Psychological emotional social Positive social norms; physical and psychological safety; opportunities for skill building

OCR for page 147
Community Programs to Promote Youth Development The Child Development Projecta Age: 11–12 Grade: 3rd–6th Location: Family and school Sessions: Integrated curriculum over school year Content: Cooperative learning, reading and language arts, developmental discipline, school community building, home activities ↑ Peer social acceptance ↓ Alcohol and tobacco use, loneliness and social anxiety; marijuana use, carrying weapons, vehicle theft (high implementation subgroup) Intellectual psychological emotional social physical Belonging; positive social norms; opportunities for skill building; integration with family, schools, and community Coping and Support Trainingd Grade: 9th–12th (indicated) Location: School Sessions: 12-sessions Content: Group life skills training ↑ Self-control, problem-solving abilities, and perceived family support ↓ Suicide risk behaviors and anger problems Intellectual psychological emotional social physical Emotional and instrumental support; belonging; positive social norms; physical and psychological safety Coping with Stress Coursed Grade: 9th–10th (indicated) Location: After school Sessions: 15, 45 min each Content: Cognitive intervention to encourage adaptive coping (cognitive restructuring, identify and challenge negative or irrational thoughts) No significant differences at the end of the intervention, but ↓ cases of “major depressive disorder” or dysthymia (milder depressive disorder) at 12 months postintervention . Intellectual psychological emotional Opportunities for skill building Counselor-CARE d Grade: 9th–12th (indicated) Location: School Sessions: 1, 3.5–4 hrs Content: Brief assessment and resource identification program using computer program and counselor ↑ Self-esteem ↓ Suicide risk behaviors and anger problems, reports of depression Psychological emotional Emotional and instrumental support

OCR for page 147
Community Programs to Promote Youth Development Program Name/Age, Grade Evaluated Program Description Outcomes Personal and Social Assets Features of Positive Developmental Settings Creating Lasting Connectionsa,b Age: 12–14 Location: Family, church, community Sessions: Youth (15 hrs); parents (55 hrs); volunteer service (18 hrs); follow-up and consultation support (1 yr) Content: Church community mobilization, parent and youth strategies to promote communication and self-management skills, follow-up case management service ↑ Youth use of community services, related action tendencies, perceived helpfulness ↓ Onset of substance abuse delayed as parents changed their substance use beliefs and knowledge Psychological emotional social physical Emotional and instrumental support; belonging; positive social norms; opportunities for skill building; integration with family, schools, and community ENABLe Age: 12–13 Grade: 7th–8th Location: School classroom, community organization Sessions: 5, 1 hr each Content: Social influence theory; help youth understand social and peer pressures to have sex, develop and apply resistance skills, emphasis on postponing sexual involvement. ↑ Pregnancies in teen-led groups, otherwise no effects ↓ Psychological emotional social physical Positive social norms; opportunities for skill building; integration with family, schools, and community

OCR for page 147
Community Programs to Promote Youth Development Friendly PEERsuasionb Age: 11–14 (indicated) Location: Girls, Inc. Sessions: 14, 1 hr each Content: Hands-on, interactive activities teach about the short- and long-term effects of substance abuse; healthy ways to manage stress; how to recognize media and peer pressure to use drugs; skills to make responsible decisions about drug use. As peer leaders, youth plan and implement substance abuse prevention activities for 6- to 10-yr-olds ↓ Drinking and delayed onset of drinking Intellectual psychological emotional social physical Emotional and instrumental support; belonging; positive social norms; opportunity for efficacy; opportunities for skill building Functional Family Therapyc Age: 11–18 (indicated) Location: Family Sessions: 8–12 hrs, 26 hrs max. Content: Behavioral systems family therapy; flexible delivery of service: 1–2 person teams to clients in-home, clinic, juvenile court, and at time of reentry from institutional placement ↓ Rates of offending, foster care or institutional placement reduced at least 25 percent Intellectual psychological emotional social Positive social norms; opportunities for skill building; integration with family, schools, and community

OCR for page 147
Community Programs to Promote Youth Development Program Name/Age, Grade Evaluated Program Description Outcomes Personal and Social Assets Features of Positive Developmental Settings Growing Healthya Grade: 4th–7th Location: School Sessions: 43–56 lessons over 1 or 2 yrs Content: Health competence promotion ↑ Positive knowledge and attitudes toward health, development and personal responsibility ↓ Smoking and intention to smoke Psychological emotional physical Positive social norms; physical and psychological safety Improving Social Awareness-Social Problem Solving (ISA-SPS) Grade: 5th Location: School Sessions: 2 yrs Content: Individual skill-building to promote social competence, decision making, group participation and social awareness; targets transition to middle school ↑ Coping with stressors related to middle school transition ↓ Teacher reports of problem behavior, psychopathology at six-year follow-up; boys: ↓ Alcohol use, violent behavior toward others, self-destructive/identity problems; girls: ↓ rates of cigarette smoking, chewing tobacco, and vandalism Psychological emotional social physical Belonging; positive social norms; opportunities for skill building Know Your Bodya Grade: 4th–9th Location: School Sessions: 2 hrs/wk for 6 yrs Content: Health promotion ↑ Healthy dietary patterns ↓ Smoking initiation Physical Positive social norms

OCR for page 147
Community Programs to Promote Youth Development Life Skills Traininga,c Grade: 7th–9th Location: School Sessions: 2 sessions/wk for 15 wks (Y1); 10 booster sessions (Y2); 5 booster sessions (Y3) Content: Competence promotion and resistance training ↑ Interpersonal skills, knowledge of smoking and substance abuse consequences ↓ Cigarette and marijuana smoking, alcohol intoxication and polydrug use Intellectual physical Positive social norms; opportunities for skill building Louisiana State Youth Opportunitiesb Age: 14–16 Location: School, community Sessions: Summer, 8 weeks Content: Live on campus, half-day in math and reading classes, half-day working at on-campus sites earning minimum wage, required to open saving account, other services provided: counseling, study skills training, health care, recreation, field trips and speakers ↑ Standardized math test scores, intention to stay in school, career maturity ↓ In reading skills, but less so than control group Intellectual psychological emotional social Positive social norms; physical and psychological safety; opportunity for efficacy; opportunities for skill building

OCR for page 147
Community Programs to Promote Youth Development Strengths and Limitations Like the Teen Outreach Program evaluation, this evaluation was excellent in many ways. However, it also had limitations. First, a substantive theory of change was not explicitly stated and mediating causal processes were not investigated. Nonetheless, the program was designed with some attention to theory and explicit cause and effect relations: for example, it emphasized the importance of community through the service component; it taught responsibility through the program requirements; it provided opportunity through the rich set of offerings; it taught investment skills through earned stipends and college savings; it stressed the importance of meaningful adult relationships throughout the high school years; and finally, it recognized the importance of continuity across time by creating a four-year program. Each of these program design components is compatible with the committee’s list of features of positive developmental settings. The multiple components of the program produced multiple positive effects using a strong pre- and postexperimental design. Youth were not only randomly assigned to the program intervention and control groups, but they were also randomly selected prior to program recruitment. The positive effects of this study are particularly impressive considering the frequent difficulties in recruitment and retention of youth ages 15 to 18. The program was well implemented according to evidence of participation and retention rates, although there was no specific information regarding the quality of program activities. The theory behind the program was not well articulated in the evaluation design. Consequently, most of the goals proposed in the program design phase were not evaluated. In summary, this program seems very promising and is grounded in research on the importance of service learning and mentoring. In general, the program effects look positive; however, there is no explicit theory of change. Consequently, we know little about why and for whom it worked. Like the Teen Outreach Program, the program offers a service-learning component for young people having difficulties in school. The program also has a mentoring component that the staff thinks is critical for positive results. The evaluation study was relatively small (100 youth from 4 cities in the intervention). Finally, the program is quite expensive. The extent to which the payments to the youth are essential has not been assessed.

OCR for page 147
Community Programs to Promote Youth Development Big Brothers Big Sisters Evaluators: Grossman and Tierney, 1998 Level of intervention: Community and individual Target population: 10- to 16-year-olds Evaluation research design: Experimental design with random selection and assignment Sample: 1,138 (571 treatment, 567 control) Attrition analysis: 487 of the 571 treatment participants; 472 of the 567 control participants Cost: Average of $1,543/match/year Big Brothers Big Sisters is a community-based mentoring program that matches an adult volunteer, known as a Big Brother or a Big Sister, to a child, known as a Little Brother or Little Sister, with the expectation that a caring and supportive relationship will develop. The match is well supported by mentor training and ongoing supervision and monitoring of the mentor relationship by a professional staff member. Ideally, the matched pair spends three to five hours per week together over the course of a year or longer. The activity goals are identified in the initial interview held with the parent or guardian and the child. The top priority is for the matched pair to develop a relationship that is mutually satisfying and provides contact on a regular basis. The goals established for a specific match are developed into an individualized case plan, which is updated by the case manager as progress is made and circumstances change over time. The case manager is responsible for maintaining contact with all parties in the match relationship. In order to ensure effective matches between volunteers and youth and to monitor program quality, the professional staff screens all applicants, youth, and their families. Orientations are conducted with youth and volunteers, and training is provided for volunteers. Staff supervises matches between youth and volunteers by contacting all parties within two weeks of the initial match and then having monthly telephone contact with the volunteer for the first year of the program. The staff also contacts the youth directly at least four times in the first year. Evaluation Design and Results An experimental design using random assignment was used to evaluate the Big Brothers Big Sisters program at eight program sites: Phoenix,

OCR for page 147
Community Programs to Promote Youth Development Wichita, Minneapolis, Rochester, Columbus, Philadelphia, Houston, and San Antonio. Youth ages 10 to 16 who came to the agencies during the study period, October 1991 to February 1993, were randomly assigned to either the mentoring or the control group. Baseline data were collected, treatment subjects were matched with a mentor when possible, and control subjects were placed on a waiting list for 18 months. Pre-and posttreatment data consisted of self-assessment questionnaires on family background information and outcome variables. The evaluation sample consisted of 959 10- to 16-year-olds (487 treatment, 472 controls) from single-parent households. At pretest, 69 percent were between the ages of 11 and 13; 43.3 percent lived in households receiving public assistance; 27.1 percent were experiencing physical, emotional, or sexual abuse; 23 percent were minority girls, 15 percent were white girls, 34 percent were minority boys, and 28 percent were white boys. Matches were found for 378 (78 percent) of the 487 members of the treatment group; 109 of the treatment group were not matched. The evaluators suggested three main reasons for the failure to match the 109 treatment youth: (1) 33 became ineligible for the program because their parent remarried, the youth became too old, or the residence of the youth changed; (2) 31 because the youth no longer wanted to participate; and (3) 21 because the staff couldn’t find a suitable volunteer mentor. The other 24 were not matched for a variety of reasons such as parent or youth not following through on the intake process. Youth in the treatment group (including both those who received a mentor and those who did not) were less likely to initiate drug and alcohol use than the waiting list youth; they also reported hitting others less often. Their grade point average was higher, and they attended school more often. Finally they also reported better parental relationships and more parental trust. Strengths and Limitations The Big Brothers Big Sisters program is a very promising program based on the multiple benefits to youth of being mentored by nonfamilial adults. The random assignment evaluation design meets high standards. One major weakness, however, is the total reliance on self-reporting by the youth for the outcome measures. Another is the failure to include implementation or process analysis. As a result, we really know little about why the average-level effects are obtained and why the program may have worked very well for some youth and not at all for others. A

OCR for page 147
Community Programs to Promote Youth Development good qualitative evaluation component would have provided preliminary answers to these questions, and such an evaluation is now under way. A third weakness is the failure to gather follow-up data. Thus we know nothing about the length of positive impact. Finally, because Big Brothers Big Sisters does not have an explicit theory of change—the analysis of implementation quality does not go beyond descriptions of the number of contacts between mentor and young person and beyond their reports of satisfaction with the relationship. And with regard to this measure, many pairs did not meet as often as recommended by the program. A much more detailed study of implementation is now under way. Three other aspects of the evaluation need to be noted. First, the treatment group included some youth who were assigned to receive the program intervention but were never matched; thus the analyses may underestimate the effect size. Second, outcome effects were reported using relative rather than absolute percentages, thereby possibly inflating effect sizes. In summary, Big Brothers Big Sisters is a very promising community program for youth. The program is grounded in the research showing that positive relationships with nonfamilial adults support positive development. The researchers included all youth assigned to the treatment when they estimated the effects of the program. This procedure is likely to underestimate the impact of the program on those who actually received the services. Nonetheless, the program yielded significant positive benefits on multiple fronts. The evaluation was exemplary in some ways and limited in others. A strong methodological design with random assignment was used to evaluate the program. However, one could be more confident about the results if there were long-term follow-up data, the analysis included site-level as well as individual-level information, and more was known about the characteristics of each of the eight sites. CONCLUSIONS ABOUT PROGRAM FEATURES The committee was very interested in understanding why programs are effective and particularly how and why they are effective in promoting adolescent development. The quality of information in most studies about the particular features of effectiveness, however, was minimal. Furthermore, these studies were not necessarily designed to collect information about those aspects of adolescent development outlined in this report. However, we read the descriptions of these studies carefully for

OCR for page 147
Community Programs to Promote Youth Development clues as to why they might have worked and how they were related to the elements of the development framework we hypothesized as being important at the beginning of this report. Each program, in fact, included several components that were consistent with this framework. We learned that many programs can effectively promote healthy development, although we learned much less about why. Without such information, we cannot say very much about which particular components or combination of components were responsible for the programs’ effectiveness. We therefore cannot draw firm conclusions about which and how many of the features of settings in Chapter 4 were critical for the success of these programs. Nevertheless, in this section we attempt to compare these programs with our framework. It is important to keep in mind that we imposed our list of setting features onto programs that were not designed with this particular framework in mind; we therefore had to rely on our interpretation of the program descriptions and other information provided in the evaluation materials to infer whether a program offered a particular feature or not. Supportive Relationships Supportive relationships were a major component of most of the effective programs reviewed. Explicit mentoring activities are the most obvious examples of nurturing supportive relationships, but most of the programs provided youth with some form of regular supportive contact with nonfamilial adults. In many cases, mentoring was not explicitly stated as a program goal, but the adult-adolescent contact in these programs often took the form of mentoring. For example, Quantum Opportunities used an intensive case management approach to tailor their program to meet individual needs and circumstances. This approach required regular supportive interactions between the case manager and the adolescent. In addition, many of the programs officially designated mentoring as the central component of the program. The best example of this is Big Brothers Big Sisters. The positive effects of mentoring were clearly documented in the evaluation of this program. Follow-up was done to understand the factors contributing to supportive mentoring relationships. Through telephone interviews with 1,101 mentors in 98 mentoring programs and through youth focus groups and youth interview data (Herrera et al., 2000), the evaluators found that mentors with the closest and most supportive relationships reported more than 10

OCR for page 147
Community Programs to Promote Youth Development hours of contact per month, shared interests between the mentor and youth, and shared decision making about activities. Opportunities to Belong Healthy development is promoted by fostering a sense of belonging, an area which the Quantum Opportunities Program stressed. Instilling a sense of belonging was at the core of its mission as reflected in their motto, “Once in QOP, always in QOP.” If participants stopped attending activities and disappeared from the program, program staff tracked them down to find out what was wrong, to assure them they were still part of their QOP “family,” and to coax them back to the program. This attitude of refusing to give up on youth was considered by Hahn et al. (1994) to be a crucial component of the program. Although this outreach might have been seen as coercive, evidence from the student evaluations of the program suggest that this was not the case: for three of the four sites evaluated, 78–92 percent of the students thought the program was “very important” and 82–92 percent were “very satisfied” with the program. It is unlikely that students would give these responses if they felt coerced to attend or hassled by unwanted adult attention. Positive Social Norms Even when not explicitly stated as a program goal, all of these programs promoted positive social norms and discouraged norms related to the major problem behaviors of concern in these programs. Programs with a community service component—Quantum Opportunities and the Teen Outreach Program—best exemplify this feature. Youth participants were matched with an adult mentor and received classroom-based social problem-solving skills training and participated in community service activities with their mentors. The discussions in school were explicitly designed to support positive social norms among the students participating. In many cases, parents also participated in workshops, to enhance their ability to help their children maintain positive social norms and reject antisocial norms and pressures from peers to participate in problem behaviors. Programs that taught resistance skills also exemplify this feature.

OCR for page 147
Community Programs to Promote Youth Development Support for Efficacy and Mattering Providing opportunities for autonomy, taking responsibility, and challenge are likely to be especially important for older adolescents as they approach the transition to adulthood. Most of the programs reviewed in this chapter did not address opportunities for developing a sense of efficacy. More such programs were reviewed in Chapter 5. However, programs with a focus on skill acquisition, such as the Penn Prevention Program, do exemplify this characteristic because such learning opportunities, if implemented as discussed in Chapter 4, are the best settings in which to acquire a strong sense of personal efficacy. We talk more about this in the next section. Support for mattering was more clearly evident in the effective programs reviewed in this chapter. This feature was most evident in programs that provided opportunities to participate in community service. Two of the model programs, Quantum Opportunities and the Teen Outreach Program, were successful in involving high school students in community service, life and family skills training, and support in planning for college and jobs. Opportunities for Skill Building Opportunities for skill building were plentiful in the programs reviewed. An emphasis on social skills was a frequent program goal. Building skills in resisting peer influences to engage in a wide range of problem behaviors (such as unprotected sex, drinking and drug use, and illegal behaviors) was common in programs primarily focused on preventing such problems as drug and alcohol use, teen pregnancy, and HIV/AIDS (see program descriptions in Table 6–1 for ENABL, Life Skills Training, Project ALERT, Positive Youth Development Program). Also, many of the programs focusing specifically on the parent-youth relationship include activities to teach youth and parents better communication skills (e.g., the Adolescent Transitions Project, Creating Lasting Connections, Functional Family Therapy, Improving Social Awareness-Social Problem Solving, the Midwestern Prevention Project, the Social Competence Program for Young Adolescents, the Teen Outreach Program). Other programs focus on individual cognitive, social, and emotional skill development. Some focus on such social and emotional skills as coping and self-regulation (Coping with Stress Course, Functional Family Therapy, the Penn Prevention Project). Others provided employment or economic management skills through part-time employment with

OCR for page 147
Community Programs to Promote Youth Development mentoring, encouragement for opening savings accounts, and financial incentives for participation (Quantum Opportunities, the Summer Training and Education Program, Louisiana State Youth Opportunities). Integration of Family, School, and Community Efforts Several programs excelled in integrating family, school, and community into their program design. The Midwestern Prevention Project includes a set of program activities to prevent adolescent drug use: (1) mass media programming with news clips, commercials, and talk show discussions on drug use incorporated with information on their program; (2) a school-based program teaching resistance and counteraction skills for drug use; and (3) parent education and organizing. Project Northland also used a community-wide approach. Students received skills training to enhance their social competency in dealing with their parents, their peers, and the norms surrounding alcohol use. Parent education and involvement was also stressed through parent-student homework activities and through newsletters to parents containing educational information. Community-level changes in alcohol-related programs and policies were also targeted. Finally, the Valued Youth Partnership Program participants were given training in how to tutor and then engaged in tutoring of younger students for at least four hours per week. Parents were involved in school activities, and students were exposed to role models in the community through presentations and field trips. Structure and Safety These two characteristics of positive developmental settings from our list—appropriate structure and physical and psychological safety—received little mention in the program descriptions. Perhaps this omission reflects that fact that these characteristics are considered so basic that they are not worthy of explicit mention. One of the fundamental reasons for community programming for youth is to provide safe places for them to go. Wisdom would dictate then that programs must enable youth to feel and be psychologically and physically safe. To do this, programs need to have clear and consistent rules, expectations, and boundaries. Nevertheless, the absence of these features is likely to be a major reason why programs fail and why some programs produce negative results.

OCR for page 147
Community Programs to Promote Youth Development None of the programs included in this chapter dealt explicitly with the use of developmental appropriate levels of structure. Some of those reviewed in Chapter 5 did. These programs provided more opportunities for self-monitoring and active participation in rule making and enforcement as the youth matured. SUMMARY Our review of experimental and quasi-experimental evaluations of community programs for youth ages 10 to 18 leads to several general conclusions about these programs and the evaluation of these programs. Community programs for youth, whether they are packaged in teen pregnancy prevention programs, mental health programs, or youth development programs, can facilitate positive outcomes for youth. This review of experimental and quasi-experimental evaluation of community programs for youth revealed that participation is associated with increases in such outcomes as motivation, academic performance, self-esteem, problem-solving abilities, positive health decisions, interpersonal skills, and parent-child relationships, as well as decreases in alcohol and tobacco use, depressive symptoms, weapon-carrying, and violent behavior. Although many of these studies were not designed around a positive youth development framework, many of the evaluations in fact included measures that reflected the personal and social assets and features of settings developed in this report. Even with the most rigorous methods of evaluations, there is limited evidence that measures the impact of these experiences on the development of young people and therefore limited evidence on why program effects are or are not obtained from the evaluations reviewed. Some of these studies explicitly involved components consistent with the developmental framework outlined in Part I. However, without appropriate information, we cannot say very much about which particular components or combination of components were responsible for a specific program’s effectiveness. Consequently, research is needed to sharpen the conceptualization of features of community programs and to explore whether other key features should be added to the list. This work should focus on how different populations are affected by different program components and features (e.g., age, gender, socioeconomic status, ethnicity, community environment, developmental readiness, personality, sexual orientation, skill levels). It should also focus on how to incorporate these features into community programs and on how to maintain

OCR for page 147
Community Programs to Promote Youth Development them once they are in place. Finally, such research should identify program strategies, resource needs, and approaches to staff training and retention that can cultivate and support the features of positive developmental settings in community programs for youth. Also, it is our view that evaluations designed with some of the principles outlined in the next chapter will tell us much more about why particular programs work and for whom they are likely to be most effective. And new programs can be designed with some of the principles outlined in earlier chapters. If such programs are designed from the beginning with a well-articulated theory of change grounded in solid developmental theory and research, we will learn a lot more about what programs might do to facilitate positive youth development and to prevent problems from emerging,

OCR for page 147
Community Programs to Promote Youth Development This page in the original is blank.