National Academies Press: OpenBook

Community Programs to Promote Youth Development (2002)

Chapter: 8 Data and Technical Assistance Resources

« Previous: 7 Generating New Information
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

CHAPTER 8
Data and Technical Assistance Resources

  • Participates in volunteer activities

  • Has a positive outlook

  • Reads for pleasure

  • Misses school regularly

  • Has been arrested

  • Participates in a gang

Over the past decade, communities, cities, states, and nations have become increasingly interested in knowing what percentage of their youth population is doing well and what percentage is doing poorly (Brown and Corbett, forthcoming; Kingsley, 1998). To estimate these numbers, scientists and policy makers have generated both a growing list of social indicators of well-being and problem behaviors and set about gathering the data. As a result, there are now data and related technical assistance resources to support youth development work. This work, however, has also produced a keen awareness among practitioners of the limitations of these resources and the need to increase their quality, breadth, and availability in the coming decade (MacDonald and Valdivieso, 2000).

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

In this chapter we review these resources in light of the youth development framework developed in Part I and the variety of practical applications for which the data and resources are needed. Our review of data and resources includes: (1) relevant administrative and vital statistics data that are commonly available at the community level, (2) community surveys and topic-specific instruments that can be used to track aspects of youth development not well covered by administrative data, and (3) selected national surveys focusing on youth. We then discuss data collection to support implementation and reflective practice in individual community programs for youth. Finally, we review the efforts of key national intermediary organizations that have developed resources and technical assistance materials for use by local youth development efforts interested in social indicator data. The chapter concludes with suggestions regarding: (a) the need for enhanced access of community youth programs to social indicator data and to the training needed to use them effectively, (b) the development and fielding of new measures, and (c) an expanded role for national intermediaries to support these goals.

USES OF SOCIAL INDICATOR DATA

The uses of social indicator data for community programs for youth and youth initiatives include needs assessment, service targeting, goals tracking, program accountability, and reflective practice to improve program and policy effectiveness over time. Any assessment of available data resources to support this work needs to be understood in the context of their use.

Needs and Resource Assessment

Assessing the needs of youth in a community is a common starting point for many community youth initiatives. Whole communities may do a general assessment using available data to identify the areas of greatest need for their youth, which can then be addressed in a coordinated fashion by multiple programs and agencies. Organizations (e.g., the United Way, the YWCA, a local synagogue) can also use such data to shape their own programs, targeting their efforts in areas of greatest need (United Way of America, 1999). Strengths in the community (cultural, financial, etc.) that could be mobilized to meet the needs are often identified as part of the same process.

Most communities are limited to available administrative data

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

sources, such as service program data, health surveillance, vital statistics, school performance data, decennial census, police reports, and so on. Because of the limited nature of these data, some communities are fielding their own surveys for a more complete assessment. In many cases, however, community programs for youth are started in response to needs that are obvious to those who work with the youth without consulting any data resources. Nonetheless, however, a more formal documenting of need can help even in these cases to elicit the cooperation and interest of other actors both inside and outside the community, including funders.

Tracking Progress Toward Goals

Virtually all youth initiatives have some goals that can be translated into social indicator language. Community-wide youth initiatives often set goals to improve selected dimensions of youth well-being in specific and measurable ways. These goals then serve to focus the activities of multiple organizations in the community. In Tillamook County, Oregon, for example, the community agreed to focus on lowering the teen birth rate, which was then one of the highest in the state. Many local organizations, some with very different ideas and strategies for addressing the issue, worked to reduce teen births in the county. Over a period of four years, the rate was cut by 70 percent, giving the county the lowest teen birth rate in the state (National Campaign to Prevent Teen Pregnancy, 1997).

The federal Healthy People 2010 initiative is another useful example. Participating states and communities adopt specific goals across a host of health outcomes to be achieved by the year 2010.1 Public and private health organizations focus their efforts on one or more of these goals so that, together, they can make measurable progress at the community or state level. Social indicator data are the primary source used to assess this progress. Individual programs can also use social indicator data as tools for setting and tracking progress toward their specific goals.

Accountability

Social indicator data are increasingly used by funders to hold programs and initiatives accountable for measurable results in many areas,

1  

For more information, visit <http://web.health.gov/healthypeople>

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

including youth well-being and development. Successful improvement in the targeted areas of development may be met with increased funding. Failure to meet specified goals may result in the provision of technical assistance to overcome problems or in the loss of funding or autonomy. Such accountability practices are becoming increasingly common in state and local educational assessment efforts, for example (Brown and Corbett, forthcoming). Community-level indicators are rarely used for this purpose, as individual programs are not generally expected to have a community-wide impact on youth well-being. Instead, they are held accountable for outcomes to program participants.

Reflective Practice

Community programs for youth can also use social indicator data to monitor their own success and to guide program refinement (i.e., they can use social indicator data as part of reflective practice). At its most formal level, programs can develop a detailed model relating program activities to interim and long-term project goals for participating youth (United Way of America, 1999; Gambone, 1998; Weiss, 1995). In the best case, such a model of the links between program activities and youth outcomes will be based on existing research (when it exists), theory, and the shared beliefs of those running the program. Both program activities and youth outcomes can then be measured and tracked over time. Failure to produce the expected results could indicate inadequate implementation of some part of the program, or it may call into question one or more of the underlying assumptions of the model. Practices, the model, or both can then be reevaluated as a result and the programs can be modified.

In many respects reflective practice functions like program evaluation, even though it lacks the methodological rigor required to draw firm causal inferences about the relations between program activities and youth outcomes (see Chapter 7 for discussion of evaluation methodologies). However, the level of certainty required to qualify results as scientific knowledge is not needed to produce good guides for responsible program management. And programs can sometimes incorporate some of the practices associated with experimental evaluations into their reflective practices. For example, they can compare the social indicator data from their catchment area with social indicator data being collected longitudinally in other comparable catchment areas (stimulating a quasi-

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

experimental design, provided that the data are being collected over time beginning before the program change is put into place).

DATA SOURCES

Communities and youth programs interested in tracking indicators of youth development have two major sources of data to draw from: administrative and related data sources (e.g., school records, crime reports, social service receipt, vital statistics, decennial census) and surveys. Every community already has data relevant to youth development from its administrative data collection efforts, although there can be substantial differences across communities in the accessibility of those data to the public, even among the agencies that collect them. Few communities go to the added effort and expense of collecting survey data, although this is an excellent way to achieve a complete picture of the status of youth and the social factors affecting their development. Over the past decade, however, the number of communities conducting surveys has increased substantially.

In this section we review the data resources available to communities through the lens of the youth development framework developed in this report. This framework identified four outcome domains of development—described as personal and social assets—in Chapter 3 and eight social setting domains—described as features of positive developmental settings in Chapter 4. We also include “negative outcomes and behaviors” as a separate outcome domain.

We examine the types of indicator data available in each of the domains in our framework for commonly available administrative data, and for three of the more advanced survey instruments used for assessing youth development at the community and state levels. These are reviewed in terms of their coverage across the domains, data and measurement quality, and the extent to which their measures are grounded in the scientific literature. Topic-specific research instruments and national surveys are also discussed.2 The section finishes by considering issues of public access to the data generated by these sources.

2  

Several major federal publications series, not reviewed here, provide regularly updated trend data on children and youth across a wide variety of domains. These include:

Trends in the Well-Being of America’s Children and Youth (<http://aspe.hhs.gov/hsp/99trends>)

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Administrative and Related Sources

Every community collects a substantial amount of data on their children and youth. Administrative sources include school records, educational assessments, vital statistics (birth and death records), police data on reports and arrests, child welfare and public assistance records, health surveillance systems, emergency room admissions data, and so on (Coulton and Hollister, 1998; Coulton, 1995). In addition, data from the decennial census provide detailed economic and demographic information on youth, their families, and their neighborhoods.

These data have a number of advantages in addition to their ubiquity. First, someone has already paid for their collection. If they are not already available to the public, they can often be made available at a relatively modest cost. Second, many of these sources are capable of generating indicator estimates down to the neighborhood level. This is particularly important for community programs for youth, many of which serve limited geographic areas within the larger community. Neighborhood-level data allow programs to make strategic location decisions to areas of greatest need, assess needs and strengths in neighborhoods they already serve, and track changes in these characteristics over time. The last is particularly important for community programs for youth when their goals extend beyond program participants to include changes at the neighborhood level (e.g., reductions in gang activity).

Third, administrative data provide information that cannot be gathered using existing community surveys on youth development. Standardized education assessments, for example, are important measures that are not covered by such surveys. Neighborhood characteristics (e.g., crime levels, teen birth rates) derived from administrative reporting systems provide indicators of social settings that are more objective in the sense that they are not dependent, as the surveys are, on youth perceptions.

Some potential problems with these data should be kept in mind. First, many of these sources can have problems with data quality and consistency (Coulton and Hollister, 1998; Coulton, 1998). For example,

Youth Indicators 1996: Trends in the Well-Being of American Youth (<http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=96027>)

America’s Children: Key National Indicators of Well-Being: 2000 (<http://www.childstats.gov>)

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

changes in police policy for the disposition of juvenile offenses may result in dramatic increases or decreases in arrest rates without any change in the actual rate of juvenile crimes committed. Even census data, which are very reliable overall, have been shown to substantially underestimate the number of both black and white children in large urban settings (West and Robinson, 1999).

Second, data sources often use different and incompatible geographic units when they report (e.g., school catchment area, census tract, health district, or other specially defined service area). This can limit the ability to draw on multiple data sources to produce a picture or profile for particular neighborhoods.

Other problems can include a lack of separate estimates for important population subgroups (e.g., Hispanics), long time lags between data collection and release, and a lack of public accessibility. In many communities, estimates that could support planning efforts outside local government never make it beyond the walls of the agencies and departments that collect them.

Table 8–1 presents the types of indicator data available from administrative data and from the three community survey instruments we reviewed, sorted by the domains in our youth development framework. In the youth outcome domains, administrative data are quite strong in the areas of cognitive development, negative outcomes or behaviors, and physical health. They are weak sources of indicator data in the areas of psychological, emotional, and social development. And, with the exception of the cognitive development domain, the indicators are heavily weighted toward negative outcomes. This should not be surprising given the reliance on crime, vital statistics, health surveillance, and social service receipt data, all of which focus primarily on negative events.

Among the social setting domains that influence youth development, administrative data are strongest in the safety, social norms, and opportunities for skill building domains. They are weak to nonexistent in the structure, emotional and intellectual support, opportunities for efficacy, and integration domains.

Community-Level Surveys of Youth

For this review we have chosen three of the best known and most widely used surveys of youth at the state and community levels. All three surveys are based entirely on youth reports, are commonly administered

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

in the classroom, are relatively inexpensive to administer and process, and take an hour or less to complete. These are:

  • Profiles of Student Life: Attitudes and Behaviors;

  • the Student Survey of Risk and Protective Factors, and Prevalence of Alcohol, Tobacco, and Other Drug Use; and

  • the Youth Risk Behavior Survey.

Profiles of Student Life: Attitudes and Behaviors

The Profiles of Student Life: Attitudes and Behaviors (PSL-AB) survey, developed by the Search Institute, has been administered in over 1,000 communities since 1989. It is designed for youth in grades 6 through 12 and is based on a comprehensive framework grounded in the youth development literature (Scales and Leffert, 1999). The framework includes eight asset areas: four internal (commitment to learning, positive values, social competencies, and positive identity) and four external (support, empowerment, boundaries and expectations, and constructive use of time). There are 40 assets in all, as well as multiple measures of thriving and risk behaviors.

The questions used in the survey were culled primarily from national surveys. About half of the assets are measured as scales based on three or four items. A psychometric analysis of these scales revealed Cronbach’s alpha coefficients ranging from to .31 to .82, with about two-thirds of the scales exceeding the .60 level, a common cutoff point used in research. The validity of the assets is primarily face validity based on their relationship in the research literature to the promotion of healthy behavior, the prevention of risk behaviors, or both. Analyses based on PSL-AB data revealed strong relationships between the number of assets a youth has and the prevalence of risk and thriving behaviors. These analyses are based on large but nonrepresentative and disproportionately white samples across multiple communities (see Leffert et al., 1998, for details).

Although the assets and their representative measures are grounded in existing academic research, the research base is, as its designers freely admit, rather thin or mixed for a number of the assets, such as empowerment, positive values, cultural competence, self-esteem, and sense of purpose (Leffert et al., 1998).

Clearly, some of the measures used in the PSL-AB race ahead of the underlying science, and a number of the constructed scales fall well be-

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

TABLE 8–1 Youth Development Outcomes and Social Settings Measures in Administrative Data and Community Surveys

 

Community Administrative Data

Profiles of Student Life: Attitudes and Behaviors (PSL-AB)

Student Survey of Risk and Protective Factors, and Prevalence of Alcohol, Tobacco, & Other Drug Use (SSRP)

Youth Risk Behavior Survey (YRBS)

Youth Outcomes

 

Cognitive development

School grades; standardized test scores/assessments; high school dropout or completion rates

Achievement motivation; school engagement; time doing homework; reads for pleasure

Academic achievement; school engagement

 

Psychological development

Suicides

Moral character (caring, social justice, integrity, honesty); self-efficacy; self-esteem; mattering; positive outlook; depression; attempted suicide

Moral character (attitudes toward negative behaviors, drug use); honesty with parents; depression

Sad/depressed; attempted suicide, suicide ideation; vomit or take laxatives to keep from gaining weight

Emotional development

 

Takes personal responsibility; plans ahead; makes choices

Rebelliousness

 

Social development

Participation rates in school clubs, organizations, and

Volunteer/service activities; participation in creative

Skills negotiating with parents; peer pressure

Hours watching TV

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

 

other extracurricular activities

activities (music, theater, and other arts); participation in sports, clubs, or other organizations, in or out of school; empathy, sensitivity, and friendship skills; cultural diversity (knowledge of and respect for other groups); peer pressure resistance skills; hours watching TV

resistance skills regarding drinking

 

Physical development

Participation rates in school sports and physical education; death rate by cause; violent youth crime victimizations; incidence of school violence; prenatal care receipt by teen mothers

Participation in sports activities; physical health behaviors; whether a victim of violence

 

Vigorous exercise; strengthening exercise; physical education classes; sports team activity; height, weight; safety (helmet, seat belt use); exercise-related injury; threatened or injured with a weapon; forced to have intercourse; tried to quit smoking; pregnancy prevention and condom use; nutrition (detailed); dieting; timing of last routine physical exam; pre

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

 

vention of pregnancy, AIDS, or other sexually transmitted diseases discussed at exam; timing of last visit to dentist; use of sun screen/sun block of SPF15 or higher

Negative outcomes and behaviors

Teen birth rate (total and nonmarital); teen rate of STD; teen arrest rate, by type of arrest (violence, drugs, truancy); school suspension or expulsion; school dropout

Drug use; sexual activity; anti-social behaviors; violence; driving and alcohol; school problems; gambling; attitudes toward drug use, sexual activity

Drug use (detailed); school suspension, arrested, carried handgun, fighting; sold drugs; gang membership; perceived risk in using drugs; intention to use drugs as an adult; engaged in negative risk-taking activities

Drug use (detailed); driving and alcohol; carry weapon, gun; fighting; physical fighting with boyfriend/ girlfriend; sexual activity (detailed); drugs and intercourse; pregnancy

Social Setting

 

Appropriate structure

 

Clear rules and monitoring (family, school); neighbors monitor young people’s behavior

Clear rules and monitoring (family); youth consulted in family decisions that affect him/her; youth perceives opportunities to shape school activities and rules

Ever asked to show proof of age when buying cigarettes?

Physical and

Reports of abuse/neglect

Feels safe in home, school,

Youth feels safe at school;

School or trip to school

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

psychological safety

of youth; youth injuries requiring hospitalization; violent crime rate; school violence reports

and neighborhood

community disorganization; perceived availability of drugs, hand guns; family conflict

seen as unsafe

Supportive relationships

 

Positive parent/youth communication; family provides high levels of love and support; support from adults besides parents; community support (caring, value youth); school environment as caring, encouraging

Positive parent/youth relationships (closeness, activities); number of home and school moves; perceived community support for prosocial involvement

 

Belonging

Gang membership (police report)

Youth cares about the school; religious attendance; hanging out with friends

Religious attendance; community attachment; gang membership

 

Positive social norms

Neighborhood crime rates

 

 

High percentage of single-parent families in neighborhood; high rates of social service receipt in neighborhood

Parents and other adults provide positive role models; friends model positive behaviors

Friends’ involvement in drug use, arrest, school suspension, school dropout, theft; peer attitudes on drug use, carrying handgun; friends in gang; neighborhood laws

 

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

 

and norms favorable to drug use; parents’ attitudes about drug use, antisocial behavior; antisocial behavior by other family members (drug use, carry gun, suspended/expelled from school

 

Support for efficacy

 

Parents help youth to succeed in school; youth encouraged to do well (parents and teachers)

Youth perceives lots of about chances to engage in class discussions and activities; teachers praise youth when they work hard and do well

Taught about AIDS/HIV in school

Opportunities for skill building

Proportion of youth required to perform community service; capacity of community-sponsored youth programs (centers, sports, etc.); students with Individual Education Plans; percent of students in gifted/talented

Youth given useful roles within community

Opportunities for prosocial involvement in community (sports, scouting, 4-H, service clubs); youth perceives many opportunities for involvement in school sports, clubs, etc.

 

Integration of family, schools, and community

Proportion of parents who come to parent-teacher conferences

Whether parents come to school meetings/events

School lets parents know when youth performs well

 

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

low accepted research minimums. However, the primary goal of the Search Institute in developing and promoting the use of this survey was to inform social action by providing communities with a common language, a unified and complete vision of positive youth development, and the means to identify and strengthen the developmental processes in their community. Science played and continues to play an important role in the development of this instrument, but there is a willingness to go beyond the science when it serves an important practical purpose.

The use of the assets framework to guide instrument construction has resulted in a very comprehensive survey that includes measures in every domain of our own youth development framework and multiple measures in most domains (see Table 8–1). It is the most well-rounded of the data sources included in this review. There are, however, no cognitive measures per se. Instead there are measures of academic motivation, school engagement, and related activities like time doing homework. Measures of psychological and emotional development include a host of measures related to moral character, as well as both positive and negative measures of mental health and measures of adolescents’ capacities to take responsibility for their actions, plan ahead, and make their own choices. The survey is particularly strong in social development measures, including participation in sports, clubs, music, art, and theater activities both in and out of school; volunteering in the community; capacity for empathy and sensitivity; friendship skills; and respect for cultural diversity. Physical health measures include participation in sports activities, whether the student “takes care of their body,” sexual activity, and whether the youth has been a victim of violence. The list of negative outcomes and behaviors is broad and includes questions about drug use (including alcohol and cigarettes), violence, drunk driving, gambling, and eating disorders.

The coverage in the social setting domains is nearly as impressive. Measures of structure include the presence of clear rules and monitoring of the youth’s activities in the family, the school, and the neighborhood. Supportive relationships are explored in the family, with adults outside the family, in the school, and in the community. Measures of the opportunity to belong are weaker and rely only on whether the youth care about their school. Measures of social norms focus on the presence of positive role models among parents, friends, and other adults. Questions on safety focus on whether the youth feels safe at home, in school, and in the neighborhood. Support for efficacy and mattering questions focus on levels of encouragement and help to do well in school and on whether

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

the youth feel that they are given useful roles in the community. There are questions on skill building per se, and information in the final domain (integration of family, school, and community) is limited to whether parents come to meetings or events at school.

Student Survey of Risk and Protective Factors, and Prevalence of Alcohol, Tobacco, and Other Drug Use

The Student Survey of Risk and Protective Factors, and Prevalence of Alcohol, Tobacco, and Other Drug Use (SSRP), developed by the Social Development Research Group at the University of Washington, focuses on the risk and protective factors influencing drug use, violence, and misbehavior in the lives of youth ages 12 to 18. The survey was specifically designed to inform the development of preventive interventions to reduce youth risk behaviors in schools and communities and to support outcome evaluations of such interventions (Pollard et al., 1999). Development was funded by the Center for Substance Abuse Prevention of the Substance Abuse and Mental Health Services Administration, U.S. Department of Health and Human Services. The survey is now being fielded with representative statewide samples in six states: Kansas, Maine, Oregon, South Carolina, Utah, and Washington.

Risk and protective factors were included in the survey only when they had been found to predict future drug use and criminal or delinquent behaviors in at least two longitudinal studies in the research literature. The final instrument resulted from a rigorous development process that included cognitive pretesting, and pilot tests, as well as validation of risk and protective factors based on statewide samples of 6th, 8th, and 11th grade students in Oregon. Risk and protective factors are measured using multi-item scales. All scales exhibit high reliability with alphas of .65 or higher, with most in the high .7 and .8 ranges. The factors have good predictive validity in their relation with the problem behaviors in the survey (Pollard et al., 1999a; Arthur et al., undated).

The survey’s coverage of the domains in our youth development framework is broad in the sense that there is at least one measure in every outcome and social setting domain except one, physical health. However, a closer look reveals that the instrument is considerably more focused on negative outcomes than the PSL-AB. While the negative outcomes and behaviors domain is richly detailed, measures in the other outcome domains are a bit thin and tend to be defined in relation to risk behaviors. In the psychological and emotional development domain, for

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

example, measures of moral character focus on attitudes toward negative behaviors and activities, and the measures of psychological health are limited to depression. One of the two social development measures asks about peer resistance skills against drinking.

The indicators of social settings are considerably more well rounded and positive, although even here measures in the social norms domain focus totally on risk behaviors on the part of friends and family, and attitudes about risk behaviors. The survey contains multiple positive measures for structure, supportive relationships, support for efficacy and mattering, and opportunities for skill building. There is overlap between the PSL-AB and the SSRP constructs in these domains, although each contains important constructs that are not present in the other. In the SSRP, for example, there are several constructs related to the youth’s role in decision making in the family and the school.

Youth Risk Behavior Survey

The Youth Risk Behavior Survey (YRBS), developed by U.S. Centers for Disease Control and Prevention (CDC), was fielded in 42 states and 16 major metropolitan areas in 1999.3 A national survey was also fielded. It is a school-based, self-report survey of students in grades 9 through 12. Several states, including Alaska, Montana, Nevada, Oregon, Utah, and Vermont, have in some years provided the additional funds needed to generate samples for all school districts in the state that wanted to participate. One of the great advantages to communities who field this survey, in addition to the intrinsic value of the data, is the availability of comparable estimates for their state as a whole, as well as for the nation. A significant limitation is that it does not provide information on out-of-school youth, a population of great interest to many community programs.

The survey was first conducted in 1990 and has been administered every two years since 1991. It was designed by national experts in the field of adolescent health, with substantial input from representatives from all 50 state and 16 metropolitan-area education agencies. A detailed rationale, grounded in adolescent health research literature, was developed for all of the measures included in the survey instrument.4

3  

2001 Youth Risk Behavior Survey: Item Rationale for the 2001 Questionnaire. <www.cdc.gov/nccdphp/dash/yrbs/2001rationale.htm>

4  

To download a copy of the 1999 report, go to <http://www.cdc.gov/nccdphp/dash/yrbs/index.htm>

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

The survey is composed entirely of single-item measures; there are no multi-item scales. States may add their own questions to the survey and may also choose not to field questions that they find problematic. It is not uncommon for states to omit certain sex-related questions, for example.

The CDC works with participating state and local education agencies, which administer the survey, to produce survey samples that are representative of their youth in-school populations. In 1999, 22 of the 42 participating states fielded representative samples. CDC staff receive raw data from participating agencies, process the data, and provide estimates back to the states. In addition, summary results that include estimates for participating states and metropolitan areas are published by the CDC (Centers for Disease Control and Prevention, 2000c).

In comparison with the other surveys in Table 8–1, this survey focuses rather narrowly on health-related conditions and behaviors. There are, for example, no questions related to cognitive or social development and only four questions refer to the social setting in which youth develop (including one focused on television watching). However, the questions in the domains of physical development and safety and negative outcomes and behaviors are very rich, covering important issues not touched on in the other surveys. Questions unique to the YRBS include: height and weight (from which one can develop an obesity measure); forced intercourse; pregnancy; the co-occurrence of drug use and intercourse; detailed nutrition information; use of sunscreen or sunblock at SPF15 or higher; timing of last routine physical exam; incidence of physical fighting with one’s boyfriend or girlfriend; and incidence of vomiting or taking laxatives to prevent weight gain. In addition, the questions about physical exercise and sexual activity are more detailed than in the other surveys.

Topic-Specific Survey Instruments

In addition to the more general-purpose surveys just discussed, there are many more narrowly focused instruments that have been developed specifically for youth program monitoring and evaluation purposes. These instruments focus on individual constructs and commonly go into more depth on a topic than the more general surveys. They are most useful to community programs for youth trying to affect a particular outcome, like youth conflict resolution skills or teen pregnancy.

An effort led by the U.S. Department of Agriculture’s Cooperative

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

Extension Service professionals working with the Children, Youth, and Families At-Risk Initiative and Evaluation Collaboration Project has resulted in the cataloguing of well over 100 such measurement instruments, with separate work groups focusing on outcomes for children, youth, the family, and the community. This information was created for use by those engaged in USDA-supported State Strengthening community projects, a major community development program.

As part of this project, the Youth National Outcome Work Group identified multiple available instruments for the following outcome areas:

  • Risk behaviors: risk-taking, substance abuse, sexual activity, academic risk, delinquency, and violence.

  • Social competencies: social competence, relationships, conflict resolution, decision making, social responsibility, communication, goal setting, problem solving, social and environmental navigation, and valuing diversity.

The workgroup produced a searchable database, available on-line,5 which includes the following information for each instrument: name, author, basic description, scales and subscales in the instrument, psychometric information (reliability and validity), notable advantages and disadvantages, cost, contact information, and references. A separate work group focused on the family is producing a similar set of reviews for measures related to parent and family well-being. Some of the measures they have collected are directly applicable to the families of adolescents. These work groups are now disbanded, but their work continues to be a valuable resource.

National Survey Instruments

National surveys that focus on aspects of youth development have a number of strengths that make them good potential sources of measures that may be appropriated for use at the local level. First, these surveys are all well designed by some of the nation’s top researchers. Second, they offer the possibility of a national benchmark against which localities can compare their own status. Third, and perhaps most important,

5  

The web site is located at <http://ag.arizona.edu/fcr/fs/nowg/index.html>/.

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

three of the national surveys we reviewed include parent and teacher as well as youth interviews. While parent interviews may be too expensive to undertake for community-wide surveys (in-school youth surveys are comparatively much less expensive), they may be appropriate at the program level, particularly if parents are actively involved in the youth program. Both parent and teacher surveys offer points of view that are different from those offered by youth, which can provide important additional information for those engaged in the development of youth policy and programs at the community level.

In addition, all of these surveys are longitudinal, meaning they collect information on the same youth at multiple points in time.6 This means that the youth development measures in these surveys can be rigorously assessed in terms of their predictive validity for future development, including the transition to adulthood.

Tables 8–2 and 8–3 list the measures and constructs, organized according to the domains in our youth development framework, for the following major national youth surveys:7

  • National Education Longitudinal Survey of Youth (NELS)

  • National Longitudinal Survey of Adolescent Health (Add Health)

  • National Longitudinal Survey of Youth, 1997 Cohort (NLSY97); and

  • Monitoring the Future (MTF).

Overall, these surveys have taken a holistic approach, offering questions in most of the 14 domains in our youth development framework. While youth outcome measures are based almost exclusively on youth reports across all surveys, measures of social setting often include responses from parents, teachers, and schools. The NELS and Add Health surveys are particularly strong in this regard, the NELS gathering information in every social setting domain from someone other than, or in

6  

Monitoring the Future is primarily a cross-sectional survey, but a longitudinal sample has also been followed over time (see Purpose and Design, Monitoring the Future Survey, at <http://monitoringthefuture.org/purpose.html>)

7  

For additional information on NELS88, visit <http://www.nces.ed.gov/surveys/nels88>

For additional information on Add Health, visit <http://www.cpc.unc.edu/addhealth>

For additional information on NLSY97, visit <http://www.bls.gov/nlsy97.htm>

For additional information on MTF, visit <http://www.monitoringthefuture.org>

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

addition to, the youth as well as measures of cognitive skills in several subject areas. These multiple sources offer richer data for understanding social settings, providing measures of settings that are not directly affected by the well-being of the youth who inhabit those settings.

Gaining Access to Data

In order for community youth development efforts to take advantage of social indicators, they need ready access to the data. For many years, it was common for administrative data collected by one agency to stay within that agency, even though other government agencies and local private organizations could greatly benefit by using it. This is still true in many communities, although the situation has improved substantially over the past decade due to advances in computerization, the rise of the Internet, an increase in comprehensive statewide and community-wide planning efforts, such as the Oregon Benchmarks and Vermont’s Framework for Collaboration, and the rise of private social indicator-focused projects such as the Annie E.Casey Foundation’s Kids Count effort (Brown and Corbett, forthcoming; Kingsley, 1998).

It is increasingly common for agencies to make their indicator data available to all through regular publications and on-line searchable databases. There has also been an increase in the development of data warehouses, through which indicator data from many sources are made accessible to the public. A prime example of this sort of effort at the community level is the Cleveland Area Network for Data and Organizing (CAN DO). Data from the decennial census, vital statistics, crime, child welfare, and other sources are made available in an on-line database that users can access and display in tables or maps for neighborhoods throughout Cleveland, Ohio, and the surrounding suburban municipalities. The data are stored in a geographic information system (GIS), which allows users to create complex social profiles of local neighborhoods and to map need in relation to available resources (e.g., the location of licensed child care programs in relation to families with young children). The intended consumers of these data include nonprofit, community-based organizations and government agencies that can use them to aid in planning, as well as academics, students, and the public. CAN DO is a member of the National Neighborhood Indicators Project, whose members include organizations in a dozen cities across the country similarly engaged in the development of neighborhood information systems (Kingsley, 1996).

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

TABLE 8–2 National Youth Surveys: Youth Development Outcomes

Youth Outcomes

National Educational Longitudinal Survey (NELS)

National Longitudinal Survey of Adolescent Health (Add Health)

National Longitudunal Survey of Youth, 1997 Cohort (NLSY97)

Monitoring the Future (MTF)

Cognitive development

Academic achievement; standardized test scores; school engagement

Academic achievement; standardized test scores; good decision making

Academic achievement; standardized test scores

Academic achievement; school engagement; standardized test scores

Psychological development

Self-worth; locus of control; self-efficacy; religiosity; moral character

Mental health; self-worth; moral character; self-efficacy; religiosity; positive outlook

Optimism; positive outlook; mental health

Religiosity; self-worth

Emotional development

N/A

Emotion self-regulation; emotional coping skills

Emotion self-regulation

Emotion knowledge

Social development

Connectedness (peers, parents; siblings); commitment to prosocial institutions (service); civic engagement

Commitment to conventional institutions; connectedness (parents, peers); connectedness to neighborhood; life skills

Connectedness (parents, peers); work involvement

Connectedness to peers; civic engagement; multicultural caring/ understanding

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

Physical development

N/A

Sexual behaviors; physical activity; physical health; sleep habits; physical difficulties; health care use; diet/nutrition; healthy habits (helmet and seatbelt use, sun exposure, safe sex); healthy habits knowledge (nutrition; safety; sex); pregnancy

Physical activity; nutrition/ diet; health knowledge; has physical/emotional condition; trouble sleeping

Physical activity; sleep; nutrition; seatbelt wearing

Negative outcomes/ behavior

Cigarette/drug/alcohol use; skipping/late for/cut/s uspended from school; fighting; illegal activities/ arrested; gang involvement; performs below ability (emotionally, cognitively, socially); teen parent

Drunk driving; school suspension; sexual victimization; sexually transmitted diseases; unsafe sex; cigarette/ alcohol/drug use; fighting; illegal activity/arrested; violent weapon possession; suicidal behavior; unsafe adventurous activities

Sexual activity; cigarette/ alcohol/drug use; illegal activities/arrested

Fighting; stealing; trespassing; property damage; arrest; anger; drugs/alcohol/cigarette use; carrying/using weapons; drunk driving

Other

Time usage

 

Time usage

 

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

TABLE 8–3 National Youth Surveys: Features of Social Settings Affecting Youth Development

Features of Settings

National Educational Longitudinal Survey** (NELS)

National Longitudinal Survey of Adolescent Health (Add Health)

National Longitudinal Survey of Youth, 1997 Cohort (NLSY97)

Monitoring the Future (MTF)a

Structure

Clear parental rules; parental monitoring; parental encouragement (P+Y for all)

Clear rules (Y); communication between environments (P); monitoring (P)

Parent: monitoring (Y); clear/fair rules (Y+P); teacher: clear/fair rules

Parent: clear/fair rules; monitoring; School: clear/fair rules

Supportive relationships

Positive parent-child interaction/relationships; positive child-other adult interaction; parental/other-adult encouragement (P+Y for all)

Positive parent-child interaction/relationship; closeness of parent-child relationship (P+Y for all)

Positive parent-child interaction/relationship; emotional support; parental encouragement (Y for all)

Positive parent-child interaction/ relationship

Belonging

Gang membership (Y); opportunity for extracurricular participation (Y+S); parental community involvement (Y+P)

Feel part of group; feel valued by others; opportunity for extracurricular participation;

opportunity for involvement with religious institution (Y for all) Opportunity for extracurricular participation; parental involvement in school/community (Y for all)

Civic involvement; opportunity for extracurricular participation; feeling a part of a group

Social norms

School policies (S); peer norms (positive and negative) (Y)

Peer norms (positive and negative); neighbor intervening (Y for all)

Peer norms (positive and negative) (Y for all)

Peer norms (positive and negative)

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

Physical and psychological safety

Perceptions of physical safety at school (Y+T)

Perceptions of physical safety; victimized; violence in home/school; presence of gun/alcohol/drugs in home (Y for all)

Perceptions of physical safety in neighborhood/school (Y)

Victimized

Support for efficacy

High parental/other-adults’/ peer expectations (Y+P); positive youth programs (Y); youth participates in rule making (Y+P)

High parental expectations; youth participates in rule making (Y for all)

Positive youth programs (Y)

High parental expectations

Opportunities for positive development/ skill building

Availability of positive extracurricular activities and sports teams (S); learning tools in the home (P)

Access to health services; availability of positive extracurricular activities (Y for all)

Learning tools in the home; availability of positive extracurricular activities (Y for all)

Availability of civic opportunities and positive extracurricular opportunities

Integration of family, school, and community

Parental involvement in community/school; parent knowledge of child’s friends and friends’ parents; English language competence (child and parent) (P+Y for all); communication between environments

Parent knowledge of child’s friends and friends’ parents (P)

Parental involvement in school/community (P); communication between environments (Y)

N/A

Note: Respondents are as follows: Y=Youth; P=Parent, T=Teacher, S=School

aYouth was respondent for all questions.

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

For social survey data for youth development, the access issues are a little different. Several states have provided funding and support to field community-level surveys that focus on youth. Oregon, Vermont, and several other states have done this in the past with the YRBS. Colorado and Vermont have done the same with the PSL-AB. The state of Minnesota fields its own Minnesota Student Survey every three years, asking questions on a voluntary basis to all public school youth in grades 6, 9, and 12. In addition, many individual communities have contracted with the Search Institute to field the PSL-AB, using the results for comprehensive youth development planning.

Most communities, however, do not have access to the unique and important information that can only be gathered using such surveys as the ones reviewed above. The costs of fielding such surveys are themselves a barrier, although clearly many communities have made the decision that the information gained is worth the expense. In 2000 the PSL-AB, for example, could be fielded for between $1.65 and $2.00 per youth, with additional charges of several hundred dollars each for the production of reports (Search Institute, 2000).

For individual community programs for youth seeking to use social indicator data to monitor the activities and the success of their own program, the items in these surveys may be too general. For many purposes, such programs need to monitor elements that are specific to their program, looking at issues of process and implementation as well as outcomes. For them, a more tailored approach is needed, one that usually requires professional technical assistance.

ASSESSING PROGRAM IMPLEMENTATION AND OPERATION

When a program has just been launched, program managers, staff, participating youth and their parents, funders, and other stakeholders need information on whether it has been implemented according to design and, if not, how services and operations differ from those envisioned in the program’s underlying model. When a program has moved past implementation into the routine operating stage, information to determine whether it has continued to operate according to design and at the desired level of quality and efficiency will be similarly useful. Also useful will be data on total program costs and average cost per youth served.

Until enough time has passed to allow collection of any but the most short-term outcome indicators, all that stakeholders have for assessing program progress and the possible need for change is information on

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

implementation, operation, and costs. Such information can be used to produce process evaluations and cost studies and to provide performance monitoring. Findings from these analyses can stimulate communication about program goals, progress, obstacles, and results among program managers, staff, participants, funders, and others.

The “developmental quality” of a youth program may be defined as the extent to which it provides a social setting and a set of activities that should facilitate positive youth development. Developmental quality is the key characteristic that a program can directly control via its internal activities. It must attend to the developmental quality of youth experiences, regardless of the specific activities (e.g., mentoring, peer tutoring, theater productions, team sports) it sponsors. Based on regular assessments of developmental quality, a program can engage in dynamic reflective practice to fine-tune its service delivery and management practices. If so indicated, it can take corrective steps to align activities with the model on which the program is premised. To produce useful process evaluations, performance monitoring, and self-assessment, youth development programs therefore need valid, reliable indicators of the developmental quality of the experiences they provide.

Indicators of developmental quality are important for a second reason. Because of their small size, informal nature, and limited resources, many youth development programs will never undergo a rigorous impact evaluation using control or comparison groups, nor will they obtain high-quality data on developmental or long-term outcomes of the youth they serve. For such programs, information about long-term outcomes over which they may exert influence but cannot directly or fully control (e.g., school completion, good character, civic involvement) does not provide a good standard for program accountability. Rather, indicators of the developmental quality of the program necessarily provide the key information for judging whether it is likely to have positive effects on youth development. If the program’s model is valid and data on the developmental quality of its activities indicate that it provides a setting and a set of activities that facilitate positive youth development, one may reasonably conclude that the program contributes to positive youth development.

That is, when good impact data are not going to be available, indicators of the developmental quality of a program’s activities provide a fair mechanism for holding it accountable for what it can best control. This is an important use of such indicators besides the short-term feedback they can provide about program operation. Hence, we focus discussion

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

on indicators of developmental quality because of their essential and unique importance for youth development programs.

Although many community programs for youth recognize the value of data on developmental quality, most lack the staffing, knowledge, and other capacity to measure it. Even if programs had wanted to do so, until recently they would have searched in vain for good measurement tools applicable to youth development programs. The development of such tools has been a major challenge to the field. As a more common understanding of how community programs for youth need to be structured to promote positive development has emerged, there have been notable advances in meeting this challenge.

The work of Public/Private Ventures (P/PV), in conjunction with other partners in the youth development community, exemplifies recent progress in developing indicators of developmental quality for any youth activity, regardless of its specific goals. Building on Gambone and Arbreton (1997), P/PV has developed a sophisticated set of forms and scales to assess seven dimensions of developmental quality, each with subcategories. They are adult-youth relations, peer support, quality of staff presentation, behavior management, opportunities for decision making and leadership, youth engagement in the activity, and the quality of the space or location. These seven dimensions encompass the eight factors that facilitate youth development identified in this report.

Observers use the instrument first to record information on an activity, focusing on observations of positive and negative behaviors for each subcategory. They then use the observations to evaluate each dimension of the activity’s quality. The instrument seeks both quantitative and qualitative observations and judgments. After using it to record three independent observations and assessments of the activity, observers prepare an overall assessment and recommendations for improvement. To foster uniform implementation of the evaluation instrument, observers receive a manual that provides detailed instructions on how to observe and assess activities and fill out the forms. The instrument does not produce a summary measure of an activity’s overall developmental quality. Rather, it yields a rich picture of the degree of success a program is achieving along multiple dimensions that contribute to developmental quality.

Independent observers have used the instrument to evaluate activities in the San Francisco Beacons Program and will use it to evaluate Extended Service Schools, supported by the DeWitt Wallace Reader’s Digest Fund. In the future, executive directors and other supervisors

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

could use it to evaluate their organization’s activities. If supervisors decide to share the evaluation forms with their activities’ leaders and tell them that the criteria on the forms are what they were going to be evaluated on, the forms could help leaders see that an activity can mean much more to youth than just an opportunity to do whatever its specific goal happens to be—making pottery, playing basketball, learning computer skills, etc. Efforts are under way to more fully automate data collection and processing via hand-held computers. Although still a work in progress, this instrument represents cutting-edge practice.

Even if a program scores high on factors that promote developmental quality, it needs to know whether each participating youth experiences meaningful involvement with its positive development environment. Thus, programs also need indicators of participation over time by individual youth to measure the “dosage” of developmental factors. Yet the best that most programs currently do is collect daily attendance information.

Youth, of course, may simultaneously attend more than one program, switch programs over time, or mix program participation with involvement in other activities (e.g., extracurricular school activities, music lessons, religious training) that may also contribute to positive development. High-quality dosage data therefore cannot be based entirely on individual agency records but require tracking individual youth across programs and other activities.

The Community Network for Youth Development (CNYD), a local intermediary organization based in San Francisco, has advanced the use of data for reflective practice through its Youth Development Outcomes Project. CNYD grounds its work with agencies in a coherent, research-based youth development framework developed in conjunction with the Community Action for Youth Project (CAYP). CNYD brings together funders and youth-serving agencies to build consensus and agency capacity around issues of assessment, accountability, and best practice to promote healthy youth development.

The outcomes project led to the creation of two surveys—one for junior high and high school youth, the other for elementary school-children—that provide concrete measures of young people’s program experiences. The questions seek to capture the extent to which participants experience the supports and opportunities that facilitate healthy youth development. The dimensions of developmental quality tapped in this survey overlap with those in the P/PV instrument but are not identical. The survey also gathers information on the duration and intensity of

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

a youth’s participation in the program. Findings that revealed program shortcomings helped engage funders and agencies in discussions of why programs were falling short in some areas and how resources might be more effectively deployed.

A complementary self-assessment form completed by agency staff examines organizational practices in areas viewed as critical to supporting a program’s developmental quality. These include creating safe, reliable, and accessible activities and spaces; providing a range of diverse, interesting skill-building activities; ensuring continuity and consistency of care; high, clear and fair standards; and several others.

Taken together, the participant and agency instruments provide comprehensive data to help organizations create focused strategies for improving the developmental quality of their programs, given their resource constraints. If used skillfully in a process of quality improvement, such data can raise the cost-effectiveness of an agency’s management and programming activities. The general capacity building process and the specific instruments show potential for replication, and the youth survey may be adapted for publication on the web.

The CNYD and P/PV approaches to assessing developmental quality share a common theoretical framework. They overlap in the constructs of developmental quality they examine and how they operationalize those constructs. But there is an important difference between the two. P/PV focuses on assessing the practices of specific activities (e.g., making pottery) in terms of their developmental quality, whereas CNYD aims at assessing how well the overall organization fosters positive youth development via specific activities as well as its general organizational environment. Both have value.

Evidence in Roth (2000) suggests that youth-serving organizations are likely to welcome assistance for building their capacity to assess the developmental quality of their programs and act on those assessments. Roth surveyed executive directors of youth-serving organizations about the goals and characteristics of their programs. Many of the program goals reported by directors match up well with the personal and social assets of positive youth development discussed in this report. She also found that many program characteristics reported by directors are closely related to our list of features of positive developmental settings.

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Qualitative Data for Community Programs for Youth

Thus far this chapter has focused on quantitative social indicators and measures of program implementation, operation, and impact. This focus reflects the fact that most resources for data collection and program evaluation have been channeled in quantitative directions. Like experimental evaluation methods, quantitative data are often viewed as more objective, easier to understand, and more highly valued by funders and policy makers than qualitative data.

Nonetheless, as has been suggested in the last few chapters, qualitative data can also play important roles in the design, implementation, and evaluation of community programs for youth and aid in the understanding of the process of positive youth development. Methods to generate qualitative data on programs and the social and cultural context in which they operate include direct observation; open-ended interviews with key informants, staff, and participants; focus groups; ethnographic studies; the use of diaries kept by informants; and detailed case studies.

Correctly understood, qualitative data neither are inferior to nor substitute for quantitative data (Lin, 2000). Rather, by trading breadth for depth, qualitative data can complement statistical evidence in several respects (Sherwood and Doolittle, 2000). Data from field research can play an important role in planning the specific way services are provided. For example, qualitative research on the determinants of successful mentoring relationships is fairly consistent about what practices make for effective mentoring (Sipe, 1996). Such findings can be translated into guidelines for coaching and supervising mentors on an ongoing basis. Identifying better mentoring styles would be very difficult with standard quantitative survey methods. Focus groups can be used to help design services that more fully respond to the realities of the intended clients’ lived experiences and motivations. Hence, such services are more likely to be used (see Furstenberg et al., 1992 as an example of this use of focus group data; see Branch et al., 1998 as an example in the youth program area).

Detailed interviews and observations of line staff and participants are essential to determine if actual program operations are, in fact, those intended by the program planners. Such information allows evaluators to decide whether the quality of program services was sufficient to produce a fair test of whether the services made a difference. It can offer insights about the reasonableness of the underlying model and about problems likely to arise during implementation and ongoing operation.

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

These insights would also provide lessons for replication. For example, Plain Talk, a community program aimed at improving adolescent knowledge of sexuality and use of methods to prevent pregnancy and sexually transmitted diseases, took a long time to implement. Field research showed that community lay trainers needed more time than initially envisioned to become sufficiently comfortable with sexuality issues before they were ready to engage other members of the community (Walker and Kotloff, 2000).

The P/PV instruments discussed earlier collect both qualitative and quantitative information for assessing the developmental quality of youth activities. The quantitative data provide a general view of which aspects of an activity are running satisfactorily and which may need improvement or intervention by senior management. But only the qualitative data can provide information on the specific behaviors of staff and youth and the details of daily activities that would allow the program managers to take concrete steps to improve operations.

Qualitative data may yield more complete, more nuanced, and context-specific understandings of the nature of correlations and causal relationships in the process of youth development that have been uncovered by quantitative data (e.g., by using some of the major datasets discussed in this chapter). Similarly, qualitative data can help explain subtle causal mechanisms through which a program works (or fails to work) —the “how,” whereas quantitative analysis typically shows only “how much” the program affected an outcome of interest. Such data can play a key role in formal, extensive impact studies if gathered early in the project and used to develop hypotheses that are investigated further through subsequent quantitative work. If time permits, additional qualitative research can then elucidate the findings from the quantitative hypothesis testing and suggest yet newer hypotheses—and so on through an iterative process that may yield much richer understanding than reliance on only one kind of data. Even community programs for youth that do not undergo rigorous impact evaluation can supplement small-scale quantitative assessment of program quality with data from focus groups, individual interviews, and careful observation of program operations. The latter can aid in interpreting quantitative findings as well as suggest new questions for future quantitative assessment.

Qualitative data can allow exploration of how program services are understood and experienced both by participating youth and by those who choose not to participate. Neighborhood youth may have some idea of what motivates them, what they found stimulating in a program,

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

what drew them to it, why they were prepared to give it time over a period of months or perhaps years, and whether they think that participating in it affected them in particular ways. Such data can also help program operators, funders, and policy makers understand what parents care about in programs their children may attend. They can clarify various stakeholders’ expectations for a program and what would constitute success in their eyes.

Program research based on qualitative data has important limitations as well. Samples are often small and not necessarily representative, even if drawn from program participants. Verifying conclusions from qualitative data is difficult. So is generalizability. Qualitative research is costly to do well. Telling stories, however interesting or compelling they may be, is not a substitute for rigorous analysis of qualitative data. Despite these and other limitations (Lin, 2000), qualitative data can play important roles in the design and analysis of community programs for youth.

National Intermediary Organizations

A number of national groups have become important in assisting community programs to enhance their capacity to collect and use data in their design, planning, and evaluation efforts. A number of these are focused specifically on youth development programs. These groups are playing an increasingly important role in the education of youth program staff around youth development concepts, in organizing available data-related materials for use by such groups, and in consulting with individual programs to develop data collection and analysis strategies to support planning and reflective practice. They are providing essential information and services to the youth program community.

  • The Youth National Outcome Work Groups, mentioned earlier, provides more than detailed information on measures relevant for youth development. They support work groups in the areas of child, youth, family, and community development. In addition to information on measures, they provide easy to understand primers for each domain of well-being covered by the work group and an extensive bibliography for those who are interested learning more about a particular domain.8

8  

For additional information, visit <http://ag.arizona.edu/fcr/fs/nowg/ythindexintro.html>

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
  • National Youth Development Information Center is a web-based information center set up by the National Collaboration for Youth to provide comprehensive practice-related information to youth— serving organizations throughout the country at low or no cost. The site includes a section on outcome measures and program evaluation, which lists major guides on youth outcome measures and evaluation approaches. There is also a section that provides youth-related statistics and collections of statistical data.9

  • The Youth Development Mobilization Initiative, Center for Youth Development and Policy Research, Academy for Educational Development has a goal to ensure long-term institutional support for youth development by creating a communication network between policy makers and practitioners at the local level. It is currently developing a project, Youth Development/Community Indicators: On the Plus Side, which seeks to maximize the development of community-level social indicators using existing administrative and related data resources to support positive youth development planning and policy. It intends to work with several localities (including their current partners in Albuquerque, New Mexico, Hampton, Massachusetts, and Milwaukee, Wisconsin), as well as a national advisory board of youth development experts in this effort, to develop best practices that can be followed by other communities.10

  • The Aspen Roundtable on Comprehensive Community Initiatives is comprised of 33, substantive experts, policy officials, and program heads who examine and discuss issues surrounding the strengthening of Comprehensive Community Initiatives. The roundtable’s web site features a catalogue of measurement instruments related to community research. This feature, called Measures for Community Research, is one of the first resources of its kind and will serve as a clearinghouse for the collection and distribution of instruments and other tools related to key community-level outcomes. This resource includes a separate section on youth-related measures.11

  • National Neighborhood Indicators Project is working with local institutions in a dozen cities to develop neighborhood-level indi

9  

For additional information, visit <http://www.nydic.org.>

10  

For additional information, visit <http://www.aed.org/us/cyd/ydmobilization.html>

11  

For additional information, visit <http://www.aspenroundtable.org>

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

cator data systems that can be used to support a variety of development activities throughout the community. Partners include groups in the cities of Atlanta, Baltimore, Boston, Chicago, Cleveland, Denver, Indianapolis, Miami, Milwaukee, Oakland, Philadelphia, Providence, and Washington, DC. In addition to facilitating a variety of peer support activities, the project is developing a National Neighborhood Data System, which will provide easily accessible data at the census tract or zip code level for major metropolitan areas, as well as a series of handbooks and other tools on the use of information in community capacity building.12

SUMMARY

Community programs for youth benefit from ready access to high-quality data that allows them to assess and monitor the well-being of youth in their community, the well-being of youth they directly serve, and the elements of their programs that are intended to benefit those youth. Programs may use social indicator data for needs assessments, service targeting, goals tracking, program accountability, and in support of reflective practice to improve program and policy effectiveness over time. They also benefit from information and training to help them use these data tools wisely and effectively. In this chapter we have reviewed surveys and other measurement instruments, the data generated from these tools, and related technical assistance resources that can be used by youth development programs; we were pleasantly surprised to find a relatively rich set of tools and information.

Every community already has a great deal of data collected through its social programs and educational systems, its vital statistics and health surveillance systems, and the decennial census. Some communities have been more active than others in developing these data resources to guide policy and program development for children and youth. A systematic review of the potential of these data sources to yield useful indicators could provide a guide for communities seeking to maximize the use of their own data resources at a reasonable cost.13

Even when exploited to their full potential, administrative, vital statistics, and related data sources can cover only limited geographic areas

12  

For additional information, visit <http://www.urban.org/nnip>

13  

The Center for Youth Development, through its proposed On the Plus Side initiative, intends to produce such a review.

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

and only some components of a youth development framework. Adding local survey data in diverse communities, as has been done in a number of states and individual communities, can help create a more complete picture.

A number of states and many individual communities have fielded youth surveys at the local level. Steps should be taken to expand opportunities so that more communities can benefit from such data. For example, the Centers for Disease Control and Prevention, which sponsors the Youth Risk Behavior Surveillance survey, could expand its use in additional metropolitan areas (it is currently fielded in 16 major metropolitan areas) and work with interested states to expand its use down to the school district level. Furthermore, in order to make it a more useful tool for positive youth development, the Centers for Disease Control and Prevention could work to develop optional modules that focus on the domains of development and social settings in which it is currently weak.

Research on many aspects of positive youth development is still in its early stages, with the result that many of the indicators used to represent youth development outcomes in existing surveys are not well tested. Areas in need of particular attention include life skills, social, emotional, and psychological development, as well as most of the domains of developmental settings. Most of the psychometric work that has been done on indicators used in the two most complete community surveys on youth development (the Profiles of Student Life: Attitudes and Behaviors and the Student Survey of Risk and Protective Factors, and Prevalence of Alcohol, Tobacco, and Other Drug Use) are based on cross-sectional data and, in the case of the PSL-AB, on nonrepresentative samples. Longitudinal surveys allow one to explore a crucial aspect of social indicators, namely their relationship to future development as one moves from adolescence into adulthood. The Values in Action initiative, led by psychologists Martin Seligman and Christopher Peterson, is attempting a rigorous classification of strengths and virtues that will include definitions, existing research and available measures, promotional and inhibiting factors, and other valuable information that should provide a useful information base to guide future research and measurement development in this area.

There is evidence that youth-serving organizations have an interest in building their capacity to assess the developmental quality of their programs. To produce useful process evaluations, performance monitoring, and self-assessment, however, community programs for youth

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

need valid, reliable indicators of the developmental quality of the experiences they provide. The evaluation research community, working with practitioners, needs to build on recent progress made in improving the quality and scope of these indicators. Research is needed to determine whether appropriate indicators vary depending on the characteristics of the specific youth population served by a program. As understanding of the determinants of positive youth development improves, the indicators should be periodically revisited and, if necessary, revised. To this end, methods for collecting information on individual youth participation in one or more community programs also require development and implementation. At a minimum, youth-serving organizations should track individuals’ attendance and their involvement in specific activities. Data on individuals’ participation across programs and in informal activities that affect developmental assets can be gathered and incorporated into basic and evaluation research.

Geographic information system databases can greatly enhance the utility of social indicator data for community programs for youth by allowing users to draw on multiple data sources to pinpoint areas of need at the neighborhood level. This is particularly important to community programs for youth, which often serve particular neighborhoods within the larger community. An increasing number of communities are putting their data into GIS systems, and some, like those participating in the National Neighborhood Indicators Project, are making them available to all community organizations. Funding to optimize systems in communities with functioning GIS systems will support the needs of local community programs for youth by increasing the relevant data available through the system and the ease with which it can be accessed.

Many community programs also lack staff knowledge and the funds to take full advantage of social indicators as tools to aid in planning, monitoring, assessing, and improving program activities. Individual programs and communities would benefit from opportunities to increase their capacity to collect and use social indicator data. Greater access to professional knowledge and advice on data and measurement issues, via the Internet and through individual consultation, will allow community programs for youth to be more effective and efficient as they design their own monitoring and evaluation strategies. This will require support from funders to develop the internal expertise and external consultants needed. National and local intermediaries, like the Center for Youth Development, and state organizations, like the Indiana Youth Institute, also pro-

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×

vide invaluable information, support, and professional advice to state and local youth programs. Efforts to move to Internet-based systems for documenting and disseminating successful assessment tools and protocols, administering assessment instruments, inputting the responses, and analyzing the data would also simplify, streamline, and lower the costs of collecting and inputting program data and also deserve support.

Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 228
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 229
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 230
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 231
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 232
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 233
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 234
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 235
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 236
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 237
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 238
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 239
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 240
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 241
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 242
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 243
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 244
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 245
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 246
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 247
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 248
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 249
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 250
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 251
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 252
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 253
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 254
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 255
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 256
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 257
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 258
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 259
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 260
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 261
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 262
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 263
Suggested Citation:"8 Data and Technical Assistance Resources." National Research Council and Institute of Medicine. 2002. Community Programs to Promote Youth Development. Washington, DC: The National Academies Press. doi: 10.17226/10022.
×
Page 264
Next: 9 Funding and Support for Programs »
Community Programs to Promote Youth Development Get This Book
×
Buy Paperback | $54.95 Buy Ebook | $43.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

After-school programs, scout groups, community service activities, religious youth groups, and other community-based activities have long been thought to play a key role in the lives of adolescents. But what do we know about the role of such programs for today's adolescents? How can we ensure that programs are designed to successfully meet young people's developmental needs and help them become healthy, happy, and productive adults?

Community Programs to Promote Youth Development explores these questions, focusing on essential elements of adolescent well-being and healthy development. It offers recommendations for policy, practice, and research to ensure that programs are well designed to meet young people's developmental needs.

The book also discusses the features of programs that can contribute to a successful transition from adolescence to adulthood. It examines what we know about the current landscape of youth development programs for America's youth, as well as how these programs are meeting their diverse needs.

Recognizing the importance of adolescence as a period of transition to adulthood, Community Programs to Promote Youth Development offers authoritative guidance to policy makers, practitioners, researchers, and other key stakeholders on the role of youth development programs to promote the healthy development and well-being of the nation's youth.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!