Implementing Randomized Field Trials in Education Report of a Workshop (2004) / Chapter Skim
Currently Skimming:
3 When are Randomized Field Trials Feasible?
3 When are Randomized Field Trials Feasible?
Pages 12-26
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 12... ...
Since there have been very few randomized field trials conducted in educational settings to date, however, the field as a whole is in the early stages of learning how to conduct them. In each of the three studies described in detail at the workshop, a researcher-practitioner team described how they implemented randomized field trials in educational settings.
|
From page 13... ...
She said that large urban school districts support randomized field trials when they are used to test program variations but oppose them when they involve excluding students from promising interventions in order to create control groups. However, committee member Jack Fletcher, in moderating the presentations and discussion of the three studies earlier in the day, argued that if interventions have not been subject to rigorous scrutiny, it is impossible to know whether the services are beneficial, have no effect, or may even be harmful.
|
From page 14... ...
Although the Tuskegee study was a natural history study and not a randomized field trial, the fact that investigators withheld treatment that was known to be effective from the sick men has influenced the public's response to many forms of research, including randomized field trials in school systems. As discussed further below (see Chapter 4)
|
From page 15... ...
Because education research often involves human participants, the need to meet ethical and legal standards (including the standards imposed by institutional review boards) applies not only to randomized field trials but also to other types of education research.
|
From page 16... ...
When required, the process of obtaining informed consent of participants in randomized field trials (and other research) in educational settings involves both technical and political challenges.3 As described in Box 3-1, the first case study pre 2This issue is a common methodological challenge associated with how to model outcomes in schools that are by their very nature "nested" (students within classrooms, classrooms within schools, and so on)
|
From page 17... ...
Fashola, the principal investigator of the study, explained that, because Johns Hopkins "has been in and out of the news" due to concerns about the protection of human research participants, dealing with the institutional review board was hard, taking away time that she had planned to use to implement and study the program. Next, during the fall of 2002, she obtained approval from the Baltimore City Public School System, which she described as another tedious process.
|
From page 18... ...
" She said she and her colleagues at the Johns Hopkins University Center for Research on the Education of Students Placed at Risk have advocated for randomized field trials as a way to provide sci entifically based evidence of program effectiveness. Fashola noted that it was timely to study an after-school program, because the Baltimore City Public School System master plan had established the goal of increasing academic achievement by implementing aca demically oriented after-school programs.
|
From page 19... ...
, McClairn said, these teachers didn't feel that they were in competition with the additional teachers hired with study funds to deliver the tutorials "because we're teammates." The study was conducted during the 2002-2003 school year in four schools. Due to the unexpectedly long process of obtaining required approvals from Johns Hopkins University and the Balti more City Public School System, the CFA teachers did not begin recruiting students to participate until after the school year began.
|
From page 20... ...
selected four tutorials for inclusion in the Power4Kids study. In making its selections, the team considered previous research on the effectiveness of the programs, including small ran domized field trials and quasi-experimental studies.
|
From page 21... ...
Durno described the challenges of recruiting over 40 schools to be randomly assigned to one of four different reading interven tions. She explained that, as executive director of the Allegheny Intermediate Unit, which provides resources, instruction, and edu cation services to the affected school districts, she had a "credible" relationship with the schools, so "it works out well if we do the re cruiting." Nevertheless, there have been challenges in implement ing this large-scale study.
|
From page 22... ...
. Previous randomized field trials have assessed the impact of different first-grade interventions designed to reduce these behaviors and learning problems (e.g., Kellam et al., 1998)
|
From page 23... ...
GROUNDING THE STUDY IN THE RELEVANT EDUCATIONAL CONTEXT Many workshop speakers agreed that a randomized field trial is most appropriate when it is responsive to the current political and economic context of schools. For example, Chinnia suggested that researchers take time to analyze the social and political structure of the school district, learn
|
From page 24... ...
The studies are designed to evaluate approaches to improved management and interaction in first-grade classrooms and to assess the effect of these approaches in reducing later drug abuse, crime, and academic failure. Some workshop speakers suggested that the No Child Left Behind Act may discourage schools from participating in randomized field trials, even though the law explicitly encourages "scientifically based" education research.
|
From page 25... ...
When funding is available, she said, it is also useful to replicate a randomized field trial of a promising intervention in several different areas, to test effectiveness in diverse settings. Gueron explained that although large-scale randomized field trials require considerable resources, they may be more cost-effective overall than studies using alternative research designs (e.g., quasi experiments)
|
From page 26... ...
When looked at from the broader perspective of policy impact per dollar spent, Gueron concluded randomized field trials may be less expensive than quasi-experimental research designs that have high "political and financial costs" when they "end in methodological disputes."
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.