Page 7
Theme 2.
The Interface of Research and Practice in Education: Linking Quality and Utility
Does quality research lead to its effective use in education policy and practice, and if so, how? What role does high-quality research play in education improvement? These questions, implicitly embedded in the charge to the committee, provided the frame for the second session of the workshop. This session featured a roundtable discussion on the interface of research and practice in education, with a focus on the relationship between research quality and research use. Six invited discussants with perspectives on education research, education practice, and their connections, engaged in a dialogue that ranged from the appropriate role of evidence in education reform to how to broker better communications between researchers and practitioners.
EVIDENCE IN AN ERA OF ACCOUNTABILITY
Roundtable discussants agreed that research can be a powerful force for improving policy and practice in education. Several discussants linked research-based knowledge to the accountability structures of standards-based reform efforts, arguing that the emphasis on performance in the K-12 education system was fueling a rising demand for evidence on the effectiveness of various strategies for improving student achievement. One participant flatly stated that educators had never asked much of education research, and “that's exactly what we [the research community] gave them.” In this new era of accountability, he asserted, that dynamic is changing.
|
We will see things scale up, because we are going to be driven by performance... [in ways] we have never seen before. Educators have never asked much of education research and development, and that's exactly what we gave them...that's not true anymore. —Paul Hood |
THE ROLE OF RESEARCH IN REFORM: POWERFUL YET MISUNDERSTOOD
Discussants agreed that oversimplified expectations about the role of research in reform efforts undermined its potential impact. Specifically, several discussants rejected the common metaphor of “translating research into practice,” arguing that even the highest quality research cannot provide simple answers to the complex problems of teaching and learning. One discussant asserted that the power of research lies in its potential to foster a public dialogue of how to improve education. He elaborated, arguing that engaging the
Page 8
public in this way would promote the democratic ideal of an educated citizenry and significantly enhance the capacity of all actors in the system to improve education.
A number of speakers underscored this problem by describing instances in which partnerships between researchers and schools broke down when quick improvements in student outcomes were not achieved. Related, a participant asked if education research was ready to respond to new (federal) requirements that schools adopt “research-based” programs. Discussants agreed that generally the answer is no.
|
[The] assumption that quality research will lead to agreement... is not true in any science... we have...a romantic notion that researchers [can] tell teachers what to do in a given situation, when the situation is very complex. —Denis Phillips |
BRIDGING THE GULF BETWEEN EDUCATION RESEARCH AND PRACTICE: QUALITY, CULTURE, AND INCENTIVES
Do education researchers and the potential users of research view quality in the same way? Discussants basically agreed that while both communities value quality, the relative emphasis on various aspects of quality differs. Participants offered examples that illustrated the contrast: in simple terms, researchers cherish scientific controls and generalizations about effects; practitioners value adaptation and richly contextualized information about implementation. One discussant argued that striking the right balance was essential. Another participant directly related the concepts of research quality and research utility by asserting that the more likely it is that research results will be implemented in practice, the more incumbent it is for the researcher to adhere to standards of rigor in the research process.
|
Researchers are trained to do research, and educators are trained to educate children. The goals are different but there needs to be give and take... the quality issue still has to be there. —Sharon Lewis |
Agreeing that striking the right balance is a difficult task, a strong theme in this discussion related to the incentive systems and attendant cultures of researchers and educators. These differences were described as major impediments to forging the connections necessary to enable collaborations between the two.
Discussants pointed to problems in the way researchers are trained and the incentives inherent to university tenure and promotion. One discussant suggested that quality could mean the same thing to researchers and practitioners if researchers had practitioners' interests in mind. Others agreed; for example, one discussant who conducts evaluations for schools suggested that quality and utility are both aspects of the overall value of research and that good evaluators need to “ensure the scientific integrity of the research while attending to its applicability in practice.”
|
I [an evaluator] often have the experience... of being surprised by my [school-based] clients when they interpret back to me what it is I told them. And for the first time, I understand something I never understood before... this experience of working back and forth is at least humbling if not illuminating. —William Quinn |
Page 9
Another discussant pointed to the pressure and incentives faced by researchers to publish in peer-reviewed journals, contrasting this situation with many physicians' practical incentives to cure patients. Participants offered two examples of effective strategies for conducting inquiry in applied education settings that are not typically valued by universities: (1) brokering sustainable partnerships with schools, and (2) engaging in jointly authored work of interdisciplinary teams of researchers. When university-based researchers try to conduct this kind of work, one discussant argued, “the incentive to do innovative work goes down.” She also suggested the requirements for tenure and promotion at most universities pose dilemmas for junior faculty who find it difficult “to articulate the value of what they are doing.”
Despite these difficulties, participants suggested that the research effort can be greatly enriched by engaging in this interface. Reflecting on his career conducting evaluations with schools, one discussant told the group that he consistently learns something new from interacting with his clients who have rich contextual understandings of their situations.
Another major thread of this discussion focused on teacher professional development. Participants pointed to preservice teacher education, arguing that schools of education should train prospective teachers to understand research and evaluation, and to be savvy consumers. There was some disagreement if this meant adding a research methods course to the curriculum of education students. Drawing a parallel to medicine, one discussant dismissed that strategy, arguing that medical research is adopted in practice not because physicians understand or investigate its methods, but because the mechanism inherent in the research makes sense to them. He further suggested that physicians assume that the profession takes care of the proper use of methodology. Following on the medical example, another participant suggested that research will never be meaningfully connected to practice without the emergence of an intermediate field—like those in engineering and medicine—to fill the gap at the nexus.
|
We need to create a much better interface... between educational research...and practice... we're simply going to have to have people in that intermediate... You find them in medicine. You find them in engineering...no matter how high the quality of the research, if you don't have people in...the intermediate position...you simply can't make the powerful...connections that you would hope to have. —Paul Hood |
WANTED: OBJECTIVE SYNTHESES OF RESEARCH FINDINGS
Discussants agreed that objective, synthesizing mechanisms that can reconcile disparate findings and reach consensus on what is known in a particular area are critical for both researchers and practitioners. Participants suggested that this need was particularly acute for educators and policy makers who commonly face inconclusive—and sometimes contradictory—evidence when they seek guidance from research. Discussants agreed it is difficult to answer simple questions like “what works?” because the highly diverse character of education by its very nature generates uneven answers. One panelist identified the lack of a common resource for education professionals—like Medline for physicians and Lexis-Nexis for attorneys—as problematic.
Page 10
In his work advising school districts about the effectiveness of various comprehensive school reform models, for example, one discussant said for every evaluation he has seen, there is always one school where a model “worked”— even though on average it does not. He went on to say an objective “voice” was needed to help practitioners understand the conditions under which certain strategies seem to work, at least sometimes, and what strategies do not seem to work at all. The new Education Quality Institute was cited as an organization that could provide such a voice. The availability of evidence to support most claims and the lack of an authority to make summary judgments about a body of evidence was described as particularly problematic because it enables vendors to create “beautiful bar graphs that show their programs work.”
|
Where do [teachers] then go when they enter into the profession to find out what works? In law or medicine, they have avenues to look to: Medline, MEDLARS, Lexis-Nexis.... I would submit that ERIC [Educational Resources Information Center] does not do that and there is nothing right now that's comparable. —Christopher Cross |
Participants argued that the lack of a synthesis mechanism makes it difficult to encourage administrators and policy makers to use evidence as well. They agreed that policy decisions made by superintendents, state aides, and federal policy makers are driven by the power of anecdote. Participants suggested that the case for systematic evidence could be made stronger by harnessing the power of a story to illustrate broad conjectures.
One discussant suggested that synthesis work was an essential exercise for the research community as well. He argued that research-based knowledge progresses when peers are forced to confront one another about their beliefs and preferences to advance consensus.
