Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 9
Learning and Instruction: A SERP Research Agenda 1 Introduction The Strategic Education Research Partnership (SERP) proposes a large-scale research and development program that is tightly coupled with educational practice. The SERP mission is to develop a well-organized and powerful knowledge base, derived from both research and the study of practice, that supports the efforts of teachers, school administrators, colleges of education, and policy officials to improve student learning. The proposed initiative is fully described in Strategic Education Research Partnership (National Research Council, 2003). At the heart of the SERP enterprise are networks through which focused, coordinated, and sustained programs of research and development are carried out collaboratively by practitioners and researchers. The proposed inaugural networks include one on learning and instruction and one on schools as organizations, with a network on education policy to follow. The broad priorities for the research and development networks are to be set by an advisory board, a group of distinguished practitioners, policy makers, and researchers who will together define the issues of greatest importance and with greatest potential payoff from focused R&D. Defining the specifics of the research and development program is the job of the SERP director and the researchers and practitioners who lead each network. To illustrate the kind of work that the SERP enterprise will undertake, a panel of practitioners and researchers was convened to design an illustrative agenda for a prototypical network on learning and instruction. The panel members were chosen in part because they themselves have worked at the intersection of research and practice. In the work presented here, we are to some extent simulat-
OCR for page 10
Learning and Instruction: A SERP Research Agenda ing the role of the future SERP advisory board and network leadership. We focus broadly on target questions and on the panel’s view of the nature of the work to be done. Ultimately the choice of topics and the finer detail of project definition will fall to those responsible for charting a course of action. The goal of the panel was narrowly defined by the task of agenda development. The broad purposes and organizational structure of the SERP enterprise, and the practical challenges of creating a successful organization, were questions addressed by the SERP committee. The panel went about its work assuming the existence of a SERP organization like that proposed in the committee’s report. The two reports, then, can be viewed as companion documents. The decision to focus on learning and instruction was a function of the state of the research base from which we could draw. The National Research Council has in recent years produced syntheses of the research literature on human learning (National Research Council, 2000) and on assessment of learning (National Research Council, 2001b), as well as subject-specific syntheses in reading and in mathematics (National Research Council, 1998, 2001c). These and other explorations of the knowledge base on learning and instruction (National Institute of Child Health and Human Development, 2000; RAND 2002a, 2002b) provide a rich foundation on which our effort could build. But our focus is not intended to suggest preeminence of the learning and instruction network. Indeed, it is in combination with the work of other networks—in particular, the proposed network on schools as learning organizations—that the R&D on learning and instruction will best be able to influence practice. The goal of the SERP initiative is to improve student learning. The panel reached a critical decision at the outset that gives structure to a learning and instruction research agenda that will further that goal: to focus on practice. How effectively students learn in school is in large part a function of the effectiveness of educational opportunities teachers provide to students, as well as the transactions between the teacher and the students that make those experiences productive. The problem before the panel, then, was to consider how research and development can support the teacher’s effectiveness in providing—and helping students make use of—powerful learning opportunities. This means that the point of departure in defining the research and
OCR for page 11
Learning and Instruction: A SERP Research Agenda development program is not the intriguing questions that are at the frontiers of disciplines relevant to education (such as cognitive science and developmental psychology) that might have important implications for practice. Rather, it is the set of questions for which a classroom teacher needs answers in the conduct of instructional practice. Although the two would intersect in what has been referred to as Pasteur’s quadrant (Stokes, 1997), the program of work will be shaped differently if practice is made central. FOCUS ON PRACTICE The potential role for research and development in supporting practice is perhaps seen more easily in health care, for which productive links between research and practice have been established over the past century. At the core of medical practice is the doctor’s decision making, which is grounded in a knowledge base about the human body, about disease, and about interventions that can promote health and cure disease. That knowledge is embedded in the training of medical professionals; in the tools, protocols, and interventions that are standard in medical practice; and in the infrastructure for communicating to medical practitioners changing standards of practice. The generation of knowledge and its effective incorporation into training, tools, and protocols are not all that is required. But there can be no question that they are a necessary condition of sound practice. A parallel core knowledge base for educational practice is that on learning, instruction, and subject matter. It is this knowledge base with which a learning and instruction network must concern itself. Broadly speaking, its task would entail research and development on the questions at the core of classroom practice: on the generation of knowledge of how students learn (both generically and in the contexts of particular content areas), the elaboration of that knowledge at the level of detail required for classroom practice, and the incorporation of that knowledge into tested curricula and assessments, educational tools, teaching protocols, and teacher education programs. Medicine and education differ in a great many respects. Teaching, for example, focuses far more on promoting growth
OCR for page 12
Learning and Instruction: A SERP Research Agenda in a diverse group of students simultaneously, while medicine focuses on curing the individual case.1 The analogy is illuminating, however, regarding the relationship between research and practice. Physicians regularly exercise a great deal of judgment and expertise in the conduct of a practice that brings new phenomena, or new presentations of familiar phenomena, daily. Yet we rely on a powerful link between research and practice to ensure that the knowledge stores from which the physician draws in the course of practice are well stocked and reasonably current. With the pace of change in medical research, ensuring currency is not a trivial problem (McGlynn et al., 2003; Avorn et al., 1982). But the need to grapple with that problem is considered central to maintaining professional standards. In any field, research informs but does not define professional practice. Teaching, like many other professional practices, is a highly complex, multidimensional enterprise. It draws on standards of practice, professional education, background knowledge, tradition, and the personal characteristics and intuitions of the teacher. Research can play a major role in shaping these influences—for example by influencing professional education and standards of practice. But it is more likely to play that role if it is focused on the problems of classroom teaching and learning. In medicine we would recognize as common professional practice the pursuit of answers to a core set of questions: How is the patient’s health currently, and how does this compare to developmental expectations? What should the practitioner do to promote health and prevent disease or medical problems given cur rent health and development? When a medical problem exists: What are the patient’s symptoms? What is the patient’s personal history? 1 Even this difference is a matter of degree and not kind, however. Recently there has been greater attention to “population health” issues that focus on characteristics of communities that produce or reduce medical risk (see National Research Council, 2001d), and the individual education plans required for special education placements focus on the learning needs and challenges of individual students.
OCR for page 13
Learning and Instruction: A SERP Research Agenda What tests can aid diagnosis? What diagnosis is warranted given the symptoms, history, and test results? What treatments are available? What is the best match between the patient, symptoms, and treatments? How is the patient responding to treatment? While experience, intuition, and traditions of practice all contribute to a physician’s approach to each question, research-based knowledge is at the core of professional practice. Indeed, the professionalization of medicine in the first half of the 20th century can be viewed as a transformation that placed research-based knowledge at the center of practice. Most certainly, knowledge flows in both directions; observations from medical practice generate many questions and hypotheses that fuel research. New discoveries quite often emerge as unanticipated by-products of medical treatments. But research is required to answer the questions and test the hypotheses and insights generated by practice. Using the example of medicine, we can view a teacher’s practice as organized around a set of core questions about human learning and instruction. These questions focus on the normal course of development and learning, as well as on diagnosing and responding to student problems in mastering new concepts and acquiring new knowledge and skills. And as in medicine, the questions provide a schema for approaching practice, as well as a set of dimensions on which knowledge must be supported through preparatory education for professional practice. The questions can be asked for any subject or topic that is taught: What should students know or be able to do? What common understandings and preconceptions do students bring to a topic? What is the expected progression of understanding, and what are the predictable points of difficulty or hurdles that must be overcome? What instructional interventions (curricula, instructional activities, etc.) can support the desired student learning?
OCR for page 14
Learning and Instruction: A SERP Research Agenda What general and discipline-specific norms and practices best comprise and support student learning? And finally, how can students’ understanding and progress be monitored and instruction redirected in a responsive fashion? Because of the disconnect between research and practice, teachers grapple with these questions for the most part without reference to a research base. The “what” and “how” might be answered by adherence to a textbook that reflects little of the research knowledge on student learning. And individual student differences in understanding are often overlooked entirely because textbooks rarely provide the tools for formative assessment and guidance on instructional responses to student difficulties. But as accumulating evidence points to the complexity of the underlying learning processes, the need for research-based knowledge to support professional practice becomes ever more apparent. The decision to focus the agenda on practice has three closely linked entailments: (1) that the program of work be highly interdisciplinary, drawing on the range of knowledge bases and competences required to improve practice; (2) that the program emphasize and tightly integrate research and development and be carried through all stages necessary for classroom relevance; and (3) that the interdependence of student learning, teacher knowledge, and the organizational environment that characterizes practice be reflected in the SERP program of work. THE RESEARCH BASE To answer the questions that define practice will require an effort to bring together types of research and development that are now done separately, often by researchers who work in isolation from each other, and from program developers. If the questions of practice raised above are presented schematically, they might appear as in Table 1.1, where the questions are mapped for both student and teacher learning. The elements in the schema for educational practice are informed by several very different knowledge bases (see National Research Council, 2000, 2001c).
OCR for page 15
Learning and Instruction: A SERP Research Agenda TABLE 1.1 Schematic Questions for Teaching and Learning Student Learner Teacher as Learner Destination What should s/he know and/or be able to do (regarding the discipline or topic)? What should s/he know about the discipline or topic? What should s/he know about student learning of, and the teaching of, the discipline or topic? Point of departure What are the typical preconceptions and informal understandings that students bring to the topic? What are the teacher’s existing understandings about the topic and about student learning? Route What is the expected progression of understanding, and what are the predictable points of difficulty or hurdles? What are the typical pre-service and in-service learning trajectories and what difficulties are likely to be encountered? Vehicle What curriculum/pedagogy and classroom norms and practices facilitate learning? What factors/experiences facilitate learning? Checkpoints/course corrections How can individual student progress be monitored instructional activities matched to current understanding? How can progress be monitored and and instructional activities matched to current understanding? What students should know or be able to do in an area is informed by disciplinary expertise. It requires an understanding of the core concepts around which the disciplinary knowledge is organized, characteristic methods of reasoning and problem solving, and language and patterns of discourse. What to teach becomes a matter not only of the information and skills considered desirable for students to possess, but also of helping the student to build the conceptual framework that transforms information into understanding.
OCR for page 16
Learning and Instruction: A SERP Research Agenda Knowledge of students’ common understandings and preconceptions of a topic and the expected progression of student thinking requires careful research on the typical trajectory of understanding. In part this research attempts to identify the nature and limits of children’s changing cognitive abilities with age and instruction. And in part it attempts to uncover common understandings that can either support learning (the ability to halve or double relatively easily in mathematics) or undermine it (the common belief that heat and temperature are the same thing). Research findings suggest that students’ everyday understandings are resilient, even after specific instruction to the contrary. That resilience highlights the importance of a carefully designed research program to inform and support teaching to achieve conceptual change. Research of this sort is generally done by cognitive scientists and education researchers, although the knowledge may emerge from the experience of expert teachers and the observation of exemplary practice. The instructional interventions to move students along a learning path constitute the core of what is generally considered education. These interventions may be designed by curriculum developers, teachers, or researchers. But regardless of the source, the contributors to skill development, knowledge acquisition, and conceptual change are themselves a research agenda, and the effectiveness of the instructional approach a matter for empirical testing. General and discipline-specific norms and practices to support student learning define the rules for interaction in the classroom. Learning takes place in classrooms that are themselves communities. Every community is distinguished by norms for work and interactions, ranging from when and how people collaborate to how they speak with one another. Some of those norms are general, rooted in an understanding of schools in a democratic society; others are discipline-specific, for what it means to do mathematics differs from what it means to do literary analysis, chemistry, or history. In all cases, the relationships between par-
OCR for page 17
Learning and Instruction: A SERP Research Agenda ticular norms and the learning outcomes of interest are a matter for empirical investigation. Assessing a student’s progress is the task of research and development on methods and systems of assessment. This knowledge base can quite naturally be developed and tested in the context of curriculum R&D, but it may also draw on more fundamental research—such as research on the nature and measurement of “comprehension.” A key element of this work is linking assessment results to instructional responses (see Appendix A). The parallel knowledge base for teacher learning would be similar in its disciplinary foundations and theoretical underpinnings. But the learning context, the learning goals, and the subject-matter content (particularly regarding pedagogical content knowledge) are different enough that the research base on teacher learning is quite distinct from (if overlapping with) that on student learning. These various knowledge bases are not adequately developed to support teaching practice. Nor is the infrastructure in place to bring together people with the variety of competences required and to link their efforts in a program of work. SERP proposes to create that infrastructure, and the agenda we propose must build those linkages. IMPROVING RESEARCH AND DEVELOPMENT While much remains to be done to shore up the knowledge base, it is also the case that existing relevant knowledge is rarely incorporated extensively into classroom teaching. That this is so is disappointing, but hardly surprising when one considers the complexity of the task with which teachers are confronted. They face a class of students with different needs, behaviors, and preparation for learning. They must make choices about how to provide appropriate and powerful learning experiences for their students. They must simultaneously attend to their students’ understanding while planning next steps. They must manage and monitor the students’ learning and their learning environ
OCR for page 18
Learning and Instruction: A SERP Research Agenda ment. A program that is coherent—that guides instruction from one class to the next—can be of tremendous help in managing complexity. While the findings of researchers may be relevant to practice, if they are not easily incorporated into the teacher’s instructional program, they may not be useful. Indeed, they may simply add to the complexity of an already highly complex task. More targeted and instruction-relevant research (Hiebert et al., 2002) would be a good place to start. We can take as an example the research on the misconceptions students harbor in physics. In the course of everyday experience, people develop understandings or models of how the physical world works: as one moves closer to a heat source, temperature rises. One then draws inferences based on one’s experiences that are very often scientifically incorrect: in the summer, the earth is closer to the sun. A persuasive body of evidence suggests that the models of physical principles that students deduce incorrectly from everyday experience are powerful and resilient (National Research Council, 2000; Vosniadou and Brewer, 1989; diSessa, 1982). While students may “learn” physics in the classroom and even perform quite well on tests, outside the classroom they revert to their untrained model (see Box 1.1). The principle at work in physics is at play in all disciplines, undermining the effectiveness of the educational process. Nu BOX 1.1 Misconceptions About Momentum Andrea diSessa (1982) conducted a study in which he compared the performance of college physics students at a top technological university with the performance of elementary school children on a task involving momentum. He instructed both sets of students to play a computerized game that required them to apply a force (using a job stick) to a simulated object moving across the computer screen (a dynaturtle) so that it would hit a target, and do so with minimum speed at impact. Participants were introduced to the game and given a hands-on trial that allowed them to apply a few taps with a wooden mallet to a ball on a table before they began. DiSessa found that both groups of students failed miserably at the task. Given the momentum of the dynaturtle, students should have applied a light force very early to ensure minimum speed at impact. Despite their training, college physics majors, just like the elementary school children, applied the force when the object was just below the target, failing to take momentum into account. Further investigation with one college student revealed that she knew the relevant physical properties and formulas and would have performed well on a written exam. Yet in the context of the game, she fell back on her untrained conceptions of how the physical world works.
OCR for page 19
Learning and Instruction: A SERP Research Agenda merous examples have been identified in history and mathematics, as well as in science (National Research Council, forthcoming). Such “relevant” knowledge regarding the tenacity of everyday understandings becomes usable for a teacher only when it is applied to the subject and topic that are being taught, and when it is incorporated into instructional activities that draw out and effectively work with students’ preconceptions. These activities must have potential for being incorporated into existing instructional practice, or be an acceptable replacement for that practice, if they are to have an effect. Only the most extraordinary teachers will be able to undertake such a task on their own. Furthermore, only the most inefficient profession would require the design of individual solutions to a general problem. To effectively “bridge” research and practice, then, a research and development program must generate and draw on existing robust, relevant knowledge from a variety of disciplines, elaborate that knowledge so that it is usable in instructional practice, and then incorporate it into carefully tested tools and programs, directed both at student learning and at teacher learning. The research and development must be closely intertwined, so that program features are designed in response to research knowledge (derived either from disciplinary research or from the study of educational practice), and so that knowledge is continuously revised in the iterative cycles of design, study, and redesign. This necessarily means that the research and practice relationship is neither linear nor unidirectional. Instead, researchers and practitioners must interact in meaningful, progressively more sophisticated ways. Research is not neatly packaged and sent out to teachers to be implemented. Instead, researchers and teachers are mutually engaged in research and development in the context of practice. Critical to the notion of follow-through is that when research findings are compelling, sustained attention is required to ensure independent replication of research and evaluation results in the range of environments of intended use for an educational intervention. Because education is a complex enterprise in which any outcome is influenced by a variety of factors, the conditions that support success in one setting may not be understood until it is attempted in other settings in which conditions differ. Moreover, evaluations are often conducted only by the designers of the intervention or their “critical friends.”
OCR for page 20
Learning and Instruction: A SERP Research Agenda While many research findings are criticized for this reason, research attention in the field of education is rarely directed at independent replication. Too often promising outcomes mark the end point of a research endeavor, rather than the beginning of a research effort aimed at replication and scientific generalization. Finally, to be broadly useful to practice, attention must be sustained through careful study of the scaling up of successful interventions. When researchers design and study educational interventions, the process of study itself entails involvement by the researchers. The tacit and explicit knowledge of the researcher may support the teacher’s effective use of the intervention. Similarly, an expert teacher’s tacit knowledge and skill may be crucial to an intervention’s success. But when the original researcher and teachers are no longer present, the innovation is often much harder to implement and sustain. Thus, the requirements for effective, sustainable use of an innovation must themselves be the subject of study. INTERDEPENDENCE: STUDENT LEARNING, TEACHER KNOWLEDGE, AND ORGANIZATIONAL ENVIRONMENT Whether in teaching or in medicine, practice is not embodied solely in the tools and protocols of the trade. Rather, these work in tandem with both the practitioner’s knowledge and skill and the organizational environment. One shudders to imagine a magnetic resonance imaging (MRI) machine provided to doctors in a hospital with no more than an instruction manual on how to use it. Patients trust that doctors will be thoroughly versed in an understanding of the diagnostic power of the tool, the meaning that can be drawn from its images, and the confidence one can have in the information it provides under various circumstances. One also trusts that the organizational structure in which the physician operates will both appropriately support and impose standards on its use. Similarly, in education, professional practice relies on all three simultaneously and interdependently: the tools and protocols designed for student learning, practitioner knowledge,
OCR for page 21
Learning and Instruction: A SERP Research Agenda and organizational structures. They serve as the three legs of the stool supporting student learning. While each is, in a sense, independent of the other two, the effectiveness of any one in supporting student achievement depends on the strength of the other two. It is a lesson learned repeatedly in education reform efforts: focusing on the strength of one without attending to the other two is a strategy that holds little promise for success. The SERP focus on practice, then, requires coordination not only of the knowledge from different fields on effective instructional programs to promote student learning, but also research on the knowledge requirements for teachers to carry out instruction effectively, and requirements of the environment to support effective instruction. Teacher knowledge and skill matters a great deal in student learning. Ferguson (1991) analyzed data from 900 Texas school districts and found that teacher licensing exam scores, master’s degrees, and experience accounted for over 40 percent of the variance in students’ reading and math achievement scores after controlling for socioeconomic status. Other studies suggest a similarly powerful effect (Ferguson and Ladd, 1996; Strauss and Sawyer, 1986). Yet despite its importance, the research base on teacher learning is relatively undeveloped. SERP proposes to strengthen that knowledge base and, importantly, to tightly link the R&D on teacher learning and knowledge to that on student learning. As we have already noted, the schematic questions that frame student learning also apply to teacher learning (see Table 1.1). For teachers, however, the questions that shape their practice provide a good start on our first question: “What should teachers know and be able to do?” The answer implied by the questions of practice is that teachers should understand the learning process of the student well enough to assess and guide it; understand the content to be taught well enough to select and use appropriate instructional materials; guide the pace and direction of instruction and flexibly respond to student questions and thoughts; understand the curriculum materials well enough to use them flexibly as a means to an end rather than as the end itself; understand the effects of classroom norms and practices well enough to create a supportive learning environment; and understand assessments well enough to interpret the outcomes and respond appropriately. What is not well defined are the forms of knowledge a
OCR for page 22
Learning and Instruction: A SERP Research Agenda teacher must master in order to reach that end and the levels of mastery needed. What mathematics must a teacher know, and what pedagogical knowledge does she or he need to make and implement appropriate decisions about the next best instructional steps to develop student thinking about gravity, for example? Although these questions are central to effective practice, little research has been done to provide answers. Moreover, learning is as complex an undertaking when the teacher is the target as it is when the student is the target. A teacher’s existing conceptions of teaching and learning, of student thinking, and of the subject matter must be understood and engaged. And experiences that bring about conceptual change for the teacher must be designed and effectively deployed for learning to occur. The task is challenging; conceptual change is difficult to achieve when everyday experiences reinforce a misconception. In many everyday experiences one can simply tell people what they want or need to know. This is likely to influence a teacher’s view about learning and instruction. When the frame of reference is common, simply telling works just fine. But when conceptions differ, telling is unlikely to be enough. “Elephants are bigger than pigs” may be completely adequate to communicate intended meaning, while “the orientation of the earth’s axis relative to the sun determines the seasons” may be entirely inadequate. If the conception of teaching as telling is to be replaced by a more variegated model, powerful experiences that facilitate conceptual change on the part of the teacher about how students learn will be required. The typical learning trajectory for teachers, and how it changes with learning opportunities, also requires empirical investigation. Much that teachers need to know cannot be learned apart from practice. This raises several questions for inquiry: Under what conditions can teachers best learn while engaged in practice? What knowledge and skill must teachers acquire at the beginning of their careers? What knowledge and skill is best acquired once they enter the profession? What organizational, material, and human resources are necessary to support and sustain teacher learning over time? Widespread adoption and use of improved instructional methods are often hampered by institutional barriers that prevent or frustrate efforts to change: these may include problems of organizational structure, incentive structures, organizational
OCR for page 23
Learning and Instruction: A SERP Research Agenda culture, career patterns of teachers and administrators, and financial constraints. Even when presented with demonstrably effective instructional reforms, school systems are frequently incapable of moving the organizational machinery to achieve systemwide adoption (Briars and Resnick, 2000). And even when change is effectively instituted with the help of program developers, it is frequently not sustained when the developer departs. Schools are certainly not unique in resisting change. Change is effortful. It imposes uncertainty and requires risk of failure. Resistance to change would be expected to weaken if uncertainty is reduced by providing effective supports for success, if risk is minimized by careful evaluation of candidate changes and the circumstances under which they are successful, and if effort and risk taking are rewarded. However change is not, in and of itself, a desirable end. There are certainly changes for the worse, and one hopes schools have mechanisms in place to resist such changes. What is desirable is an organization that can systematically assess its own performance, evaluate the potential of alternative approaches to improve performance, monitor the effect of change, and alter course to improve outcomes—an organization that can learn. In the field of business management, attention has been devoted in the past few decades to the features of learning organizations. The focus of concern is corporations, and learning refers to the ability to incorporate new knowledge and technology required for effective competition and changing products to align with, or create, market demand. Business schools have drawn from a variety of disciplines: economics, statistics, political science, behavioral psychology, organizational psychology, and others. The potential contribution of these disciplines to the organization of medicine, agriculture, and the military has also been considered. However, their attention has not as yet been turned in a sustained way to the organization of schools. If advances in instructional programs and teacher knowledge are to have a sustained impact on student learning, the organizational structure of schools must support that change. Because the success of each component (instructional program, teacher knowledge, and organizational structure) in contributing to improving student learning depends on the success of the others, all three must be integrated in a SERP research program.
OCR for page 24
Learning and Instruction: A SERP Research Agenda The work of our panel, however, was to focus on learning and instruction. The committee that proposed a design for the SERP organization considers organizational issues to be of critical importance, as do we, and argues for a network on schools as organizations as a companion to the learning and instruction network from the start. To avoid mounting a research program that could otherwise be narrow and insular, it is essential that these two networks be closely tied as intended. FRAMEWORK FOR A RESEARCH AGENDA The committee considered two common strategies for organizing the research agenda: one focuses on specific subject areas taught in schools (science, history, etc.), and the other highlights research questions that cross subject domains (integrated assessment, teacher education, etc.). Much of the work on cross-domain issues is relevant regardless of the subject. This would include, for example, research on the relative effectiveness of professional development tools (like videotaped demonstrations, small group lesson study, etc.). The proposed agenda embraces both, embedding the cross-cutting issues in subject-matter research. The rationale for this choice stems from the overarching commitment to focus on practice. We have argued that research is often not used in practice because it is not elaborated at the level of classroom practice, and classroom practice is subject specific. Furthermore, many of the cross-cutting issues are best illuminated with subject-specific examples. While the productive role that high-quality assessments can play in supporting effective instructional practice crosses topics, a deep understanding of the issues can be seen clearly in looking at a specific case, like the Force Concept Inventory described in Chapter 4. The organization described in the Strategic Education Research Partnership is one in which regular stock-taking and coordination across research domains is given a high priority. In the panel’s view, this coordination will be central to maximizing the potential of the program of research and development. Throughout the chapters that follow, the parallels across subjects are striking, and much can be learned if those engaged in different
OCR for page 25
Learning and Instruction: A SERP Research Agenda subject areas inform each others’ work. Indeed, a good deal of the value added by a SERP organization is the opportunity created for the accumulation of knowledge, and the coordination of research protocols and data collection efforts across subjects that will make that accumulation possible. Among the many cross-cutting issues, assessment deserves specific mention at the outset. The development of effective measures of the outcomes of learning and instruction is critical to producing high-quality evidence that can support the work of practitioners, policy makers, and researchers. Without an initial investment in developing reliable measures, even good-quality research leaves critical issues unresolved. Different assessments may show different instructional approaches to be beneficial, and often none of the assessment instruments is designed to fully capture the range of competences that are the desired outcome of instruction. Appendix A describes the nature of the work that in the committee’s view needs to be done in the area of assessment research, development, and testing. In the agenda described below, then, both cross-cutting and subject-specific topics are integrated into a program of research designed to support educational practice in the specific subjects targeted. The schematic questions posed in Table 1.1 provide a starting point for evaluating what is known currently that can support effective practice and where the research and development foundation is weak. We portray learning as a journey, using a travel metaphor to illuminate certain features of the schematic questions. Planning the trip requires first that one is clear about the destination, although it may change for a variety of reasons. There are multiple routes to get from a departure point to a destination, but routes will differ in the opportunities they afford for interesting experiences along the way, as well as in their efficiency at reaching the endpoint. Yet to reach the destination, the directional options are constrained, and there are likely to be critical points (passes) through which the path must lead. As with the route, there is no single vehicle required for a journey, but some work far better than others over a particular terrain. Finally, the challenge of monitoring progress is very different for different journeys. When a route is well marked, it can be a simple effort to clock the miles; but when the area is poorly mapped, frequent assessment and course correction are critical. For any subject matter taught, one can assess the quality of the research base by the knowledge it provides to
OCR for page 26
Learning and Instruction: A SERP Research Agenda support effective decision making regarding each of these issues. In the long run, providing research-based knowledge to support answers to the schematic questions for every subject taught in schools is a desirable end, just as one expects the treatment by a physician of any ailment to be based on research-based knowledge. Yet the reality of the limited resources devoted to education research, as well as the existing capacity to conduct that research, suggest the need for focus on a limited set of subjects in order to ensure that work can be carried through all stages necessary for usability. As a knowledge base is consolidated in some areas, attention can be devoted to new areas. CRITERIA FOR CHOOSING TOPICS The areas in which research and development could be useful to educational practice are innumerable. To narrow the focus, the panel asked two questions: Are there examples of rigorous research and development efforts that already show impressive gains in student achievement in research trials? These would provide the starting point for R&D that is intended as one of the hallmarks of SERP: following promising program outcomes with research on the circumstances under which the results are obtained, the feasibility of the intervention in the classroom and the school context, the teacher knowledge and support required for success, and the organizational factors that influence outcomes. We refer to these as “downstream” cases because the work in these areas has already traveled some distance toward classroom usability. Are there pervasive problems of practice that are widely recognized as critical, but for which the knowledge base is too weak to guide instructional interventions? We refer to these as the “upstream” cases, since the work is still in early stages. Since the upstream research will need to go through all phases of research and development required for usability, the impact on practice is likely to require more time. While question 1 is driven by the opportunities provided by
OCR for page 27
Learning and Instruction: A SERP Research Agenda the current knowledge base, question 2 is driven by the needs of practitioners. Precisely because the knowledge base is weak for the latter, R&D on these questions would draw more heavily on the study of practice and the extension of research from the social and behavioral sciences to strengthen the education knowledge base. As we worked through agenda development, the upstream-downstream distinction became somewhat blurred. The downstream examples often involve unanswered questions about teacher learning or taking practice to scale that are likely to require a sustained research effort. And in areas in which the knowledge base is not well consolidated, there are individual practices or curricula that appear promising. Nonetheless, the distinction continued to be useful as an indicator of the potential impact on practice in the short run. RESEARCH DOMAINS The panel recommends three areas for focus: reading, mathematics, and science. These three domains emerged when we looked for downstream areas in which current research and development shows promising results that could substantially influence practice. In both mathematics and reading, the strongest research bases address learning in the early elementary years. In science, in contrast, physics was identified as the area that is furthest downstream—a subject generally taught in high school. All three domains are recommended for upstream research as well, but for reasons that are not uniform. Practitioners on the panel emphasized the central role standardized tests play in the lives of students, particularly as they reach the upper grades. There was considerable disagreement regarding the desirability of these tests and a common concern regarding the ability of tests to distort and undermine good instructional practice. But the tests play a role in setting the emotional stage for students, touching their sense of identity and self-confidence. Since performance in mathematics and in reading comprehension is a major contributor to test results, these are seen as important areas for improving both student outcomes and the experience of schooling itself. The choice of these areas was further supported by the need for proficiency in order to successfully meet the demands of modern life.
OCR for page 28
Learning and Instruction: A SERP Research Agenda Reading comprehension also features prominently in the teaching of most subject areas, including English, history and social studies, mathematics, and science. Teachers of these subjects must decide to what extent reading comprehension instruction is required as part of their effort to teach the subject matter. There is currently little guidance in this regard. Research on reading comprehension therefore has the potential to provide benefits in virtually all subject areas. Furthermore, reading comprehension is poorly measured. Current tests emphasize the speed of reading and short-term recall of factual information. But existing research suggests that both speed and short-term recall are weak predictors of the construction of understanding from text that comprehension requires. Given the importance of reading comprehension in standardized tests, making progress on test measurement issues would have substantial potential to influence practice. In contrast to mathematics and reading comprehension, science was identified for upstream research because the development of science curricula, particularly for the elementary and middle school years, has been remarkably weak. The American Association for the Advancement of Science recently reviewed widely used textbooks in middle school science. “The study probed beyond the usual superficial alignment by topic heading and examined each text’s quality of instruction aimed specifically at the key ideas, using criteria drawn from the best available research about how students learn” (Roseman et al., 1999). Not one of the middle school science texts evaluated by the project was rated as satisfactory. High school biology texts scored slightly higher than the middle school texts, but the evaluation found serious shortcomings in both their content coverage and instructional design (Budiansky, 2001). Textbooks across the grades were characterized as “overstuffed and undernourished,” with presentation of a great many facts and too few opportunities to present the concepts that make those facts meaningful (Budiansky, 2001). In contrast to mathematics, there is little agreement in science as to the sequence and content of study, or even when science education should begin. The areas the committee has chosen for focus are strategic: they provide either the opportunity to leverage existing investments in research by carrying promising findings through to practice, or they hold promise for providing new knowledge in areas of critical need. We emphasize, however, that a SERP
OCR for page 29
Learning and Instruction: A SERP Research Agenda network would probably undertake a more thorough investigation of candidate opportunities, and it is likely to be shaped in significant measure by the opportunities that emerge for productive work in schools as SERP is launched. In particular, the link between cognition and context is likely to be better understood and incorporated into the developing agenda as the SERP work progresses. ORGANIZATION OF THE REPORT In describing our proposed agenda for research and development, we organize the work by domain. Chapters 2, 3, and 4 address reading, mathematics, and science in sequence. Each discipline is treated at some length because our purpose is to consider the program of research and development that would be required to strengthen the knowledge base on the entire set of questions that define teaching practice, as well as an understanding of the knowledge requirement for teachers of the subject. Because we focus so broadly, however, we do not go into depth in any single area. Readers looking for more depth are referred to other reviews intended for that purpose. Structuring the program around the questions of practice as we propose brings coherence to the agenda, but it may tax the reader who finds the same issues of assessment or teacher knowledge treated similarly in three different chapters. We consider the repetition in the agenda important, however, because the state of the art in education research and development is one in which many of the most critical principles of instruction are understood at a general level, but the work of R&D to incorporate those principles into the teaching of individual subjects has not been done. That work must be repeated for each subject if learning and instruction are to be improved. At a time when the quality of education research is a matter of heated debate not only among researchers, but also in the halls of Congress, careful consideration of methodological rigor and quality control are critical elements of a research agenda. The final chapter discusses how the proposed agenda addresses contemporary concerns about education research quality and impact. We also consider the extent to which proposed SERP features are required to effectively carry out that agenda.
Representative terms from entire chapter: