|
Items in cart [0] |
Questions? Call 888-624-8373 |
PAPERBACK + PDF PAPERBACK PDF BOOK PDF CHAPTERS Free Resources |
||||||||||||||||
|
||||||||||||||||
|
|
|||||||||||||||
| ||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 79
A
SERP Networks:
10
J
hat an
r l
IT
J Ill ~ U 1l l
flea loo?
n Chapter 3 we sketched a picture of each of the SERP
organizational Darts. In this chanter we turn to anima-
. ~ ~ ~ ~~ ~ ~ ~ ~ ~ ~
tion of the 5~;l~l' core: the research and development
networks. What incentives would bring researchers,
practitioners, and school systems to collaborate in the networks?
And once they joined the efforts, what type of work would they
do? Finally, we consider the incentives for those engaged in the
delivery of education to make use of what SERP would offer.
· ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
CREATING NETWORK PARTNERSHIPS
She nature of the collaboration contemplated for SERP is
different in qualit~v and scope from any currently in place in
education. She committee has found the teaching hospital a
useful metaphor for the envisioned collaborative relationships:
the functions of practice and research would in good part be
located in the same site (SERP field sites), professional prepara-
tion of both practitioners and researchers would be merged
with high-level research activity, and links with university re-
search departments would be strong.
· ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
WHO WOULD COME?
Where are many examples of existing partnerships between
researchers and practitioners, including the cases described in
Chapter 2. When scaled against the size of the K-12 education
S E R P N E T W O R K S
79
OCR for page 80
system, however, the number is dwarfed by the need. Even
more importantly, these partnerships generally require
Herculean efforts on the part of individuals and are very diffi-
cult to sustain when those individuals move on. How can SERP
create a set of incentives for participation by both researchers
and practitioners that makes collaboration more commonplace
and that facilitates the maintenance of those collaborations in a
sustainable fashion?
B R ~ N G ~ N G R E S E A R C H E R S T O S E R P
To create an organization with the capacity to attract high-
quaTit~v researchers, we need to know what motivates the deci-
sions of researchers' regarding the work they pursue. The
committee's hypothesis is that researchers decisions can be
roughly characterized as balancing five considerations, although
the weight given to each varies tremendously across individu-
als. The five are reputation, career opportunity, intellectual
stimulation, income, and the ability to make civic contributions
through their work to the public good. While an individual
researcher may value one of these highly and another very little,
we think each of the five is important to some researchers. The
more participation in SERP can advance or at least not jeopar-
dize these goals, the more successful the institution will be at
attracting outstanding researchers.
Reputation and career opportunity are closely linked. Both are
promoted when researchers produce high-quaTity, publishable
work. SERP would promote productivity and quaTit~v in re-
search in its program in several ways, among them:
· By building a high-quaTit~v, well-maintained, longitudinal
database.
High-quaTity, longitudinal datasets are the workhorses of
empirical research. They allow tests of hypotheses regarding
causal relationships that cross-sectional data cannot support.
And they allow correlations over time to be observed that sup-
port new hypotheses and theory building. They also fuel a great
number of publications. The National Education Longitudinal
Study (NELS), conducted by the National Center for Education
Statistics, is a case in point. A large body of research has been
80
STRATEGIC EDUCATION RESEARCH PARTNERSHIP
OCR for page 81
spawned in the years since the first follow-up in 1990 to the
initial data collection in 1988.
In order to carry out the program of research envisioned for
SERP, longitudinal data collection that allows for empirical in-
vestigation of the long-term effects of curricular interventions,
instructional strategies, organizational environment, and policy
choices will be required. In this sense, the needs of the research
program overlap directly with the professional needs of re-
searchers.
To meet both needs will require that data collection be a
high priority and be adequately funded by the institution from
the start.
· By supporting the capacity for rigorous research design.
The overall quality of research produced through SERP will
determine whether affiliation with the institution enhances or
detracts from a researcher's reputation. Providing institutional
standards and support to engage in high-quaTity work can there-
fore enhance both the quality of the individual's work and the
draw of the institution in the community of researchers. SERP
would support quality by creating and maintaining high stan-
dards for research design through the scientific review board
and peer review processes described in Chapter 3.
Quality research is likely to enhance one's reputation more
if it is noticed. By building a coherent program of research, we
would expect individual contributions to carry more signifi-
cance because they are part of a body of work that together
supports a set of understandings and provides a foundation for
decision making. In this sense, the more effective the SERP
effort to steer the research and development program in
productive directions, the greater will be the draw of the
institution.
Many researchers see their greatest career opportunities in
university positions. For this reason, affiliation with SERP must
be compatible with university commitments. The proposed SERP
structure, with most of the research and development being
carried out in dispersed field sites where researchers and practi-
tioners partner, allows for a simultaneous commitment to a
university and to SERP research. In addition, SERP would need
to work actively with universities to obtain permissions for
S E R P N E T W O R K S
81
OCR for page 82
researchers who come to SERP headquarters to obtain multiyear,
fixed-term leaves of absence.
.
We would expect intellectual stimulation to be a natural by-
product of SERP. We would argue that stimulation is likely to
be highest in two situations: when people from the same field
come together to push the boundaries of understanding in that
field, and when people from different fields work together on a
problem, allowing for multiple paradigms and models to gener-
ate new understandings and ideas. SERP would expand the
opportunities for both types of stimulation substantially. The
research agendas of the networks would keep research teams in
close contact and collaboration in order to advance the knowI-
edge base. And SERP would create a venue for a type of inter-
disciplinary work on education issues that has been relatively
rare because education departments in universities have gener-
ally not been successful at attracting scholars from other disci-
plines. But perhaps the most powerful lever SERP would wield
to increase the intellectual options for researchers would come
from the opportunities that SERP is specifically designed to
create and nurture: allowing for disciplinary researchers to work
closely with research-oriented practitioners. Negotiating the re-
lationships with expert practitioners that lead to real collabora-
tive research is not easy, and yet anyone who has benefited
from such a relationship recognizes that it is highly productive
and enormously stimulating.
Researchers who value making a civic contribution through
their work may be thwarted by the logistical difficulty of doing
so. The lack of an infrastructure that links research and practice
makes it difficult for education researchers to contribute to the
education system. Education Week recently published a story
about a Harvard education researcher, Richard Murnane, who
decided to spend a sabbatical year offering his help to the local
school district. While Murnane's effort eventually resulted in an
ongoing connection with the schools through which he is help-
ing train practitioners to effectively use data about their stu-
dents, getting there was not easy. Neither Murnane nor the
schools knew at the outset what, if anything, he could do to be
helpful. His determination kept him in a situation that was at
first quite awkward. By facilitating links between researchers
and schools, SERP would allow for those interested in making a
civic contribution to do so without having to leap such a high
barrier to search for a place in which to be useful individually.
82
STRATEGIC EDUCATION RESEARCH PARTNERSHIP
OCR for page 83
Attracting researchers to problems of education like all
other research problems will be greatly facilitated if there are
financial rewards for doing so. There are clear precedents in
military, space, and medical research. The relatively meager
funding of education research and the instability of the federal
commitment of funds for education research over the years
have not created a strong pull on intellectual resources to the
problems of education. If SERP operates with a relatively stable,
sizable budget as described in Chapter 3, there will be a finan-
cial incentive for researchers to turn their attention to the prob-
lems of educational practice that could become quite powerful
as the commitments of states grow.
ATTRACTING PRACTITIONERS TO SERP
.
What would motivate teachers to participate in collabora-
tive research? We hypothesize rather different incentives in the
case of teachers. Because the rewards of the profession are quite
different, what is likely to be required in order to compete for a
teacher's time commitment will be different as well. While the
financial rewards of teaching are not as great as those in re-
search, what teaching does provide is an opportunity to engage
with students in a meaningful, sometimes life-shaping, way. It
gives one an opportunity to open new worlds, and new oppor-
tunities, to students. SERP, we believe, will be attractive to
teachers if the SERP work allows them to do their jobs more
successfully and to influence the lives of their students more
positively. SERP will provide an opportunity for teachers in the
sense that the knowledge from research will be brought to bear
in context and on the problems of practice. We envision SERP as
providing professional development opportunities for teachers
and summer stipends to attend SERP seminars and workshops.
Teaching can also be attractive for the job stability it pro-
vides and for the predictability of the schedule. But precisely
those characteristics can pose a challenge to attracting teachers
to SERP. The rigidity of a teacher's schedule does not typically
allow for other commitments during a school year, nor for job
security or seniority in the case of an extended absence. For
SERP to attract teachers, institutional arrangements will need to
be made with schools to provide for either a teacher's extended
leave without penalty or a reduced workload to allow for par-
ticipation in a research project during a school year. The nego-
S E R P N E T W O R K S
83
OCR for page 84
tiation of these arrangements through the office of research
services should create opportunities for teachers that are now
lacking.
As with researchers, teachers will vary tremendously in
their drives for intellectual stimulation and to make a contribu-
tion to their profession writ large. Teachers who seek venues for
these purposes currently have few options. Teachers and ad-
ministrators lead activit~v-driven lives; they are rewarded for
working long hours and being endlessly "available." They are
not rewarded for reflection on their practice. Moreover, much of
their work is done in isolation from others in the profession. As
several of the cases in Chapter 2 suggest, practitioners who
want to empirically test teaching approaches or interventions
are often left to find their own path to the world of research
through unmarked territory. Certainly not every teacher is in-
terested in working with research teams. But for those who are,
SERP will provide a venue that does not now exist.
Finally, much of the SERP work would be designed to learn
from teaching practice, so the knowledge that teachers bring
will be actively employed and valued. For teachers who do look
to research, it can be quite frustrating that what is available
often does not address the complexity of the problems they face.
The opportunity to influence the research agenda is likely to be
an important draw for this group.
ATTRACTING FIELD SITES
Why would schools, school districts, or states agree to serve
as field sites for SERP research? The primary answer, in our
view, is that education leaders at all three levels have a desire-
whether motivated by civic contribution, reputation, career op-
portunity, or accountability measures to do a better job of
educating students. If SERP is successful in its early years at
linking research and practice to improve student learning out-
comes, we think that attracting new field sites will be facilitated
by the expectation of continued success. It may be more of a
challenge, however, in the start-up period before proof of con-
cept has been demonstrated.
For this reason, early efforts to establish field sites should
target schools and districts that have demonstrated interests in
engaging in research and development efforts. School districts
in Boston, Chicago, Pittsburgh, and Providence, for example,
84
STRATEGIC EDUCATION RESEARCH PARTNERSHIP
OCR for page 85
currently work with researchers. While drawing on exciting
opportunities is not likely to produce a representative sample of
schools or students, it should provide the foundation for suc-
cess early on that will create the possibility of a representative
sample of field sites further down the road.
The benefits of the SERP research will be available to all
schools in all states that join the SERP compact. It therefore
~ .
stands to reason that the costs of the field site research should be
borne primarily by the organization rather than the participat-
ing schools. To encourage field site participation and an effi-
cient distribution of costs, we propose that SERP funds be used
to buy out teacher time and cover other new costs imposed by
the research. When expenditures substitute for those made rou-
tinely by a participating site (e.g., an investment in data collec-
tion), costs should be shared so that the school is neither taxed
nor subsidized by involvement with SERP.
· ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
WHAT WOULD THEY DO?
If SERP is successful at attracting researchers, practitioners,
and field sites, what will they do when they join a research and
development network? The answer to that question will be
determined by those who are given leadership responsibility in
the new organization. Below we provide our vision of what that
leadership might do.
The committee is proposing the inauguration of three net-
works during the start-up years, although others may eventu-
ally be added. We recommend that a learning and instruction
network be initiated at the very start because the relative matu-
rity of research in the cognitive and developmental sciences
holds promise for real improvement in how teachers are pre-
pared and what they do in the classroom. A second network on
schools as organizations would, we think, be a critical compan-
ion to the learning and instruction network because the organi-
zational environment of the school creates the conditions and
incentives required to support and sustain instructional change.
We anticipate that as work progresses during the start-up years,
each network would develop five to eight related strands of
work that speak to its hub question, employing the full panoply
of research methods (experimental design, longitudinal studies,
S E R P N E T W O R K S
85
OCR for page 86
observational techniques, formative and summative evaluations,
etc.) in an iterative process that produces ever more refined
knowledge.
We propose a third network to house research and develop-
ment on issues that profoundly influence both instruction and
organization management: education policy. This network
would embrace research on issues like accountability testing,
class/school size, education finance, school choice, desegrega-
tion, and other policy issues that create the incentives and envi-
ronment to which teachers and schools respond. How quickly a
third network is put in place will depend in large part on the
funds available to launch the SERP enterprise.
AN ILLUSTRATIVE AGENDA FOR A
SERP NETWORK ON LEARNING AND
I NSTRUCTION
To make the case for the value of a SERP research and
development program more concretely, an expert pane! of prac-
titioners and researchers was convened by the National Re-
search Council to complement the work of our Committee. Its
task was to design an illustrative agenda for one network that
would simulate, in a sense, the role of the agenda-setting advi-
sory board and the leadership of a learning and instruction
network. We selected this network from among those proposed
because the National Academies have in recent years produced
important syntheses of the research literature on human learn-
ing (National Research Council, 1999) and on assessment of
learning (National Research Council, 2001), as well as disci-
pline-specific syntheses in reading and in mathematics (Na-
tional Research Council, 199S, 2001~. These and other explora-
tions of the knowledge base on learning and instruction (National
Institute on Child Health and Human Development, 2000;
RAND, 2002a, 2002b) provided a rich foundation on which our
effort could build.
This focus is not intended to suggest preeminence of the
learning and instruction network. In our judgment, the organi-
zational and implementation issues associated with educational
change are also absolutely central to SERP's mission. Efforts to
improve learning and instruction, the motivation and engage-
86
STRATEGIC EDUCATION RESEARCH PARTNERSHIP
OCR for page 87
ment of students and teachers, and the quality of curricula,
assessments, and instructional materials cannot succeed unless
they are attentive to the variety of organizational and institu-
tional contexts in which instruction occurs. Indeed, in our view,
it is in combination with the work of other networks that the
research and development on learning and instruction will have
maximum likelihood of influencing practice.
GENERATING A CONCEPTUAL FRAMEWORK
Because the intended focus of the SERP program of research
and development is educational practice, the conceptual frame-
work proposed by the learning and instruction panel begins not
with the questions posed by any research discipline, but with
the questions that define teaching practice. The point of present-
ing this conceptual framework here, before we get to the specif-
ics of the agenda proposed for the area of early mathematics, is
to demonstrate how broadly one needs to be able to think about
educational research and improvement how many different
perspectives and sources of knowledge are needed to think
productively about even a single, specific, focused research prob-
lem. SERP's competitive advantage is precisely that it can bring
together the array of researchers and practitioners who can
keep these many different perspectives in play simultaneously.
We can view classroom instruction as organized around a
set of core questions that apply no matter what the subject:
· What do we want students to know or be able to
do?
· What are the typical understandings and precon-
ceptions students hold on this topic at the outset?
· What is the expected progression of understanding
and skill mastery, and what are the predictable points of
difficulty or hurdles that must be overcome?
· What instructional interventions can move students
along a path from their initial understandings and skills
to the desired outcome (curricula, instructional activi-
ties, etch?
· What general and discipline-specific norms and
practices best comprise and support student learning?
· And finally, how can the individual student's
progress be monitored and the student be engaged in
S E R P N E T W O R K S
87
OCR for page 88
the instructional activities that she or he needs to take
the next step toward increased understanding and skill?
Whether explicitly or implicitly, by design or by default,
teachers answer the above questions in the course of their teach-
ing practice. In doing so, teachers (like professionals in other
fields) draw on standards of practice, professional preparation,
background knowledge, tradition, and personal inclinations and
intuitions. A program of research and development can im-
prove the answers to those questions by providing a solid knowI-
edge base to support both teaching practice and professional
preparation, as well as by expanding the instructional and as-
sessment tools available to teachers.
The questions that define educational practice can be in-
formed by several very different fields of research and knowI-
edge:
88
· What students should know or be able to do in an area is
informed (but not fully determined) by disciplinary ex-
pertise. It requires an understanding of the core con-
cepts around which the disciplinary knowledge is orga-
nized, characteristic methods of reasoning and problem
solving, and language and patterns of discourse. What
to teach becomes not only a matter of the information
and skills considered important but also of helping the
student to build the conceptual framework that trans-
forms or helps to organize information into understand-
ngs.
· Knowledge of common student conceptions of a topic
and the expected progression of student thinking requires
careful research on the typical trajectory of understand-
ing. In part this research attempts to identify the nature
and limits of children's changing cognitive abilities with
age and instruction. And in part it attempts to uncover
common understandings that can either support learn-
ing (the ability to halve or double relatively easily in
mathematics) or undermine it (the belief that heat and
temperature are the same thing). Research findings dem-
onstrate the remarkable resilience of students' everyday
understandings even after instruction to the contrary
(and often to the great surprise of teachers). This high-
STRATEGIC EDUCATION RESEARCH PARTNERSHIP
OCR for page 89
lights the importance of a carefully designed research
program to inform and support practice. Research of
this sort is often done by cognitive scientists and educa-
tion researchers, although the knowledge may emerge
from the experience of expert teachers and the observa-
tion of exemplary practice.
· Educative experiences intended to move students
along a learning path constitute the core of what we
consider to be "instruction." These experiences are ulti-
mately created by the teacher, but teachers usually draw
on materials generated by curriculum developers or (less
often) researchers. Instructional programs involve as-
sumptions about the contributors to skill development,
knowledge acquisition, and conceptual change that
should themselves be a research agenda, and the effec-
tiveness of the instructional approach is a matter for
empirical testing.
· General and discipline-specific norms and practices that
support student learning. Learning takes place in ciass-
rooms that are themselves communities. Every commu-
nit~v is distinguished by norms for work and interac-
tions, ranging from when and how people collaborate to
how they speak with one another. Some of those norms
are general rooted in the understanding of schools in a
democratic society; others are specific what it means to
do mathematics differs from what it means to do literary
analysis or chemistry or history. How individual norms
contribute to or undermine student learning, and how
this differs by community, are empirical questions that
draw on sociological and psychological understandings,
as well as on a rigorous evaluation of classroom prac-
tices.
· Assessing the current level of an individual student's
understanding is itself an interdisciplinary undertaking
because it requires an understanding of both what con-
stitutes learning and how to measure it. To be useful in
the learning process, the assessment must be tied to
instructional responses. Assessment, then, requires that
the knowledge relevant to each of the above questions
be incorporated into the design and testing of specific
instruments.
S E R P N E T W O R K S
89
OCR for page 96
companion assessment tool (the Number Knowledge Test) to
help the teacher monitor and guide instruction. If results in
controlled trials (see Chapter 1) could be attained in schools
across the country that serve disadvantaged populations, this
would represent a major success with respect to narrowing the
achievement gap a long-standing national goal that has proven
difficult to realize.
With respect to the second concern, research done in the
1990s investigated the effects on student achievement of in-
struction that builds on informal understandings and empha-
sizes mathematical concepts and reasoning. Cobb et al.'s Prob-
lem Centered Mathematics project (Wood and Sellers, 1997) and
Cognitively Guided Instruction (CGI) in problem solving and
conceptual understanding (Carpenter et al., 1996) both reported
positive effects. With support from the National Science Foun-
dation (NSF), several full-scare elementary mathematics cur-
ricula with embedded assessments have been developed, di-
rected at supporting deeper conceptual understanding of
mathematics concepts and building on children's informal
knowledge of mathematics to provide a more flexible founda-
tion for supporting problem solving. Three curricula developed
separately take somewhat different approaches to achieving
those goals: the Everyday Mathematics curriculum, the Investi-
gations in Number, Data and Space curriculum, and the Math
Trailblazers curriculum (Education Development Center, Inc.,
2001~.
While theories of learning help to identify problems these
curricula have been designed to address, the curricula them-
selves involve theories of instruction that must be tested. Do
their efforts to provide more contextual learning opportunities
that link students' informal thinking to mathematical problem
solving produce students with stronger mathematics skills over-
all, or are there trade-offs among the component skills? Do they
perform as well for students who excel in mathematics as for
students who struggle?
All three curricula show positive gains in student achieve-
ment in implementation studies in which the developers collect
data on program effects. While such findings are encouraging,
they must be viewed with a critical eye, both because those
providing the assessment have a vested interest in the outcome
and because the methodology employed does not allow for
96
STRATEGIC EDUCATION RESEARCH PARTNERSHIP
OCR for page 97
direct attribution of the results to the program. Third-party
evaluations using comparison groups have been done, but none
of these has involved random assignment (the condition that
maximizes confidence in attributing results to the intervention).
Nor do these studies measure either fidelity of implementation
of the reform curriculum for the experimental group or the
specific program features of the alternative used with the con-
tro! group (see, for example, Fuson et al., 2000~.
From the perspective of practice, these are important omis-
sions. To make informed curriculum decisions, teachers and
school administrators need to know what type of implementa-
tion of a specific curriculum produces what results, compared
with what alternatives. Ideally, the reform curricula would be
compared with traditional curricula that are highly rated or
widely used (or both) in order to advance the knowledge base
for practice. Yet to provide the information that is most useful to
practice is a major undertaking. These questions are answer-
able, but research carefully designed to provide the answers
will take a substantial, Tong-term investment.
A s s e s s m e n ~
The curricula described above have embedded assessments
that allow teachers to track student learning. As previously
mentioned, a key feature of the Number Worlds curriculum is
the Number Knowledge Test that allows teachers to closely link
instructional activities for children to the assessment results.
How well other curricula link assessment and instruction is an
issue worthy of investigation.
A separate issue is the assessment over time of the five
strands that constitute mathematical proficiency. The last de-
cade has seen the emergence of a spate of new tests and mea-
sures. No consensus has emerged, however, on critical mea-
surest While there are some standard and widely used
assessment tools to appraise young children's emergent reading
and language skills and competence, no such tools are used on
any comparable basis in primary mathematics.
This type of assessment will be required to evaluate the
effectiveness of a particular curriculum and to make compari-
sons across curricula. For the most part, we lack sophisticated
methods for tracking student learning over time or for examin-
S E R P N E T W O R K S
97
OCR for page 98
ing the contribution of any particular instructional interven-
tions, whether large or small, on students' learning. A research
project that focused on mathematics teaching and learning might
begin by developing such tools.
Teacher Knowlecige
Little is known about what it might take for teachers to use
particular instructional approaches effectively, a necessary ele-
ment of taking any particular approach to scale. The challenges
can be substantial. The curricula mentioned above introduce
major changes in approach to teaching mathematics, and effec-
tive implementation will require that teachers change their view
of mathematics teaching and learning dramatically. In Every-
day Mathematics, for example, teachers are expected to intro-
duce topics that will be revisited later in the curriculum. Com-
plete mastery is not expected with the first introduction. This
has created some confusion for teachers, who are often unclear
about when mastery is sufficient to move on to the next topic
(Fuson et al., 2000~. All of the curricula encourage building on
students' own strategies for problem solving and supporting
engagement through dialogue about the benefits of alternative
strategies. The change required on the part of the teacher to
relinquish control of the answer in favor of a dialogue among
students, where it has been studied, has proven difficult to
master (Palincsar et al., 1989~. The risks of change must be
considered as well: if a teacher does turn control of the discus-
sion over to students but is not prepared to guide that discus-
sion productively, precious little learning may go on. Critical to
the effectiveness of efforts to implement such curricula on a
large scale, then, is that there be adequate teacher preparation
and ongoing support for an entirely different approach to teach-
ing. This is clearly an important area for further study.
One clue regarding teacher knowledge requirements can be
found in research pursued for the most part separately from the
work on student learning and the design of curriculum ap-
proaches, tools, and materials discussed above. Investigations
of teachers' knowledge reveal that although teachers can, for
the most part, "do" the mathematics themselves, they often are
unable to explain why procedures work, distinguish different
interpretations of particular operations, or use a model to closely
map the meaning of a concept or a procedure. For example,
98
STRATEGIC EDUCATION RESEARCH PARTNERSHIP
OCR for page 99
teachers may be able to use concrete materials to verify that the
answer to the subtraction problem in Box 4.1 is 37 and not 43.
They can operate in the world of base ten blocks to solve 51 - 14
but may not be able to use base ten blocks to demonstrate the
meaning of each step of the conventional (or other) algorithm.
Similarly, teachers may be able to compute using familiar
standard algorithms but not be able to recognize, interpret, or
evaluate the mathematical quality of an alternative algorithm.
They may not be able to ascertain whether a nonconventional
method generalizes or to compare the relative merits and disad-
vantages of different algorithms (for example, their transpar-
ency, efficiency, compactness, or the extent to which they are
either error-prone or likely to avert a calculation error). Over
and over, evidence reveals that knowing mathematics for one-
self (i.e., to function as a mathematically competent adult) is
insufficient knowledge for teaching the subject. In the domain
of early number, studies suggest that most teachers' own knowI-
edge is solid, but that their understanding of conceptual foun-
dations is uneven.
Following this work, some materials for use in teachers'
professional development have been developed.2 Modules and
other curriculum materials contain focused work aimed at help-
ing teachers learn the sort of mathematical knowledge of whole
numbers and operations that is needed for teaching. As with the
curricula developed for students' learning discussed above, de-
velopers of teacher learning materials provide some evidence of
teachers' learning of mathematics for teaching, but they have
studied less the role of this learning in the teacher's subsequent
instructional practice and effectiveness.
Still less is known about what teacher developers them-
selves need to know to support teachers' learning and how their
professional learning might be supported. The demand for
skilled leaders who can teach teachers is growing, but the field,
though highly remunerative, is unregulated and all too often
typified by inadequately trained instructors and badly designed
~ Base ten blocks are a common material used to model place value con-
cepts and operations that rely centrally on place value. The materials consist of
a unit cube, a ten-stick built of 10 cubes, a flat square built of 100 cubes or 10
ten-sticks, and a block composed of 1,000 cubes, or 10 flats, or 100 ten-sticks.
2 See, for example, work by Schifter and her colleagues at Education Devel-
opment Center, Developing Mathematics Instruction (Schifter et al., 1999~.
S E R P N E T W O R K S
99
OCR for page 100
delivery methods. Scaling up materials that can support teach-
ers' learning of mathematics for teaching will require worrying
about the knowledge requirements of those who will guide and
support the teachers.
The SERP Agenda
Given the current state of practice and knowledge about
learning and teaching of early number, then, what might a
SERP program of research and development seek to do? How
might it build on what currently exists and begin to extend and
fill gaps in what is known and done, with the ultimate goal of
more reliably and productively building evidence-based instruc-
tional practice? In other words, how could work be planned and
carried out that would extend what is known and take that to
scale in U.S. schools?
The proposed agenda is comprised of three major initia-
tives. The first focuses on developing assessments to measure
student knowledge, a second evaluates promising curricula and
the effects of their particular design features on student out-
comes, and a third focuses on the teacher knowledge require-
ments to comfortably and effectively use curricula that are built
on research-based findings regarding student learning.
Initiative I: Developing Early Mathematics Assessments. Qual-
ity assessments depend on three things: (1) clarity about the
competencies that the assessment should measure; (2) tasks and
observations that effectively capture those competencies; and
(3) appropriate qualitative and quantitative techniques to give
interpretive power to the test results. Clarity about the compe-
tencies to be measured requires a theoretical understanding
(that is empirically supported) of mathematics learning. Unlike
many other areas of the curriculum, early mathematics has the
theoretical and conceptual models, as well as supporting em-
pirical data, on which to build quality assessments. Substantial
work has already been done to specify critical concepts and
skills within this domain, providing assessment developers with
resources on which to draw in drafting the elements of a mea-
surement strategy.
Even with a strong foundation on which to build in early
mathematics, much work remains in designing and testing as-
sessment items to ensure that inferences can be accurately drawn
too
STRATEGIC EDUCATION RESEARCH PARTNERSHIP
OCR for page 101
about student knowledge and competencies. And this work
must be carefully crafted for the specific purpose and use of the
assessment, for example, formative assessment for use in the
classroom to assist learning; summative assessment for use at
the classroom, school, or district level to determine student
attainment levels; or assessment for purposes of program evalu-
ation.
Formative assessments are essentially diagnostic. They can,
for example, provide feedback to the teacher on a student's
mastery of a particular skill or concept or on whether individual
students need more time and practice before moving on to new
material. Summative assessments are also used in the class-
room, but they come at the end of a unit. They give a teacher
feedback on how well the students have mastered and brought
together the set of concepts and skills taught in the unit. These
may be helpful to the teacher in redesigning instruction for the
next year, providing valuable data on students' strengths and
weaknesses that can inform instruction at the next level. School-
or district-level assessments have more general policy purposes,
most commonly to determine attainment levels for groups of
students in order to evaluate the effectiveness of an instruc-
tional program; to monitor attainment by racial, ethnic, or dis-
ability category; and in some cases to hold schools accountable
for the performance of their students.
Currently the different types of assessment are loosely con-
nected at best. Tensions are introduced when strong instruc-
tional programs and accountability assessments are at odds.
Better aligning assessments and tying all assessments firmly
to the theoretical and empirical knowledge base are widely
regarded as critical to improving learning outcomes. The con-
struction of such a system represents a major research, develop-
ment, and implementation agenda that would require the kind
of stability, longevity, and support that SERP intends as its
hallmark.
The above work should be pursued as a collaborative effort
involving teachers, content area specialists, cognitive scientists,
and psychometricians. The effort could use as a departure point
well-established standards in mathematics (e.g., National Coun-
ci! of Teachers of Mathematics), standards-based curricular re-
sources, and rigorous research on content learning to identify
and define what students should know in early mathematics,
S E R P N E T W O R K S
]0]
OCR for page 102
how they might be expected to show what they know, and how
to appropriately interpret student performance. In the case of
formative assessment, this extends to an understanding of the
implications of what the evidence suggests for subsequent in-
struction. In the case of summative assessment, this means un-
derstanding the implications of student performance for mas-
tery of core concepts and principles and the growth of this
mastery over time.
While there are several possible approaches to developing
such a system of student assessments in early mathematics, one
obvious place to begin is with a review of the assessment mate-
rials in existing widely used and exemplary curricular pro-
grams for formative and summative assessments, commercial
testing programs, and state and national tests for policy making
and accountability. These can be reviewed in light of cognitive
theories of mathematical understanding, including empirical
data regarding the validity of specific assessments. Research
needs to focus on evidence of the effectiveness of specific assess-
ments for capturing the range of student knowledge and profi-
ciency for particular mathematical constructs and operations. A
related line of inquiry would focus on issues of assessment
scoring and reliability, particularly ease of scoring, consistency
of scoring within and across individuals, and consistency of
interpretation of the results relative to the underlying cognitive
constructs.
The development of assessments in early mathematics
should be closely tied to complementary initiatives in the areas
of teacher knowledge and curriculum effectiveness. Thus a
strand of research focused on implementation issues should
address the set of questions critical to successful use of quality
assessments:
· What teacher knowledge is necessary to support
effective use of assessments in their instructional prac-
tice? These include teacher understanding of the assess-
meets and their purpose, as well as practical consider-
ations of the time to administer, score, and interpret
results.
· What forms of technology support are needed to
assist teachers in the administration, scoring, and inter-
pretation of a range of standards-based and theory-based
assessments?
102
STRATEGIC EDUCATION RESEARCH PARTNERSHIP
OCR for page 103
· How and to what extent does the process of imple-
menting curriculum-based and standards-based assess-
ments lead to changes in teachers' instructional prac-
tices, and how do these changes affect student learning
outcomes? This investigation should focus both on
changes in the near term and the stability of changes in
the Tong term.
High-quality evidence that permits practitioners, research-
ers, and policy makers to ask and answer comparative ques-
tions will be critical to making the SERP research and develop-
ment usable in practice.
Initiative II: Teacher Knowledge. To take advantage of existing
investments in research and development in elementary math-
ematics will require further work regarding teacher learning
and knowledge requirements and the supports that allow teach-
ers to use these curricula comfortably and effectively. This re-
search should begin with a clear articulation of the principles
and assumptions about student learning that the curriculum
incorporates, comparing these to carefully solicited understand-
ings of teachers. Learning experiences should be designed to
address the points of divergence and tested for their power to
change teacher conceptions.
Further research should test the effectiveness of different
components of professional development on both teacher learn-
ing and the learning of their students. The relative benefits of
teacher guides, videotaped cases, and opportunities to pose
questions and receive support should be tested, as well as the
timing effect (before instruction begins, during instruction, etc.)
for different teacher learning opportunities.
Initiative III: Curriculum Evaluation. The identification (and
further development) of a set of approaches to the teaching of
number and operations that vary on distinct and theoretically
important dimensions would permit careful comparisons of
how particular instructional regimes impact students' learning.
Programs and approaches already developed, such as Number
Worlds, Cognitively Guided Instruction, the three NSF-
supported curricula mentioned above, and well-regarded and
widely used traditional curricula would form the initial core,
S E R P N E T W O R K S
'03
OCR for page 104
but analysis would permit such a core set to be complemented
with other theoretically and practically important alternatives.
Many of the evaluations of the curricula set out to answer
the question, "Does the curriculum improve student achieve-
ment?" While this is an important question and of particular
interest to those who market a curriculum the questions of
importance for Tong-term improvements in practice are why,
for whom, and compared with what? Number Worlds shows
very promising results for disadvantaged children; Everyday
Mathematics does as well. How, and for whom, do those out-
comes differ? Are there trade-offs in the competencies children
gain from each? Does the context in which they work best
differ? Each of the three NSF elementary mathematics curricula
takes a somewhat different approach to instruction. How are
those differences reflected in outcomes for students? Does one
better address the needs of Tow- or high-achieving students?
What are their respective organizational and implementation
requirements? Are there lessons in the outcomes that could be
used to improve any of the curricula or to combine features not
now found in a single curriculum?
An analysis of existing candidate materials could illuminate
important differences, and strategic selections could be made.
The implementation, adaptation, and use of these different ap-
proaches could be followed over time, attending to instructional
practice, students' opportunities to learn, and implementation
issues. In addition, based on what is known about teachers'
knowledge of whole number and operations for teaching, as
well as about teacher learning, systematic variations could be
designed to support the implementation of these different in-
structional approaches. For example, in one set of schools, a
teacher specialist model might be deployed, and, in others,
teachers might engage in a closely focused study of practice
(instruction, student learning, mathematical tasks), co-planning
and analyzing lessons across the year. In still others, teachers
might be provided with both incentives to spend time planning
and adequate teacher guides.
The work could be conducted in carefully controlled, Tongi-
tudinal studies carried out in SERP field sites. A SERP organiza-
tion like that described in Chapter 3 would be well positioned to
carry out such work. Because it would have mutually beneficial
relationships established with a number of field sites and data
collection efforts in those sites already under way, taking on a
]04
STRATEGIC EDUCATION RESEARCH PARTNERSHIP
OCR for page 105
controlled experimental study of alternative curricula would be
a far less daunting task than it would be for researchers working
independently. Moreover, the concern for undertaking research
that is maximally useful to educational practice and the ability
to design and conduct or oversee the conduct of that re-
search will be combined in a single organization. This is a situa-
tion that does not now exist.
The research initiatives described above provide a glimpse
through a single window of a large-scale SERP research and
development enterprise. The companion report provides a more
extensive agenda, but even that is limited to the learning and
instruction network. Perhaps the greatest benefit of the pro-
posed SERP organization is that programs of research on schools
as organizations and on education policy will be developed
alongside that of learning and instruction. Yet, even within the
confines of the early mathematics agenda considered here, dis-
tinguishing features of SERP are apparent. This includes an
effort to define a program of research that focuses on the prob-
lems of practice, strategically building on strengths in the exist-
ing knowledge base and shoring up its weakness.
· ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
WOULD SERP CHANGE PRACTICE?
Even the highest quality SERP research and development
will make a difference only if it is used in practice. What incen-
tives will teachers, schools, districts, and states have to make
use of the SERP work?
AS we mentioned in the opening chapter, the current cli-
mate in which schools are being held accountable for student
performance creates some motivation to search for the means to
improvement. While this may lubricate the wheels of change,
large-scare accountability will not be a sufficient motivator. It is,
by its nature, low-level accountability it generally attends to
gross measures of skill performance. The SERP program, in
contrast, is targeted at improving learning for understanding.
This more difficult change in instruction will require more pow-
erful motivators. These, in the committee's view, are (a) solid
evidence that change will bring clear benefits in student perfor-
mance and (b) support for implementing the change in real
classroom environments.
S E R P N E T W O R K S
'05
OCR for page 106
Regarding the first, we have proposed an investment at the
outset in measures of program outcome for all of the SERP
work. As the case of the Cognitive Tutor Algebra I in Oklahoma
suggests (see Chapter 2), the resistance to making a change
often lies in the doubt that the new program will in fact be
better, as well as the risk that it may be worse. Careful efforts to
document student gains will, we think, serve as a significant
inducement for teachers and schools to change their practices.
In the proposed research and development agenda, the effort to
measure impact permeates every strand of research. Further-
more, the organizational design assigns to those responsible for
quality assurance the task of measuring the impact of SERP
research and development. In doing their job, an additional
source of pressure will be placed on the research and develop-
ment program to define clearly the expected program outcomes,
so that impact data can be compiled as programs are imple-
mented. These data will help inform school districts and states
about the potential improvements associated with change. To-
gether we expect these efforts to provide a powerful motivation
for change.
But motivation is not itself enough, either for the higher-
level change sought by SERP or the Tower-level change encour-
aged through accountability standards. Motivated teachers still
must have the support to change their practice. The idea of
attending seriously to what it takes to use research-based prac-
tices at the school level runs throughout the envisioned SERP
program. Carrying out much of the work in a range of class-
room settings will allow the problems of classroom use to them-
selves be a subject of study and an issue for development and
program design. Indeed, in the Oklahoma case, the combination
of evidence of success and support from the school district and
the program developer to change curriculum resulted in all
teachers preferring the new program, making continued con-
trolled experimentation difficult. It is the unique combination of
attention to carefully measured outcomes and attention to the
requirements for classroom use that leads the committee to be
convinced that SERP can make the kind of change possible that
has been so difficult to achieve in the past.
106
STRATEGIC EDUCATION RESEARCH PARTNERSHIP
Representative terms from entire chapter:
research partnership
