Chapter 2
Toward a Theory of Action

The task of the Committee on Title I Testing and Assessment was to develop a guide for states and districts to assist them in implementing the Title I statute. This guide includes criteria for the components of an effective education improvement system, along with examples of ways states and districts have applied these criteria. But to understand the committee's point of view, we need first to present the big picture—the “theory of action” that animates the entire system.

The 1994 law that reauthorized Title I of the Elementary and Secondary Education Act drew on a powerful strain of education thinking that has grown increasingly prominent in the past decade. Beginning with the publication of the National Council of Teachers of Mathematics Standards in 1989, and accelerating after the establishment a year later of the national education goals, educators and policy makers have increasingly focused on standards for student performance as the centerpiece of education reform; indeed, the idea has since acquired the name “standards-based reform.” The Title I statute fits squarely within that tradition.

Generally, the idea of standards-based reform states that, if states set high standards for student performance, develop assessments that measure student performance against the standards, give schools the flexibility they need to change curriculum, instruction, and school organization to enable their students to meet the standards, and hold schools strictly accountable for meeting performance standards, then student achievement will rise.

This idea is not unique to education. A number of businesses have implemented similar principles and have won acclaim as high-performing organizations. The Saturn Corporation, for example, which was created by General Motors and the United Auto Workers during the steep slump in the domestic automobile industry, has attracted considerable attention for its innovative



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 15
--> Chapter 2 Toward a Theory of Action The task of the Committee on Title I Testing and Assessment was to develop a guide for states and districts to assist them in implementing the Title I statute. This guide includes criteria for the components of an effective education improvement system, along with examples of ways states and districts have applied these criteria. But to understand the committee's point of view, we need first to present the big picture—the “theory of action” that animates the entire system. The 1994 law that reauthorized Title I of the Elementary and Secondary Education Act drew on a powerful strain of education thinking that has grown increasingly prominent in the past decade. Beginning with the publication of the National Council of Teachers of Mathematics Standards in 1989, and accelerating after the establishment a year later of the national education goals, educators and policy makers have increasingly focused on standards for student performance as the centerpiece of education reform; indeed, the idea has since acquired the name “standards-based reform.” The Title I statute fits squarely within that tradition. Generally, the idea of standards-based reform states that, if states set high standards for student performance, develop assessments that measure student performance against the standards, give schools the flexibility they need to change curriculum, instruction, and school organization to enable their students to meet the standards, and hold schools strictly accountable for meeting performance standards, then student achievement will rise. This idea is not unique to education. A number of businesses have implemented similar principles and have won acclaim as high-performing organizations. The Saturn Corporation, for example, which was created by General Motors and the United Auto Workers during the steep slump in the domestic automobile industry, has attracted considerable attention for its innovative

OCR for page 15
--> standards-based structure. At Saturn, the company sets high goals for performance (e.g., a standard of zero defects), measures performance regularly, and gives extraordinary authority to teams of workers, including line workers, while linking their pay and job security to performance. Likewise, a number of public-sector agencies are “reinventing government” by adopting similar principles (Osborne and Gaebler, 1993). In education, the idea of standards-based reform grew in part out of the same notions that drove the reforms in business and government, but also out of a critique of previous education reform efforts, particularly the experience with Title I and Chapter 1. Yet despite the prominence of standards-based reform in the policy debate, there are few examples of districts or states that have put the entire standards-based puzzle together, much less achieved success through it. Some evidence is beginning to gather. Grissmer and Flanagan (1998), for example, found that North Carolina and Texas have produced gains in student performance through the implementation of standards-based systems. Other evidence comes from Europe and Asia, where national systems of education have produced curriculum guides and related assessments, and where many countries outperform the United States on international assessments (Schmidt et al., 1998). In large part, the limited body of evidence in this country reflects the complexity of the concept. It requires substantial changes in a number of major interlocking dimensions, and education policy seldom occurs in such a systematic fashion. Moreover, it poses the technical challenge of creating new instruments and systems, all of which are exceedingly controversial and costly, in the center of a highly charged political arena. The Standards-Based Reform Model The theory of action of standards-based reform rests on four major components: standards, assessments, flexibility, and accountability. It is represented graphically in Figure 2-1. Setting Standards. As its name suggests, standards-based reform rests primarily on standards for student performance. The standards should be clear, Figure 2-1 Model of the theory of action of standards-based reform

OCR for page 15
--> high, and the same for all students. Reformers argue that setting clear, high standards for all students will help improve their performance by giving students, parents, and teachers a vivid picture of what good work looks like and what they have to do to produce it. Aligning Assessments. Assessments are linked to standards so closely in discussions of standards-based reform that the two are often referred to almost as one word: “standards-and-assessments.” But the link is important. Assessments make the standards concrete by providing students with opportunities to demonstrate the knowledge and skills the standards call for. At the same time, they serve as a means by which students, parents, teachers, and administrators can know the extent to which students are meeting the standards. Providing Flexibility. For years, educators have complained that the plethora of rules associated with Title I have hamstrung their efforts to redesign their instructional programs and have forced them to use questionable practices in order to comply with statutory mandates. For example, administrators say, schools have pulled Title I students out of their regular classrooms in order to provide specialized instruction for them, even though research suggests that such programs have been implemented in ineffective ways, because schools were required to demonstrate that they were in fact providing compensatory education services to eligible children. Standards-based reform changes the rules of the game by measuring performance against standards rather than compliance with procedures. Policy makers will know if their money is spent well if student performance improves, not if schools follow rules faithfully. Thus, lawmakers can relax rules that mandate how schools must go about their jobs. And that, in turn, will help improve student performance, reformers say, by reducing the impediments schools now face in designing instructional programs appropriate for their student populations. Requiring Accountability. Accountability is the flip side of the coin of flexibility. In exchange for the freedom to design instructional programs according to local needs, schools in standards-based systems are no longer held accountable for following rules and procedures and making sure that funds are spent as intended. Rather, they are accountable for results—for ensuring that student learning improves. Holding schools accountable for results serves a number of purposes. Accountability helps keep educators' “eyes on the prize,” reducing the possibility that they will spend their time on issues less directly related to improving student performance. On the other hand, accountability creates an incentive for teachers and administrators at all levels to use standards to guide curricular and instructional decisions, and to use assessment results to diagnose problems and suggest ways to improve. On the other hand, holding schools accountable for some other set of instructional goals will encourage schools to focus on those goals, rather than the standards, regardless of how compelling the standards may be.

OCR for page 15
--> Although each of these elements—standards, assessments, flexibility, and accountability—is itself complex and challenging to administer, the essence of standards-based reform is the idea that states must implement all of them. Reformers argue that previous education reforms failed because they were piecemeal; they addressed one aspect of the system while leaving the rest untouched, and failed to address the core of schooling. Without a comprehensive change, standards-based reform will suffer the same fate. How, then, to implement such a massive change? The Title I statute lays out a precise schedule for implementing standards-based reform. The law's sequence is as follows: flexibility, standards, assessments, and accountability. Not all states and districts have followed this linear sequence. In some places, political exigencies have led policy makers to put in place accountability measures before standards and assessments were revised. Others followed a different approach because of a different conception of how to achieve change. For example, Community District 2 in New York City started with a vision of teaching and learning and invested heavily in developing teachers' knowledge and skills to be able to realize the vision. They held teachers and administrators accountable for the quality of instruction and made sure that everyone in the system, from teachers all the way to the deputy superintendent, knows the quality of the staff, the quality of teaching, and the quality of student work in each school. Only after years of developing teachers' abilities—and after rising from 16th to 2nd among New York City districts in performance on conventional tests—did the district adopt standards and a testing system that they believed reflected their instructional goals (Elmore and Burney, 1998). Regardless of the approach, the expectation is the same: comprehensive standards-based reform systems will result in students' meeting high standards for performance. The Theory in Practice In a study conducted for the National Education Goals Panel, David Grissmer and Ann Flanagan (1998) examined two states that registered large gains in student performance in mathematics and reading in the 1990s, North Carolina and Texas. They found that many of the factors often associated with improved student performance—increases in education spending, reductions in class size, changes in the student population—did not explain the results in the two states they studied. Rather, they suggested, what the two states had in common were a set of statewide policies that coincided with the increases in test scores. These policies were: statewide academic standards, by grade, for clear teaching objectives, holding all students to the same standards, statewide assessments closely linked to the standards, accountability systems with consequences for results, increasing local flexibility for administrators and teachers, computerized feedback systems and data for continuous improvement, shifting resources

OCR for page 15
--> to schools with more disadvantaged students, and an infrastructure to sustain reform. These policies were, in short, the main elements of standards-based reform. Grissmer and Flanagan found few data to show how teachers and administrators in North Carolina and Texas changed their practices in ways that produced higher test scores. “But,” they conclude, “it appears to be the changed design of the organizational environment and competitive incentive structure which is responsible for teachers and administrators finding creative ways to foster higher achievement in their students” (p. 21). Other evidence suggests that standards-based reform can be effective when district policies to establish standards-based assessments and accountability mechanisms are coupled with strategies for instructional improvement. Case studies of reform efforts in San Antonio, Philadelphia, and Memphis, for example, show that these districts achieved gains after instituting standards-based accountability systems and assistance to local schools to revise curricular and instructional practices (Citizens' Commission on Civil Rights, 1998). Other studies suggest that if the link between standards-based policies at the state and district levels and instructional improvement at the school level is not clear-cut, then higher student performance may not result. In these instances, the theory of standards-based reform may not work as designed. For example, an examination of district policies that call for “reconstitution” of failing schools (breaking up the faculty and staff and rebuilding it from the ground up) found that schools threatened with severe penalties are not changing their instructional practices in fundamental ways. Instead, they seem to focus on short-term gains in test scores, rather than deep improvements in student learning (O'Day, in press). Another study of 20 schools found that the internal accountability within schools—that is, teachers' collective responsibility for improving student learning and for making the changes necessary to bring such improvements about—varies widely (Abelmann and Elmore, 1999). When such internal accountability is weak, the willingness of teachers to change their practice in fundamental ways to respond to external accountability pressures may be lacking. These studies and observations from our own experience have led the committee to call into question some of the assumptions that appear to be embedded in the theory of action underlying the standards-based reform model in general and the Title I law in particular. Chief among these assumptions is the idea that teachers would institute effective practices if they had both the freedom and the motivation to do so. Relaxing rules would provide the freedom; holding schools accountable for results would provide the motivation. The committee found that this idea may be overoptimistic. First, it assumes that teachers—indeed, the education profession generally—know enough about what it takes to educate all children to challenging standards of performance. The experience since 1994 suggests that, although some schools and communi-

OCR for page 15
--> ties are showing success, their practices are not widely shared, and knowledge about how to implement effective instructional strategies to help all students learn to challenging standards is also largely unknown. Second, implicit in the theory is the notion that motivated teachers would seek guidance about improving instruction and districts would provide the support teachers need, largely by making more widely available the existing array of professional development opportunities. Recent research suggests, however, that the amount and kind of professional development is inadequate to meet teachers' needs, and that teachers continue to feel unprepared to teach all students to challenging standards (National Center for Education Statistics, 1999; National Partnership for Excellence and Accountability in Teaching, 1999). As a result of our examination of the theory of action, the committee concludes that the theory needs to be expanded to make explicit the link between standards, assessments, accountability, instruction, and learning. In our view, standards-based policies can affect student learning only if they are tied directly to efforts to build the capacity of teachers and administrators to improve instruction. An Expanded Theory What would such a system look like? In our view, the focus would be on teaching and learning, and the theory of action revolves around the links between all the elements and instruction. We call the expanded system an “education improvement system,” and it is represented graphically in Figure 2-2. The theory of action behind an education improvement system relies on information and responsibility. Everyone—students, parents, teachers, principals, district administrators, state officials, and policy makers at the district, state, and federal levels—knows what it is expected, what they will be measured on, and what the results imply for what they should do next. Those directly responsible for raising student performance—teachers and schools—have access to high-quality information about performance and about the effects of their instruc- Figure 2-2 Expanded model of the theory of action of standards-based reform: An education improvement system

OCR for page 15
--> tional practices. They are then responsible for using this information to adjust their practices and seek support for additional resources for improvement. But others have responsibilities as well, since student performance depends on the capacity of teachers and administrators to deliver high-quality instruction. Therefore, the education improvement system also provides information on the progress of efforts to develop instructional capacity. In all cases, the information the system provides is transparent—that is, it shows results and suggests remedies. In addition, the information provides a means for states and districts to monitor the effects of their changes and make course corrections when warranted. As with the conventional model, the theory of action for education improvement systems is based on the idea that a number of components work hand in hand. States and districts can develop these components in any order; what matters is coherence among the components. The components of an education improvement system are: standards, assessments, indicators of the conditions of instruction, and accountability. Standards. As with standards-based reform, challenging standards for student performance drive instructionally valid standards-based systems. Content standards set expectations for learning for all students, and performance standards are the benchmarks against which progress is gauged. Performance standards also provide instructional guidance by offering clear ideas of classroom strategies to enable students to reach the standards. Assessments. Assessments provide information on progress toward the standards, but they do so in different ways for different constituencies. Assessments serve a number of purposes—guiding instructional decisions, monitoring progress, holding schools and districts accountable. Classroom assessments provide frequent and detailed information about individual student strengths and weaknesses, district assessments monitor school progress toward standards, and state assessments provide data for use in accountability systems. School reports consist of a range of measures—which include indicators of instructional practices, as well as student work and test scores—that provide a complete picture of performance. The reports indicate the performance of groups of students within the school or district; overall average scores may be misleading. Not all assessments are equally capable of providing useful information. The most informative measures are ones that respond to instructional changes aimed at teaching toward the standards. Such measures inform students, teachers, and parents about the effects of instruction and suggest directions for improvement. The array of assessments include assessments that are appropriate for young children, as well as assessments that accurately and validly measure the achievement of students with disabilities and those with limited English proficiency.

OCR for page 15
--> Indicators of the Conditions of Instruction. The link between instructional practice and student performance suggests the need to collect information on the conditions of instruction to which students are exposed, in addition to the student outcome measures used in the assessment system. Such indicators serve as “leading” indicators of school progress and suggest needed areas of improvement. They also could point out possible inequalities. Measures of instructional practice at the district level also indicate the extent to which districts are fulfilling their role in building local capacity to improve instruction and student performance. Accountability. Accountability creates an incentive for students, teachers, and administrators to focus their attention on the standards. It also closes the loop in the system by providing an explicit link to instructional improvement; rather than hit the hammer harder, administrators provide assistance where the accountability measures suggest it is needed, and direct teachers' and school administrators' attention to the standards. Such a system is never “complete.” States and districts need to examine each component and the system as a whole, continually, to determine the extent to which it is achieving the goal of improving teaching and learning. In the following chapters we outline the criteria for each of the components.