National Academies Press: OpenBook
« Previous: Front Matter
Suggested Citation:"Executive Summary." National Research Council. 1999. Testing, Teaching, and Learning: A Guide for States and School Districts. Washington, DC: The National Academies Press. doi: 10.17226/9609.
×

Executive Summary

Title I of the Elementary and Secondary Education Act is the largest federal effort in precollegiate education. Created in 1965, when the federal government for the first time agreed to provide aid to elementary and secondary schools, the program was designed to level the playing field for disadvantaged students by providing financial assistance to their schools to compensate for the advantages enjoyed at schools with students from more affluent families. Now with an annual budget of approximately $8 billion—a fourth of the U.S. Department of Education's total annual budget—the program reaches more than 11 million students in two-thirds of all elementary schools and a fourth of all secondary schools.

The 1994 reauthorization of Title I represented a profound shift in the program; perhaps the most far-reaching changes were in the assessment arena. Specifically, the law requires states to develop challenging standards for student performance and assessments that measure student performance against the standards. Significantly, the law states that the standards and assessments are expected to be the same for all students, regardless of whether they are eligible for Title I. Thus for the first time, the 1994 statute enshrines into law the principle that Title I students are to be held to the same standards as all other students.

The record of states in implementing the new law shows that the 1994 statute poses a substantial challenge. For example, although nearly every state has adopted content standards, as the law requires, reviews of such standards show that their rigor and usefulness vary widely. In addition, only 21 states adopted performance standards by the law's deadline of 1997–1998; the rest received waivers to allow them more time. The uneven pace of implementation has led some commentators to suggest revising the program substantially.

The purpose of this document is to help states and districts meet the

Suggested Citation:"Executive Summary." National Research Council. 1999. Testing, Teaching, and Learning: A Guide for States and School Districts. Washington, DC: The National Academies Press. doi: 10.17226/9609.
×

challenges posed by the law by guiding them in making appropriate decisions in implementing it. The Committee on Title I Testing and Assessment was charged with assessing research on the use of testing and assessment for accountability purposes, examining the experience of states and districts in this domain, and developing a “decision framework that incorporates technical quality, effects on teaching and learning, costs and benefits, fairness, and other criteria for evaluating assessment strategies.” Our goal was to produce a practical guide for states and districts to use in developing the systems they were creating under the Title I law.

As we studied the research, examined our own experiences, and listened to testimony from state, district, and school officials, the committee kept in mind three underlying principles. First, the committee agreed that the purpose of assessments and accountability is to contribute to and support high levels of student learning, particularly for disadvantaged students who have lagged behind their more advantaged peers. Second, the committee agreed that the education improvement system should be conceived of and implemented as just that—a system. That is, the system should consist of a number of components, at various levels (classroom, school, school district, and state), each of which plays a role in measuring and contributing to student learning, yet which are interrelated, not separate from one another. Third, the committee agreed that a hallmark of the state and district systems should be continuous improvement, at all levels—for students, for teachers and administrators, and for the system itself. For these and other reasons, the committee developed criteria not for the one best system—which does not exist—but for systems that continually change and adapt to new knowledge and circumstances. States and districts need to continually monitor the effects of their policies and practices to ensure that they are attaining their goals. The committee's framework is appropriate for states and districts just starting out on redesigning their education improvement system, as well as for states and districts that have had redesigned systems in place for several years.

Standards-Based Reform

The provisions of the 1994 law carry with them an implied “theory of action” that suggests how implementing them will achieve the larger goal of improving student learning.

As we understand it, the theory of action underlying the 1994 law is relatively straightforward. The centerpiece of the system is a set of challenging standards for student performance. By setting these standards for all students, states would hold high expectations for performance; these expectations would be the same regardless of students' backgrounds or where they attended school. Aligning assessments to the standards would allow students, parents, and teachers to monitor student performance against the standards. Providing flexibility to

Suggested Citation:"Executive Summary." National Research Council. 1999. Testing, Teaching, and Learning: A Guide for States and School Districts. Washington, DC: The National Academies Press. doi: 10.17226/9609.
×

schools would permit them to make the instructional and structural changes needed for their students to reach the standards. And holding schools accountable for meeting the standards would create incentives to redesign instruction toward the standards and provide appropriate assistance to schools that need extra help.

Embedded in this theory are a number of assumptions that experience since 1994 has led the committee to call into question. Chief among these assumptions is the idea that teachers would institute effective practices if they had both the freedom and the motivation to do so. In addition, we question the assumption that motivated teachers would seek guidance about improving instruction and districts would provide the support teachers need, largely by making more widely available the existing array of professional development opportunities.

As a result of our examination of the theory of action, the committee concludes that the theory needs to be expanded to make explicit the link between standards, assessments, accountability, instruction, and learning. In our view, standards-based policies can affect student learning only if they are tied directly to efforts to build the capacity of teachers and administrators to improve instruction.

An Expanded Theory

What would such a system look like? In our view, the focus would be on teaching and learning, and the theory of action revolves around the links between all the elements and instruction. We call the expanded system an “education improvement system.”

The theory of action behind an education improvement system relies on information and responsibility. Everyone in the system—students, parents, teachers, administrators, and policy makers at every level—needs high-quality information about the quality of instruction and student performance. At the same time, everyone needs to be responsible for fulfilling his or her role in improving results. The key is transparency: everyone should know what it is expected, what they will be measured on, and what the results imply for what they should do next.

Such a system is never “complete”; educators and policy makers continue to modify and adapt it as they learn from their own experience and the experience of others. States and districts need to examine each component, and the system as a whole, continually, to determine the extent to which it is achieving the goal of improving teaching and learning. In the following section we outline the criteria for the components.

Suggested Citation:"Executive Summary." National Research Council. 1999. Testing, Teaching, and Learning: A Guide for States and School Districts. Washington, DC: The National Academies Press. doi: 10.17226/9609.
×

Components of an Education Improvement System

Standards

Standards for student performance are at the heart of the system. Standards set the expectations for student learning, and signal that all students, regardless of background or where they happen to attend school, are expected to demonstrate high levels of knowledge and skill. In addition, they focus the attention of everyone in the system on the results schooling is expected to achieve—academic performance—rather than the resources or effort put into the system.

Content standards spell out what students should know and be able to do in core subjects. They should be clear, parsimonious, and rigorous. Performance standards indicate the level of performance students should demonstrate. They should include: performance categories, performance descriptors, exemplars of performance in each category, and decision rules that enable educators to determine whether students have reached each category.

Assessments.

Assessments in standards-based systems serve a number of purposes: guiding instruction, monitoring school and district performance, holding schools accountable for meeting performance goals, and more. No single instrument can serve all purposes well. Assessment should involve a range of strategies appropriate for inferences relevant to individual students, classrooms, schools, districts, and states.

In order to provide information on the quality of instruction and provide cues to help educators improve teaching and classroom practices, the overwhelming majority of standards-based assessments should be sensitive to effective instruction; that is, they should detect the effects of high-quality teaching. Districts, schools, and teachers should use the results of these assessments to revise their practices to help students improve performance.

Assessments are essential to measure the performance of all children. Yet, although 49 percent of children served by Title I are in grades 3 and below, the 1994 statute does not require states to establish assessments before grade 3. Without some form of assessment, schools and districts would have no way of determining the progress of this large group of students to ensure that they do not fall too far behind.

To measure the performance of young children, teachers should monitor the progress of individual children in grades K to 3 at multiple points in time by using direct assessments, portfolios, checklists, and other work sampling devices. And schools should be accountable for promoting high levels of reading and mathematics performance for primary grade students. For school accountability in grades 1 and 2, states and districts should gauge school quality through the use of sampling, rather than the assessment of every pupil.

Suggested Citation:"Executive Summary." National Research Council. 1999. Testing, Teaching, and Learning: A Guide for States and School Districts. Washington, DC: The National Academies Press. doi: 10.17226/9609.
×

Including students with disabilities and English-language learners in assessments also poses significant challenges. Although state policies vary widely, many states exclude large numbers of students with disabilities and English-language learners from assessment mandates. Others include such students but use measures that may not be appropriate.

States and districts should develop clear guidelines for accommodations that permit students with disabilities to participate in assessments administered for accountability purposes.

Similarly, states and districts should develop clear guidelines for accommodations that permit English-language learners to participate in assessments administered for accountability purposes. Especially important are clear decision rules for determining the level of English language proficiency at which English-language learners should be expected to participate exclusively in English-language assessments. English-language learners should be exempted from assessments only when there is evidence that the assessment, even with accommodations, cannot measure the knowledge or skill of particular students or groups of students.

In an education improvement system, data from assessments provide information that teachers and administrators can use to revise their instructional program to enable students to reach challenging standards. For that reason, assessment results should be reported so that they indicate the status of student performance against standards. To ensure accuracy, reports of student performance should include measures of statistical uncertainty, such as a confidence interval or the probability of misclassification. States, districts, and schools should disaggregate data to ensure that schools will be accountable for the progress of all children, especially those with the greatest educational needs.

Monitoring the Conditions of Instruction

The theory of action of the basic standards-based reform model suggests that, armed with data on how students perform against standards, schools will make the instructional changes needed to improve performance. Research on early implementation of standards-based systems shows, however, that many schools lack an understanding of the changes that are needed and lack the capacity to make them. The link between assessment and instruction needs to be made strong and explicit.

One way to forge such a link is by monitoring the conditions of instruction and instructional support. Information about the effects of instructional change—particularly student work that shows the quality of assignments—sends a strong signal about the kinds of changes needed and the impact of new practices. In addition, such information serves as “leading indicators” of performance.

Schools and districts should monitor the conditions of instruction—the

Suggested Citation:"Executive Summary." National Research Council. 1999. Testing, Teaching, and Learning: A Guide for States and School Districts. Washington, DC: The National Academies Press. doi: 10.17226/9609.
×

curriculum and instructional practices of teachers—to determine if students are exposed to teaching that would enable them to achieve the standards they are expected to meet. Schools should use such information to demand support for instructional improvement in every classroom, and districts should use the information to provide such support.

Districts should also use data on the conditions of instruction, along with results from student assessments, to design their professional development program.

Accountability

Accountability is one of the most prominent issues in education policy today. Accountability mechanisms create incentives for educators to focus on important outcomes. They also provide a means for allocating resources, such as instructional assistance, to schools in which performance measures indicate problems.

In designing accountability mechanisms, states and districts must first determine an adequate level of progress for schools. Measures of adequate yearly progress should include a range of indicators, including indicators of instructional quality as well as student outcomes. In addition, the criterion for adequate yearly progress should be based on evidence from the highest-performing schools with significant proportions of disadvantaged students.

Accountability should follow responsibility: teachers and administrators—individually and collectively—should be held accountable for their part in improving student performance. Teachers and administrators should be held accountable for the progress of their students. Districts and states should be held accountable for the professional development and support they provide teachers and schools to enable students to reach high standards.

Accountability provides a way to focus assistance to schools. Assistance should be aimed at strengthening schools' capacity for educating all students to high standards and to building the internal accountability within schools. Without developing school capacity, accountability leads to inappropriate practices, such as efforts to increase test scores without improving student learning.

Education improvement systems continually change, based on new knowledge and new circumstances. States and districts should continually monitor and review their systems to determine where improvements are needed and make the changes necessary to improve educational opportunities for all children, and particularly for the disadvantaged children Title I was established to support.

Suggested Citation:"Executive Summary." National Research Council. 1999. Testing, Teaching, and Learning: A Guide for States and School Districts. Washington, DC: The National Academies Press. doi: 10.17226/9609.
×
Page 1
Suggested Citation:"Executive Summary." National Research Council. 1999. Testing, Teaching, and Learning: A Guide for States and School Districts. Washington, DC: The National Academies Press. doi: 10.17226/9609.
×
Page 2
Suggested Citation:"Executive Summary." National Research Council. 1999. Testing, Teaching, and Learning: A Guide for States and School Districts. Washington, DC: The National Academies Press. doi: 10.17226/9609.
×
Page 3
Suggested Citation:"Executive Summary." National Research Council. 1999. Testing, Teaching, and Learning: A Guide for States and School Districts. Washington, DC: The National Academies Press. doi: 10.17226/9609.
×
Page 4
Suggested Citation:"Executive Summary." National Research Council. 1999. Testing, Teaching, and Learning: A Guide for States and School Districts. Washington, DC: The National Academies Press. doi: 10.17226/9609.
×
Page 5
Suggested Citation:"Executive Summary." National Research Council. 1999. Testing, Teaching, and Learning: A Guide for States and School Districts. Washington, DC: The National Academies Press. doi: 10.17226/9609.
×
Page 6
Next: 1 Introduction »
Testing, Teaching, and Learning: A Guide for States and School Districts Get This Book
×
Buy Paperback | $31.00 Buy Ebook | $24.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

State education departments and school districts face an important challenge in implementing a new law that requires disadvantaged students to be held to the same standards as other students. The new requirements come from provisions of the 1994 reauthorization of Title I, the largest federal effort in precollegiate education, which provides aid to "level the field" for disadvantaged students.

Testing, Teaching, and Learning is written to help states and school districts comply with the new law, offering guidance for designing and implementing assessment and accountability systems. This book examines standards-based education reform and reviews the research on student assessment, focusing on the needs of disadvantaged students covered by Title I. With examples of states and districts that have track records in new systems, the committee develops a practical "decision framework" for education officials.

The book explores how best to design assessment and accountability systems that support high levels of student learning and to work toward continuous improvement. Testing, Teaching, and Learning will be an important tool for all involved in educating disadvantaged students—state and local administrators and classroom teachers.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!