Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Investigating the INFLUENCE OF STANDARDS A Framework for Research in Mathematics, Science, and Technology Education Committee on Understanding the Influence of Standards in K-12 Science, Mathematics, and Technology Education Iris R. Weiss, Michael S. Knapp, Karen S. Hollweg, and Gail Burrill, editors Center for Education Division of Behavioral and Social Sciences and Education National Research Council NATIONAL ACADEMY PRESS Washington, DC
NATIONAL ACADEMY PRESS ⢠2101 Constitution Avenue, NW ⢠Washington, DC 20418 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance. This study was supported by Contract/Grant No. ESI-9618770 between the National Academy of Sciences and the National Science Foundation. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project. Library of Congress Cataloging-in-Publication Data Investigating the influence of standards : a framework for research in mathematics, science, and technology education / Iris R. Weiss ... [et al.]. p. cm. Includes bibliographical references and index. ISBN 0-309-07276-X (pbk.) 1. MathematicsâStudy and teachingâStandardsâUnited StatesâEvaluation. 2. ScienceâStudy and teachingâStandardsâUnited StatesâEvaluation. 3. Technical educationâStandardsâUnited StatesâEvaluation. I. Weiss, Iris R. QA13 .I55 2001 507.1â²073âdc21 2001006094 Additional copies of this report are available from National Academy Press, 2101 Constitution Avenue, NW, Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu Printed in the United States of America Copyright 2002 by the National Academy of Sciences. All rights reserved. Suggested citation: National Research Council. (2002). Investigating the influence of standards: A framework for research in mathematics, science, and technology education. I.R. Weiss, M.S. Knapp, K.S. Hollweg, and G. Burrill (Eds.), Committee on Understanding the Influence of Standards in K-12 Science, Mathematics, and Technology Education, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.
National Academy of Sciences National Academy of Engineering Institute of Medicine National Research Council The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M. Alberts is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineer- ing programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Wm. A. Wulf is presi- dent of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine. The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academyâs purposes of furthering knowledge and advising the federal govern- ment. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communi- ties. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M. Alberts and Dr. Wm. A. Wulf are chairman and vice chairman, respectively, of the National Research Council.
COMMITTEE ON UNDERSTANDING THE INFLUENCE OF STANDARDS IN K-12 SCIENCE, MATHEMATICS, AND TECHNOLOGY EDUCATION Iris R. Weiss (Chair), Horizon Research, Inc. William Browder, Princeton University Maria Alicia Lopez Freeman, California Science Project, University of California, Los Angeles Michael Fullan, Ontario Institute for Studies in Education, University of Toronto Margaret Goertz, Consortium for Policy Research in Education, University of Pennsylvania Michael S. Knapp, Center for the Study of Teaching and Policy, University of Washington Paul LeMahieu, Hawaii Department of Education Mari Muri, Connecticut Department of Education Patrick M. Shields, SRI International Judith Sowder, Department of Mathematical and Computer Science, San Diego State University Elizabeth K. Stage, Office of the President, University of California Kendall Starkweather, International Technology Education Association P. Uri Treisman, The Charles A. Dana Center for Mathematics and Science Education, University of Texas at Austin Marc Tucker, The National Center on Education and the Economy Staff, Center for Education Karen S. Hollweg, Project Director Gail Burrill, Senior Program Officer Tina Winters, Senior Project Assistant* Viola Horek, Administrative Associate** Henry Heikkinen, Consultant Writer/Editor *Through March 2001 **Since March 2001 v
COMMITTEE ON SCIENCE EDUCATION K-12 J. Myron Atkin (Chair), School of Education, Stanford University Carol Brewer, Division of Biological Sciences, University of Montana Caryl Edward Buchwald,* Carleton College George Bugliarello,** Polytechnic University Elizabeth A. Carvellas,* Chittenden Central School District, Vermont Christine Chopyak-Minor,**** Keystone Science School, Colorado Beatrice Chu Clewell,*** Urban Institute Peter B. Dow,**** First Hand Learning, Inc. William E. Dugger, Jr.,* International Technology Education Association Hubert Dyasi, City College (City University of New York) Wade Ellis, Jr.,**** West Valley College Maria Alicia Lopez Freeman, California Science Project Norman Hackerman,* The Robert A. Welch Foundation Patty Harmon, San Francisco Unified School District Robert M. Hazen,**** Carnegie Institute of Washington Leroy Hood,** Institute for Systems Biology Anne Jolly, SERVE Jane Butler Kahle,**** Miami University, Ohio Michael G. Lang,**** Maricopa Community College Ron Latanision, Massachusetts Institute of Technology William Linder-Scholer, SciMathMN Jim Minstrell, Talaria Inc. John A. Moore,** University of California, Riverside Darlene Norfleet,** Flynn Park Elementary School, Missouri Carolyn L. Ray,* Urban Systemic Initiative, Euclid, Ohio Cary I. Sneider, Boston Museum of Science William E. Spooner,*** CBSG Judith Sydner-Gordon,**** Los Angeles County Office of Education Jerry Valadez, Fresno Unified School District Rachel Wood,* Delaware State Department of Education Robert Yinger, School of Education, Baylor University *Term ended January 2001 **Term ended January 2000 ***Term ended July 1999 ****Term ended January 1999 vi
MATHEMATICAL SCIENCES EDUCATION BOARD Joan R. Leitzel (Chair), University of New Hampshire Jere Confrey (Vice Chair), University of Texas at Austin Judy Ackerman, Montgomery College Richard A. Askey,* University of WisconsinâMadison Sherry Baca,** Prescott High School, Arizona Deborah Loewenberg Ball, University of Michigan Hyman Bass,** University of Michigan Benjamin Blackhawk,*** St. Paul Academy and Summit School Richelle Blair,* Lakeland Community College Patricia Campbell,*** University of Maryland, College Park Ingrid Daubechies,* Princeton University Jan de Lange, Freudenthal Institute Keith Devlin, St. Maryâs College of California Karen Economopoulos,* TERC Susan Eyestone,* National PTA Joan Ferrini-Mundy, Michigan State University Arthur M. Jaffe, Harvard University Lee Jenkins,*** Enterprise School District, California Dan Kennedy, The Baylor School, Tennessee Glenda T. Lappan,** Michigan State University Karen Longhart, Flathead High School, Montana Miriam Masullo,* IBM Corporation David S. Moore,*** Purdue University Thomas L. Moore, Grinnell College Mari Muri,** Independent Consultant Richard S. Normington,** TQM Services Group Debra Paulson, Hornedo Middle School, Texas Marge M. Petit, The National Center for the Improvement of Educational Assessment Mark Saul,** Bronxville High School, New York Anthony Scott, Chicago Public Schools Edward A. Silver,** University of Michigan William Steenken, GE Aircraft Engines Lee V. Stiff,* North Carolina State University James W. Stigler,* University of California, Los Angeles William F. Tate,** University of WisconsinâMadison Jerry Uhl,* University of Illinois at UrbanaâChampaign Susan S. Wood,** J. Sargeant Reynolds Community College *Term ended June 2001 **Term ended June 2000 ***Term ended June 1999 vii
ACKNOWLEDGMENTS his report could not have happened without support from a number T of people, and I am grateful for their contributions. On behalf of the Committee on Understanding the Influence of Standards in Science, Mathematics, and Technology Education, I want first of all to acknowledge our sponsor, the National Science Foundation, and in particular Janice Earle and Jane Butler Kahle for making this project possible. A number of former staff members from the Center for Science, Mathematics, and Engineering Education (the precursor to the current Center for Education) were instrumental in getting this project off the ground. Susan Loucks-Horsley was the initial project director and saw the Committee through the first year of its work. The former executive and associate executive directors, Rodger Bybee and Joan Ferrini-Mundy, conceptualized the project and provided ongoing support. Their presentations on the nation- ally developed science and mathematics standards were invaluable to the Committee. Harold Pratt, former director of the Centerâs Division on K-12 Policy and Practice, gave ongoing advice through- out the first year of the project. Kristance Coates, the former senior project assistant, provided essential support during the early months of the Committeeâs work. We are also grateful to Suzanne Woolsey, the Centerâs acting director in fall 1999, for her vigilance in seeing that this project received the support it needed during the Centerâs transition. The Committee was aided greatly by individuals who partici- ix
ACKNOWLEDGMENTS pated in our meetings and helped us understand the complex issues involved in examining how nationally developed mathematics, science, and technology standards have influenced K-12 education in the United States. The participants in the May 2000 workshop, listed in Appendix A, also very generously gave two days of their time to discuss the Committeeâs work and provided critical input for the final stages of the Committeeâs work following the work- shop. Many others have supported this project. The Mathematical Sciences Education Board and the Committee on Science Educa- tion K-12 provided oversight and advice to the Committee through- out the project. Several individuals provided advice along the way, including Richard Askey, Audrey Champagne, James J. Gallagher, Daniel Heck, Thomas Romberg, and Mark St. John. Joan Ferrini- Mundy and Harold Pratt, who were instrumental in the early stages of the project, rejoined our efforts in the final stages of the project, providing in-depth comments and advice on drafts of the report. Tina Winters and Viola Horek provided valuable support, coordi- nated many aspects of the project, and were important in seeing that the project moved forward. The Committee was resolute in working through the many difficult issues that arose until we achieved consensus on the document. Special thanks are due to Mike Knapp, whose substan- tial written contributions throughout the process led us to include him as an editor, and to a subset of the Committee that participated in numerous conference calls and attended informal meetings, e.g., at the American Educational Research Association Annual Meeting, as we continued to hammer out the details: Peg Goertz, Mike Knapp, Maria Alicia Lopez Freeman, Patrick Shields, Judy Sowder, and Elizabeth Stage. And it is certainly no exaggeration to say that without the ceaseless support from National Research Council (NRC) staff members Karen Hollweg and Gail Burrill, this docu- ment would never have seen the light of day. x
Acknowledgments This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the NRC Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report: Henry L. Alder, University of California, Davis Lynn Cornett, Southern Regional Education Board, Atlanta, GA Karen Economopolous, TERC, Cambridge, MA Manuel Gomez, University of Puerto Rico Daniel Koretz, RAND, Arlington, VA Ronald M. Latanision, Massachusetts Institute of Technology Bill Linder-Scholer, SciMathMN, Roseville, MN Roland J. Otto, E.O. Lawrence Berkeley National Laboratory, Berkeley, CA Richard Satchwell, Illinois State University Scott Stowell, Spokane Public Schools, WA Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by Michael W. Kirst, Stanford University, and R. Stephen Berry, University of Chicago. Appointed by the National Research Council, they were responsible for making certain that an independent examination of this report was carried out in accor- dance with institutional procedures and that all review comments xi
ACKNOWLEDGMENTS were carefully considered. Responsibility for the final content of this report rests entirely with the authoring committee and the institution. Iris R. Weiss, Chair Committee on Understanding the Influence of Standards in Science, Mathematics, and Technology Education xii
PREFACE tandards have served as a basis of educational reform across the S nation as educators and policy makers respond to the call for a clear definition of desired outcomes of schooling and a way to measure student success in terms of these outcomes. Public policies at the federal, state, and local levels have focused on standards as a way to raise achievement levels for all students. At the national level, content standards have been developed in nearly every academic discipline. As the standards movement has gained momentum, there has been increasing interest among educators and the public concerning the impact of standards on what happens in classrooms. In response to this interest, and as part of its mission of working for improved mathematics and science teaching and learning across the nation, the National Research Councilâs Center for Science, Mathematics, and Engineering Education (which became the Center for Educa- tion in 2000)âwith funding from the National Science Founda- tionâundertook the task of designing a framework that would aid in the design, conduct, and interpretation of research regarding the influence on student learning of the standards created at the national level in mathematics, science, and technology. In 1999, the Committee on Understanding the Influence of Standards in K- 12 Science, Mathematics, and Technology Education began its work. The charge of the Committee was to âdevelop a framework that can be used to understand the influence of science, mathemat- ics, and technology education standards on programs, policies, and xiii
PREFACE practices.â This Framework was to provide guidance for the design, conduct, and interpretation of research regarding the influences of nationally developed standards on student learning in mathematics, science, and technology education. Hence, the Committeeâs primary concern has been to develop a structure and process for understanding the influence of these educational standards on practices, programs, and policies in K-12 education. The Commit- tee was responsible to two standing boards in the Center, the Committee on K-12 Science Education (COSE K-12) and the Mathematical Sciences Education Board (MSEB), and the Committeeâs work reflects the oversight of these two boards. An overarching problem faced by the Committee was how to structure thinking about the impact of standards in a way that would consider the complex system in which the standards oper- ateâa system that consists of several layers of policy makers, various educators involved in multiple ways, and an array of players with different responsibilities for what takes place in classrooms. It was clear from the beginning that we were not to answer the question of what influence the standards have had on educational reform. It was a constant struggle, however, to keep the focus on a framework and not an analysis of where and how the standards had taken root in education systems and whether they were affecting student learning. As the Committee worked, we came to a common understand- ing of some of the issues in the daunting task of laying out a way to think about the influence of the nationally developed standards on what happens within this complex system and eventually on what happens to student learning. We agreed, for example, to focus our thinking on separate but interacting channels within the education systemâcurriculum, teacher development, and assessment and accountabilityâand to consider how reform ideas might traverse these channels to reach the classroom. At the same time, we recognized that contextual forces outside the education system, in the public and political world, directly and indirectly affect what xiv
Preface happens in the classroom, and we wanted to be sure the Framework considered those as well. We agreed to concentrate on the stan- dards created by the National Council of Teachers of Mathematics, the National Research Council, and the International Technology Education Association that focus on teaching and learning in mathematics, science, and technology. We also agreed that the focus of our work was not on the quality of the standards documents themselves but rather on their influence. We did not analyze the nationally developed documents nor make judgments about their focus and content, but concentrated on developing ways to think about whether and how these standards were being used in the system and with what results. As our work progressed, we came to see even more clearly the importance of having the document be neutral, neither advocating nor denigrating the standards and their impact. Our work was to lay out questions that should be answered in order to come to under- stand the influence of standards, regardless of whether or not the answers supported the thinking in the national documents. The Framework presented in this document is the result of our discus- sions. We eventually came to agree that the Framework would describe the system in which the standards operate, then suggest places where the effect of the standards might be observed in this system, which became the organizing structure of the document. We have drawn on a broad body of research related to various aspects of the task, and have also relied heavily on the practical experience of the Committee members and outside experts in designing the Framework. The Committee produced a background paper for its work, and in spring 2000, convened a group of educa- tors and researchers in a workshop to discuss and critique the paper. The Committee used the insights and input from workshop participants to help define the Framework and to stimulate ideas regarding its future uses. Investigating the Influence of Standards is aimed at researchers xv
PREFACE and those who use research in their work, including federal, state, and local policy makers. It is intended to provide a lens for consid- ering the national standards and the role such standards play in improving mathematics, science, and technology education. This book is not intended to be a simple template prescribing a solution but rather lays out the major issues involved in considering the effects of standards in improving student learning, offering ques- tions that should be asked when considering research about the standards. The document can be used to understand particular studies, to identify what the studies do and do not address, and to help consumers interpret their claims. The document is also designed to spur additional research in areas where answers to the questions are not well-formed. While the focus of this document is the nationally developed standards in mathematics, science, and technology, we believe the Framework is general enough to serve as a guide for those at state and district levels and in other disci- plines to analyze the impact of their own standards. Iris R. Weiss, Chair Committee on Understanding the Influence of Standards in Science, Mathematics, and Technology Education xvi
CONTENTS Executive Summary 1 1 Introduction 11 2 Standards for Mathematics, Science, and Technology Education 17 The Context in Which Standards Evolved, 18 Developing National Standards in Mathematics, 20 Developing National Standards in Science, 22 Developing Standards for Technological Literacy, 24 Commonalities Across the Mathematics, Science, and Technology Standards, 24 3 A Framework for Investigating the Influence of Education Standards 29 An Overview of the U.S. Education System, 29 One View of the Education System, 31 The Elements of the Framework, 33 xvii
CONTENTS 4 Curriculum as a Channel of Influence: What Shapes What Is Taught to Whom? 39 Curriculum in the Education System, 39 Implications of Policy Decisions, 39 Development of Instructional Materials and Programs, 41 Selection Mechanisms, 42 How Standards Might Influence Curriculum, 44 The Curriculum Channel and Nationally Developed Standards, 46 5 Teacher Development as a Channel of Influence: How Do Teachers Learn What and How to Teach? 49 Teacher Development in the Education System, 50 Teacher Preparation, 50 Certification and Licensure, 52 Ongoing Professional Development, 53 How Standards Might Influence Teacher Development, 55 The Teacher Development Channel and Nationally Developed Standards, 57 6 Assessment and Accountability: What Kinds of Assessment Are Used and for What Purposes? 59 Assessment and Accountability in the Education System, 60 Accountability, 60 Classroom Assessment, 62 State and District Assessment and Accountability Policies, 63 College Entrance and Placement Practices, 64 Impact and Unintended Consequences of Assessment, 65 xviii
Contents How Standards Might Influence Assessment and Accountability , 67 The Assessment and Accountability Channel and Nationally Developed Standards, 68 7 Contextual Forces that Influence the Education System 71 Who Affects the Education System from âOutsideâ?, 71 Outside Forces Affecting Decision Making, 73 Outside Forces Affecting Components Within the Education System, 74 How Can Nationally Developed Standards Influence Those âOutsideâ the System? How Might They, in Turn, Influence the Education System?, 75 Contextual Forces and Nationally Developed Standards, 76 8 Using the Framework 79 The Framework in Review, 79 How the Framework Can Be Used, 83 Situating Current Studies, 86 Examining Claims and Inferences Reported in Current Studies, 88 Generating Questions and Hypotheses for Future Investigations, 92 Aspirations for Framework-Driven Research on Nationally Developed Standards, 94 References 97 xix
CONTENTS Appendixes A Participants at the Workshop on Understanding the Influence of Standards in K-12 Science, Mathematics, and Technology Education 109 B A Framework for Investigating the Influence of Nationally Developed Standards for Mathematics, Science, and Technology Education 113 C Biographical Sketches 115 Index 125 FIGURES, BOXES, AND TABLE Figures 1-1 The Black Box, 12 3-1 The Layers of Education Governance and Channels Through Which Reform Might Flow, 32 3-2 A Conceptual Map for Investigating the Influence of Nationally Developed Standards for Mathematics, Science, and Technology Education, 34 3-3 A Set of Guiding Questions for Investigating the Influences of Nationally Developed Standards, 35 8-1 A Framework for Investigating the Influence of Nationally Developed Standards for Mathematics, Science, and Technology Education, 80 8-2 Parts of the Framework Addressed in the Briars and Resnick Study, 87 xx
Contents Boxes 2-1 Basic Principles and Features of Principles and Standards for School Mathematics, 21 2-2 Basic Principles and Features of National Science Education Standards, 23 2-3 Basic Principles and Features of Standards for Technological Literacy: Content for the Study of Technology, 25 Table 8-1 Hypothetical Studies That Address One or More of the Framework Questions, 84 xxi