|
Items in cart [0] |
Questions? Call 888-624-8373 |
| Copyright © 2010. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page R1
Assessing Accomplished Teaching: Advanced-Level Certification Programs
Assessing Accomplished Teaching
Advanced-Level Certification Programs
Committee on Evaluation of Teacher Certification by the National Board for Professional Teaching Standards
Milton D. Hakel, Judith Anderson Koenig, and Stuart W. Elliott, Editors
Board on Testing and Assessment
Center for Education
Division of Behavioral and Social Sciences and Education
NATIONAL RESEARCH COUNCIL OF THE NATIONAL ACADEMIES
THE NATIONAL ACADEMIES PRESS
Washington, D.C.
www.nap.edu
OCR for page R2
Assessing Accomplished Teaching: Advanced-Level Certification Programs
THE NATIONAL ACADEMIES PRESS
500 Fifth Street, N.W. Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
This study was supported by Contract No. ED-04-CO-0139 between the National Academy of Sciences and the U.S. Department of Education. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the organizations or agencies that provided support for the project.
Library of Congress Cataloging-in-Publication Data
Assessing accomplished teaching : advanced-level certification programs : Committee on Evaluation of Teacher Certification by the National Board for Professional Teaching Standards / Milton D. Hakel, Judith Anderson Koenig, and Stuart W. Elliott, editors.
p. cm.
ISBN 978-0-309-12118-7 (pbk.) — ISBN 978-0-309-12119-4 (pdf) 1. Teachers—Certification—United States. I. Hakel, Milton D. II. Koenig, Judith A. III. Elliott, Stuart W. IV. National Board for Professional Teaching Standards (U.S.)
LB1771.A77 2008
371.120973—dc22
2008026490
Additional copies of this report are available from the
National Academies Press,
500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu.
Copyright 2008 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
Suggested citation: National Research Council. (2008). Assessing Accomplished Teaching: Advanced-Level Certification Programs. Committee on Evaluation of Teacher Certification by the National Board for Professional Teaching Standards. Milton D. Hakel, Judith Anderson Koenig, and Stuart W. Elliott, editors. Board on Testing and Assessment, Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
OCR for page R3
Assessing Accomplished Teaching: Advanced-Level Certification Programs
THE NATIONAL ACADEMIES
Advisers to the Nation on Science, Engineering, and Medicine
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council.
www.national-academies.org
OCR for page R4
Assessing Accomplished Teaching: Advanced-Level Certification Programs
This page intentionally left blank.
OCR for page R5
Assessing Accomplished Teaching: Advanced-Level Certification Programs
COMMITTEE ON EVALUATION OF TEACHER CERTIFICATION BY THE NATIONAL BOARD FOR PROFESSIONAL TEACHING STANDARDS
MILTON D. HAKEL (Chair),
Department of Psychology, Bowling Green State University
JULIAN BETTS,
Department of Economics, University of California, San Diego
MARK DYNARSKI,
Mathematica Policy Research, Inc., Princeton, New Jersey
ADAM GAMORAN,
Department of Sociology, University of Wisconsin, Madison
JANE HANNAWAY,
The Urban Institute, Washington, DC
RICHARD INGERSOLL,
Graduate School of Education, University of Pennsylvania
MICHAEL T. KANE,
National Conference of Bar Examiners, Madison, Wisconsin
DEIRDRE J. KNAPP,
Human Resources Research Organization, Alexandria, Virginia
SUSANNA LOEB,
School of Education, Stanford University
JAMES (TORCH) H. LYTLE,
Graduate School of Education, University of Pennsylvania
C. FORD MORISHITA,
Clackamas High School, Clackamas, Oregon
LYNN W. PAINE,
College of Education, Michigan State University
NEIL J. SMELSER (Emeritus),
University of California, Berkeley
BRIAN STECHER,
RAND Corporation, Santa Monica, California
ANA MARIA VILLEGAS,
College of Education and Human Services, Montclair State University
DOROTHY Y. WHITE,
Department of Mathematics and Science Education, University of Georgia
KAREN K. WIXSON,
School of Education, University of Michigan
JUDITH A. KOENIG, Study Director
STUART W. ELLIOTT, Senior Program Officer
ALIX BEATTY, Senior Program Officer
VIOLA HOREK, Manager, Program Operations
TERESIA WILMORE, Senior Program Assistant
OCR for page R6
Assessing Accomplished Teaching: Advanced-Level Certification Programs
BOARD ON TESTING AND ASSESSMENT
LYLE F. BACHMAN,
Department of Applied Linguistics, University of California, Los Angeles
STEPHEN B. DUNBAR,
College of Education, University of Iowa
DAVID J. FRANCIS,
Department of Psychology, University of Houston
ANDREW J. HARTMAN, Independent Consultant,
Denver, Colorado
MICHAEL NETTLES,
Educational Testing Service, Princeton, New Jersey
JAMES W. PELLEGRINO,
Department of Psychology, University of Illinois, Chicago
DIANA C. PULLIN,
School of Education, Boston College
STUART W. ELLIOTT, Director
OCR for page R7
Assessing Accomplished Teaching: Advanced-Level Certification Programs
Acknowledgments
The Committee on Evaluation of Teacher Certification by the National Board for Professional Teaching Standards (NBPTS) was formed in response to legislation passed by the U.S. Congress asking the National Academies to develop a framework for evaluating programs that offer advanced-level certification to teachers and to apply that framework in an evaluation of the impacts of the NBPTS. The committee began its work in September 2005 and, over the course of the next 30 months, held six meetings, commissioned numerous papers and analyses, and collected a variety of information as part of its evaluation. The committee’s work benefited tremendously from the contributions of many people, and the committee is grateful for their assistance and support.
The work was overseen by the U.S. Department of Education with management handled by Susan Sanchez. We thank Susan for her expert guidance and quick response to our many questions.
Over the course of the project, we made many requests of NBPTS staff, including numerous requests for information, documentation, research reports, and data files; visits to their office and meetings with their staff; and presentations at our committee meetings. We extend our heartfelt thanks to Mary Dilworth for her diligence and patience in responding to our extensive inquiries. Mary gave generously of her time, and we thank her for all the information she provided in response to the committee’s questions. We also thank NBPTS president Joseph Aguerrebere and staff members Lillie Saunders and Joan Auchter for the valuable information they provided.
OCR for page R8
Assessing Accomplished Teaching: Advanced-Level Certification Programs
During the course of this evaluation, the Educational Testing Service (ETS) served as the test development contractor to NBPTS, and we wish to acknowledge the assistance of several ETS staff members. We are grateful to Mari Pearlman for recapturing for us the history of the development of the assessment program and to Drew Gitomer for providing his review of research on the NBPTS program. We especially thank Steve Schreiner for providing an overview of the assessment scoring process and for guiding us through a review of sample portfolio materials and videotapes submitted by teachers. This review contributed greatly to our understanding of the assessment and the requirements that teachers must meet to earn board certification.
Our evaluation drew on work conducted for us by several researchers. Our initial review of the research base revealed a large number of studies focused on the impact of board-certified teachers on their students’ achievement. The findings from these studies presented a complex set of somewhat conflicting results. We thank Henry Braun and Paul Holland, both at ETS, for their insights about these studies, which helped us to sort out details of research methodologies and statistical analyses. We decided to conduct a full-scale review of these studies combined with work to identify and conduct additional analyses to help resolve the differing findings. The researchers who assisted us in this endeavor greatly contributed to our evaluation, and we are indebted to each of them, including Doug Harris at the University of Wisconsin–Madison, Helen Ladd at Duke University, Daniel McCaffrey at the RAND Corporation, Steven Rivkin at Amherst College, and Tim Sass at Florida State University. We are especially grateful to Tim Sass for the many additional analyses he conducted for us. We also thank Jon Fullerton and Tom Kane at Harvard University, Steve Raudenbush at the University of Chicago, and Bill Sanders at the SAS Institute, Inc., for their presentations at our third committee meeting and their insights about this body of work.
We also commissioned a psychometric review of the NBPTS assessments. This was an extensive task, given that NBPTS awards certification in 25 areas, each using different assessments. We are indebted to Teresa Russell at the Human Resources Research Organization (HumRRO), who led this work for us. Her thorough and painstaking efforts to review a multitude of documents and summarize the psychometric characteristics of the assessments were a tremendous resource to the committee.
In addition, we commissioned analyses of NBPTS participants and comparisons of their characteristics with those of nonparticipating teachers. David Perda, a doctoral student at the University of Pennsylvania, assisted us with these analyses. We sincerely appreciate his thorough and careful
OCR for page R9
Assessing Accomplished Teaching: Advanced-Level Certification Programs
analyses and are particularly grateful for his responsiveness to our numerous follow-up requests for additional analyses.
To learn more about the history of the NBPTS program, we conducted interviews of a number of key people involved with its early development. We are very grateful for the time these individuals spent with us and the wealth of information they provided. We thank former NBPTS staff members Joan Baratz-Snowden at the American Federation of Teachers (former vice president of assessment and research); Chuck Cascio at ETS (former director of test development); Ann Harman at Harman and Associates (former director of research); Jim Kelly, retired, who served as the first president of NBPTS; and David Mandel at Carnegie-IAS Commission on Mathematics and Science Education (former vice president for policy development at NBPTS). We are also grateful for the information provided by Sally Mernissi (former vice president and corporate secretary), who died in January 2006. We also thank the following individuals who assisted the NBPTS staff with development of the program: Lloyd Bond at the Carnegie Foundation (former director of the NBPTS Technical Analysis Group); Emerson Elliott at the National Council for Accreditation of Teacher Education; Mary Futrell at George Washington University (former president of the National Education Association and member of the NBPTS board of directors); Lee Shulman at the Carnegie Foundation (former consultant to the NBPTS board of directors); Gary Sykes at Michigan State University (former consultant to the NBPTS board of directors); and Suzanne Wilson at Michigan State University (former consultant to the NBPTS board of directors). We also thank Joshua Boots at the American Board for Certification of Teacher Excellence (ABCTE) and Kathy Madigan, formerly at the ABCTE, for the information they provided about their assessment program.
We sought to gather information from teachers and teacher educators with regard to their experiences with and perceptions of the NBPTS program. Members of the Teacher Advisory Council of the National Research Council (NRC) spoke with us on numerous occasions. In addition four teachers and teacher educators attended our third meeting: Sara Eisenhardt with the Cincinnati school system, Maxine Freund and Mary Futrell at George Washington University, and Carol Matern with the Indianapolis public schools and Indiana University–Purdue University, Indianapolis. We thank these individuals for their insightful comments about the program.
During the course of this project, we spoke with and heard presentations from numerous individuals who conducted research on the NBPTS program. We are particularly grateful to the following researchers for their
OCR for page R10
Assessing Accomplished Teaching: Advanced-Level Certification Programs
responsiveness to our questions and the time they took to help us better understand their findings: Linda Cavaluzzo at CNA Corporation, Carol Cohen at the Finance Project, Dan Goldhaber at the University of Washington, Michael Hansen at the University of Washington, Dan Humphrey at SRI International, Julia Koppich at J. Koppich Associates, David Lustick at Michigan State University, Jennifer King Rice at the University of Maryland, and Jannese Woodward Moore at East Tennessee State University.
Senior staff members of the NRC’s Division of Behavioral and Social Sciences and Education helped the committee move this project forward. Michael Feuer, executive director, enthusiastically backed the project and lent his wisdom and advice at key stages. Patricia Morison, associate executive director and acting director of the Center for Education, provided sage advice throughout this project. Eugenia Grohman, associate executive director, lent her deep knowledge and experience with NRC procedures and the committee process. Christine McShane, senior editor, provided expert editing assistance. Kirsten Sampson Snyder, senior report review officer, ably guided the report through the NRC review process.
Special thanks are due to Teresia Wilmore, senior project assistant, for her masterful handling of the logistical aspects of this project. Teresia very capably managed all of the committee meetings and is now pursuing a graduate degree in nursing—we wish her great success. We also thank Viola Horek and Dorothy Majewski, who ably stepped in to assist at various stages of the project. We are grateful to Lisa Alston, who provided support throughout the project and to Monica Ulewicz for her assistance with the literature reviews during the early stage of this project.
On behalf of the committee and as its chair, I wish to recognize three members of the NRC staff who did everything needed to ensure the quality and timeliness of this project. First, we are grateful to senior program officer, Alix Beatty, for her expert research, writing, and critical thinking skills. Alix provided initial drafts for several portions of the report, a service that was invaluable because it immediately propelled the committee’s deliberations. Second, Stuart Elliott, director of the Board on Testing and Assessment, provided calm and steady input at every stage of our deliberations. Stuart was particularly adept at helping us as individuals to understand ideas from others’ disciplines and to establish shared interpretations of particular research findings.
Third, and deserving a solo bow, is Judith Koenig, our study director. Beginning with preparation of the committee proposal and then the vetting of nominees for membership on the committee through to the completion and publication of this report, Judy did it all. Although that is probably true for every NRC study director, what stands out about Judy are three attributes: she is very smart and learns quickly, she always listens and hears
OCR for page R11
Assessing Accomplished Teaching: Advanced-Level Certification Programs
both what is said and what is not said, and she takes the initiative collaboratively. While the committee could have conducted the evaluation and issued its report without her, our tasks would not have been completed as rapidly nor as well. Thank you, Judy, for your graceful leadership at the center of our efforts.
Finally, I wish to thank the committee members for their dedication and outstanding contributions to this study. They drafted text, prepared background materials, reviewed numerous versions of this report, and gave generously of their time. In its first meeting, the committee adopted a pattern of collaborative inquiry and meticulous analysis that endured through the final signoff. Despite marked differences in our fields of expertise and differing viewpoints about what constitutes noteworthy evidence, the entire committee remained focused and engaged throughout. I am deeply grateful to each member.
This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the NRC’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.
We wish to thank the following individuals for their review of this report: Lisa Barrow, Economic Research, Federal Reserve Bank of Chicago; Stephen B. Dunbar, College of Education, University of Iowa; David N. Figlio, Department of Economics, University of Florida; Robert E. Floden, Institute for Research on Teaching and Learning, College of Education, Michigan State University; Robert M. Hauser, Center for Demography of Health and Aging, University of Wisconsin–Madison; David H. Monk, Dean’s Office, College of Education, Penn State University; Gary J. Natriello, Teachers College, Columbia University; John J. Norcini, Office of the President, Foundation for Advancement of International Medical Education and Research, Philadelphia; Ruth J. Palmer, Educational Psychology, School of Education, College of New Jersey; and Mark R. Wilson, Graduate School of Education, University of California, Berkeley.
Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations nor did they see the final draft of the report before its release. The review of this report was overseen by Lauress Wise, HumRRO, Monterey, California, and Charles E. Phelps, university professor and provost emeritus, University of Rochester. Appointed by the NRC, they were
OCR for page R12
Assessing Accomplished Teaching: Advanced-Level Certification Programs
responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report, however, rests entirely with the authoring committee and the institution.
Milton D. Hakel, Chair
Committee on Evaluation of Teacher Certification by the National Board for Professional Teaching Standards
OCR for page R13
Assessing Accomplished Teaching: Advanced-Level Certification Programs
Contents
Summary
1
1
Introduction
15
2
The Evaluation Framework and Collection of Data
20
3
The Historical Context and Overview of the National Board
38
4
The Assessment Program
59
5
The Psychometric Quality of the Assessments
79
6
Teacher Participation in the Program
119
7
The Impact of Board-Certified Teachers on Student Outcomes
154
8
The Effects of the Certification Process on Practice
182
9
The Impact of Certification on Teachers’ Career Paths
198
10
The Effects of Certification on the Education System
210
OCR for page R14
Assessing Accomplished Teaching: Advanced-Level Certification Programs
11
The Cost-Effectiveness of Certification as a Means of Improving Teacher Quality
223
12
Overall Evaluation
248
References
264
Appendixes
A
Reviews of Studies That Provided Evidence for the Evaluation
275
B
Biographical Sketches of Committee Members and Staff
315
