Appendix A
Workshop Agendas

Best Practices for State Assessment Systems

Workshop 1

December 10-11, 2009


National Academy of Sciences

2100 C Street, NW

Washington, DC

Auditorium


Thursday, December 10


OPEN

9:15-9:45

Welcome

Stuart Elliott, Director, Board on Testing and Assessment

Judith Rizzo, Executive Director and CEO, James B. Hunt, Jr. Institute for Educational Leadership and Policy

 

Overview of Workshop Goals

Diana Pullin, Chair, Committee on Best Practices for State Assessment Systems

 

  • Précis of previous workshop series and report

  • Overview of the goals and plans for the current workshop series

  • Introduction of the idea of innovative assessment

  • Discussion of the current status of the common standards movement

9:45-11:30

Session I. Examining the Status Quo: What Are the Benefits and Limitations of the Current Approaches to Assessment in This Country?

Moderators: Diana Pullin, Dirk Mattson



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 121
Appendix A Workshop Agendas Best Practices for State Assessment Systems Workshop 1 December 10-11, 2009 National Academy of Sciences 2100 C Street, NW Washington, DC Auditorium Thursday, December 10 OPEN 9:15-9:45 Welcome Stuart Elliott, Director, Board on Testing and Assessment Judith Rizzo, Executive Director and CEO, James B. Hunt, Jr. Institute for Educational Leadership and Policy Overview of Workshop goals Diana Pullin, Chair, Committee on Best Practices for State Assessment Systems • Précis of previous workshop series and report • Overview of the goals and plans for the current workshop series • Introduction of the idea of innovative assessment • Discussion of the current status of the common standards movement 9:45-11:30 Session I. Examining the Status Quo: What Are the Benefits and Limitations of the Current Approaches to Assessment in This Country? Moderators: Diana Pullin, Dirk Mattson 

OCR for page 121
 STATE ASSESSMENT SYSTEMS (9:45-10:15) Overview of Current Assessment Practices This session will provide a review of the current test-based accountability system, the goals and purposes it has developed to serve, and its strengths and limitations. Presenter: Margaret Goertz, University of Pennsylvania (10:15-10:45) Changes in Assessments and Assessment Systems Over the Past Decade This session will review the ways assessments and approaches to assessment have changed over the past decade, including changes in item types, uses of local and interim assessments, and advancements in assessment of special populations. Presenter: Scott Marion, National Center for Improvement of Educational Assessment (NCIEA) (10:45-11:15) Synthesis of Key Ideas Discussant: Joan Herman, National Center for Research on Evaluation, Standards, & Student Testing (CRESST) (11:15-11:30) Focused Discussion Moderators lead focused discussion with presenters and audience members. 11:30-12:15 Working Lunch 12:15-3:45 Session II. Changing the Status Quo Moderators: Joan Herman, Rebecca Maynard (12:15-1:00) Developing Standards That Lead to Better Instruction and Learning This session will discuss ways to specify standards so that they (1) more accurately delineate the skills and knowledge to be learned and (2) can be more accurately and readily translated into instruction and assessment. Examples will be drawn from the draft common core standards. Presenters: Joe Krajcik and Shawn Stevens, University of Michigan (1:00-1:45) Developing Assessment Tasks That Lead to Better Instruction and Learning This session will explore ways to use more elaborated standards to develop assessment tasks that accurately measure the intended

OCR for page 121
 APPENDIX A skill and knowledge, with a particular focus on ways to ensure that assessments measure higher-order, critical thinking skills using a variety of item types. Presenter: Mark Wilson, University of California, Berkeley (1:45-2:30) Technical Challenges of Implementing Innovative Assessments This session will explore the technical challenges associated with developing more innovative assessment tasks that measure challenging content and skills, tradeoffs associated with these kinds of assessments/tasks, and ways that the information gathered from innovative assessments might be used to support better decision making about students and instruction. Presenter: Stephen Lazer, Educational Testing Service (ETS) 2:30-2:45 Break 2:45-3:15 Synthesis of Key Ideas Discussant: Scott Marion, NCIEA 3:15-3:45 Focused Discussion Moderators lead focused discussion with presenters and audience members. 3:45-5:00 Session III.A What Is the Status of Innovative Assessment? Moderators: Diana Pullin, Mark Wilson (3:45-4:30) Lessons from the Past and Current Efforts This session will provide an overview of the experiences of pioneers in the area of innovative assessment, such as programs developed for Kentucky (KIRIS), Maryland (MSPAP), Vermont (Portfolio Assessment Program), and California (CLAS performance assessment) which are no longer in operation. Examples from currently operational assessment programs, international assessments, and in fields other than K-12 education will also be discussed. Presenters: Brian Stecher and Laura Hamilton, RAND (4:30-5:00) Focused Discussion Moderators lead focused discussion with presenters and audience members.

OCR for page 121
 STATE ASSESSMENT SYSTEMS 5:00 Presentation by Lauren Resnick, Learning Research and Development Center, University of Pittsburgh 5:30 Adjourn Workshop Reception 6:00 Working group Dinner (in Lecture Room) Friday, December 11 OPEN 8:30-10:30 Session III.B What Is the Status of Innovative Assessment? (cont.) Moderators: Diana Pullin, Mark Wilson (8:30-9:30) Panel Discussion: Political Considerations This session will explore the political/practitioner perspective on the pioneer program discussed in Part A. Panelists representing several of the programs will address the following questions: • What was the motivation for the assessment? Why was it considered? Who wanted it? Who wasn’t in favor of it? • What was involved in getting the assessment adopted? What, if any, obstacles were encountered? How were they overcome? • What was involved in developing the assessment? What, if any, obstacles were encountered? How were they overcome? • What issues were encountered with implementation of the assessment? What, if any, obstacles were encountered? How were they overcome? • What were primary reasons for the demise of the program? Panelists: Steve Ferrara (MSPAP), Brian Gong (KIRIS), and Dirk Mattson (Minnesota) (9:30-10:00) Synthesis of Key Ideas Discussant: Lorraine McDonnell, University of California, Santa Barbara

OCR for page 121
 APPENDIX A (10:00-10:30) Focused Discussion Moderators lead focused discussion with presenters and audience members. 10:30-10:45 Break 10:45-2:30 Session Iv. Exploring the Opportunities Moderators: Rebecca Maynard, Dirk Mattson (10:45-11:30) What Opportunities Does the Common Standards Movement Offer for Improving Assessment? This session will explore the opportunities the common standards movement might offer for moving to more innovative assessments that assess challenging content and also give more information to teachers and local decision makers. The presentation will address technical issues and potential benefits of collaboration across states, drawing on examples from the experiences of Vermont, New Hampshire, Rhode Island, and Maine (New England Common Assessment Program) Presenter: Laurie Wise, HumRRO 11:30-12:30 Working Lunch 12:30-2:15 Session Iv. Exploring the Opportunities (cont.) (12:30-1:15) Using Common Standards to Enable Cross-State Comparisons This session will focus on the elements that would need to be in place in order for test results to be compared across states, including issues associated with adding state-specific items. The presentation will address the inferences that policy makers and test users might want to make and what is required to support each kind of inference. Presenter: Ron Hambleton, University of Massachusetts, Amherst (1:15-1:45) Synthesis of Key Ideas Discussant: Rebecca Zwick, ETS and University of California, Santa Barbara (1:45-2:15) Focused Discussion Moderators lead focused discussion with presenters and audience members.

OCR for page 121
 STATE ASSESSMENT SYSTEMS 2:15-2:30 Break 2:30-4:00 Session v. Setting Research Priorities Moderators: Diana Pullin, Scott Marion (2:30-3:00) Research Priorities The U.S. Department of Education has set aside $350 million for developing tests to measure common standards. This panel will listen to the workshop discussions and consider the implications for research. The presenter and discussants will address the following questions: • Given the issues raised during the workshop, what are realistic priorities for research? • What projects/efforts are most in need of research? • How would you proportionally allocate the funding? Presenter: Lorrie Shepard, University of Colorado (3:00-3:45) Responses Discussants: Laurie Wise, HumRRO; Joan Herman, CRESST; Rebecca Maynard, University of Pennsylvania (3:45-4:15) Focused Discussion Moderators lead focused discussion with presenters and audience members. 4:15 Closing Remarks, Adjourn Diana Pullin, Chair

OCR for page 121
 APPENDIX A Best Practices For State Assessment Systems Workshop 2 April 6-7, 2010 National Academy of Sciences 2100 C Street, NW Washington, DC Auditorium Tuesday, April 6 CLOSED 8:00 Working breakfast for committee Discussion of agenda 8:30 Discussion of plans for the workshop and moderator assignments 9:00 Break to transition to open session OPEN 8:45-9:00 guests arrive, register 9:15-9:45 Opening Remarks • Stuart Elliott, Director, Board on Testing and Assessment • Judith Rizzo, Executive Director and CEO, James B. Hunt Jr. Institute for Educational Leadership and Policy (9:30) goals for the Workshop • Diana Pullin, Boston College, Workshop Steering Committee Chair 9:45-2:45 Session I. Developing Coherent Systems, Implementing Innovative Assessments Issues to Discuss • What does it mean to have a coherent system of assessments, with vertical, horizontal, and developmental coherence? Why is a coherent system important? What advantages does it offer?

OCR for page 121
 STATE ASSESSMENT SYSTEMS • What are the different purposes of assessment and what assessment strategies best suit each purpose? How can the information provided by different forms of assessment be used to guide instruction, improve student learning, and evaluate effectiveness of teachers and schools for accountability purposes? • What is involved in moving to a coherent system of assessment that incorporates multiple assessment types? What examples of such systems exist (nationally and internationally)? • How can assessment of higher level thinking skills and deeper understandings be incorporated into a coherent system of assessments? What examples are currently being tried or considered? (9:45-10:30) Coherent System of Assessments Moderators: Diana Pullin Rebecca Maynard, University of Pennsylvania, Workshop Steering Committee Presenter: Joan Herman, CRESST, Workshop Steering Committee (10:30-10:45) Break (10:45-11:30) Moderated Discussion: Policy Perspective Panelists: Roy Romer, College Board Ed Roeber, Michigan State University (11:30-12:00) Audience Discussion 12:00-1:00 Working Lunch 1:00-2:30 Reconvene to Continue Session I

OCR for page 121
 APPENDIX A (1:00-2:00) Moderated Discussion: Research and Practice Perspective Moderators: Diana Pullin Dirk Mattson, Minnesota Department of Education Panelists: Linda Darling-Hammond, Stanford Tony Alpert, Oregon Department of Education Marc Tucker, National Center for Education and the Economy (2:00-2:30) Audience Discussion 2:30-2:45 Break 2:45-5:00 Session II. Issues to Consider with Innovative Assessments Issues to Address • Innovative assessments pose challenges with respect to item development, administration, scoring, equating, and other technical issues. What are these challenges and how can they be addressed? How can we ensure that scores obtained from innovative assessments are reliable and valid? • What issues need to be considered in using innovative assessment approaches with special populations? • How can we ensure that these students have the opportunity to learn the material/skills needed for assessments that measure deeper understandings and higher order thinking skills? (2:45-3:15) Ensuring That Innovative Assessments Provide Reliable and valid Information Moderators: Scott Marion, National Center for Improvement of Educational Assessment, Workshop Steering Committee Mark Wilson, UC Berkeley, Workshop Steering Committee Presenter: Rebecca Zwick, UC Santa Barbara and ETS

OCR for page 121
0 STATE ASSESSMENT SYSTEMS (3:15-4:00) Moderated Discussion: Special Populations Perspectives Panelists: Robert Linquanti, West Ed Martha Thurlow, National Center on Education Outcomes (4:00-4:45) Moderated Discussion: Policy and Practice Perspective Panelists: Joe Willhoft, Washington Department of Education Deborah Sigman, California Department of Education (4:45-5:00) Audience Discussion 5:00-5:15 Closing Remarks for Day 1 Gene Wilhoit, Council of Chief State School Officers Diana Pullin 5:15 End of Formal Agenda for Day 1 Reception 6:00 Working group Dinner (in Lecture Room and great Hall) Wednesday, April 7 8:30-11:45 Session III. Strategies for Using Results from a Coherent System to Inform Instructional Decision Making Issues to Address • How can assessment best be used to guide instruction? • What changes can be made to content standards, assessment tasks, and reports of assessment results to make them more useful for instruction? • What role can innovative assessment approaches play in guiding instruction? • What structures and supports need to be in place? What training do teachers need in order to effectively use assessment information?

OCR for page 121
 APPENDIX A (8:30-9:15) Strategies for Using Assessment to guide Instruction Moderators: Joan Herman Mark Wilson Presenter: Linda Darling-Hammond, Stanford (9:15-10:15) Moderated Discussion: Researcher Perspective Panelists: Margaret Heritage, CRESST Karin Hess, National Center for the Improvement of Educational Assessment 10:15-10:30 Break (10:30-11:15) Moderated Discussion: Policy Maker, Practitioner Perspectives Panelists: Teri Siskind, South Carolina Department of Education Peg Cagle, LA Unified School District, California Teachers Advisory Council (11:15-11:45) Audience Discussion 11:45-12:45 Working Lunch 12:45-2:15 Session Iv: The Role of Technology • What role can technology play in the implementation of innovative assessment strategies? • What role can technology play in the scoring and reporting of innovative assessments? • What role can technology play in making assessments and assessment results more useful for instruction?

OCR for page 121
 STATE ASSESSMENT SYSTEMS (12:45-1:15) Technology and Innovative Assessment Moderators: Rebecca Maynard Dirk Mattson Presenter: Randy Bennett, ETS (1:15-2:00) Policy maker, Practitioner Perspective Panelists: Wendy Pickett, Delaware Department of Education Tony Alpert, Oregon Department of Education (2:00-2:15) Audience Discussion 2:15-2:30 Break 2:30-4:00 Session v. Synthesis and Extension of Ideas (2:30-3:15) Aggregating Information from Different Sources Moderators: Diana Pullin Scott Marion Presenter: Laurie Wise, HumRRO The concept of “through-course” or “distributed summative assessment components” (e.g., assessment evidence collected over multiple points of time) has been discussed at several of the U.S. Department of Education-sponsored assessment hearings. This presentation will discuss how multiple types of assessment evidence can be used and aggregated to inform summative judgments. The presentation will address using aggregated information to evaluate students, teachers, and schools. (3:15-4:00) Moderated Discussion with full steering committee 4:00 Closing Remarks Diana Pullin 4:15 Adjourn