4
The Assessment Program

An important responsibility of the National Board for Professional Teaching Standards (NBPTS) is to operate its assessment program, although it also provides a number of services for board-certified teachers and prospective candidates. To learn about the board, its services, and the assessment process, we reviewed the information available on its web page (http://www.nbpts.org), information provided to candidates, and technical documentation about the assessment. We also visited the board’s office, met with staff members, and arranged for briefings on specific topics, such as an orientation on what candidates must do as part of the certification process, an overview of the scoring process, and a review of studies that the board has funded. To learn more about the assessment and the scoring process, we arranged for a presentation by Steve Schreiner, the Educational Testing Service (ETS) staff member who leads the NBPTS portfolio scoring process, at our fifth committee meeting. At that time, we reviewed samples of the materials that teachers submit as part of their portfolio and viewed two sample videotapes.

In this chapter, we describe the board as an organization, the experience teachers undergo as they pursue board certification, the content standards that are assessed, the assessment exercises, and the scoring. We conclude the chapter with our observations from reviewing the sample portfolio materials and videotapes. The overview in this chapter is intended to provide a framework for the detailed psychometric analysis described in Chapter 5. The board now offers advanced-level certification in 25 areas. In this chapter and Chapter 5, we focus on two certificate areas, one generalist



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 59
4 The Assessment Program An important responsibility of the National Board for Professional Teaching Standards (NBPTS) is to operate its assessment program, al- though it also provides a number of services for board-certified teachers and prospective candidates. To learn about the board, its services, and the assessment process, we reviewed the information available on its web page (http://www.nbpts.org), information provided to candidates, and technical documentation about the assessment. We also visited the board’s office, met with staff members, and arranged for briefings on specific topics, such as an orientation on what candidates must do as part of the certification process, an overview of the scoring process, and a review of studies that the board has funded. To learn more about the assessment and the scoring process, we arranged for a presentation by Steve Schreiner, the Educational Testing Service (ETS) staff member who leads the NBPTS portfolio scoring process, at our fifth committee meeting. At that time, we reviewed samples of the materials that teachers submit as part of their portfolio and viewed two sample videotapes. In this chapter, we describe the board as an organization, the experience teachers undergo as they pursue board certification, the content standards that are assessed, the assessment exercises, and the scoring. We conclude the chapter with our observations from reviewing the sample portfolio ma- terials and videotapes. The overview in this chapter is intended to provide a framework for the detailed psychometric analysis described in Chapter 5. The board now offers advanced-level certification in 25 areas. In this chapter and Chapter 5, we focus on two certificate areas, one generalist 

OCR for page 59
0 ASSESSING ACCOMPLISHED TEACHING assessment (middle childhood generalist) and one subject-area assessment (middle childhood through early adolescent mathematics), for detailed ex- amples and analysis. Our rationale for selecting these two certificate areas is explained in Chapter 5. THE ORGANIZATION The NBPTS is an independent, nonprofit, nonpartisan, and nongovern- mental organization with a staff of approximately 60 located in Arlington, Virginia.1 A 27-member board of directors, of whom 13 are board-certified teachers, oversees its work. During the early years of the board’s work, foundations, corporations, and the federal government provided the bulk of its financial support; applicant fees now cover most of the board’s operat- ing costs. A major responsibility of the NBPTS is the management and opera- tion of the assessment program through which teachers earn certification. Running this program entails developing and updating standards for the 25 areas in which certification is offered and, with the assistance of the National Board’s contractor, the ETS, developing, administering, and scor- ing the assessments. Five standing committees (drawn from the board of directors) assist the board with its work. In addition to committees that oversee finances and other standard responsibilities, the board has an Education Committee that is responsible for identifying education reforms in which the board should involve itself and a Certification Council that develops policies related to areas in which certification is offered, standards, methods of certification, and other issues. Several groups drawn from outside the board’s member- ship also offer support, including the Assessment Certification Advisory Panel, which advises the board on the technical aspects of its assessment; the Visiting Panel on NBPTS Research; the National Board-Certified Teach- ers Advisory Group (made up of 12 board-certified teachers), which reviews product plans and development; and the NBPTS President’s Roundtable, a group of philanthropic, business, and community leaders who work to enhance the NBPTS profile. The NBPTS offers a number of resources and supports for candidates and for board-certified teachers, such as a directory of teachers who have earned board certification and a state-by-state list of financial supports 1 The NBPTS also employs 11 regional outreach directors who are responsible for building awareness of the program, and it has opened a facility in San Antonio, Texas, to handle cus- tomer service, candidate materials, and fee processing. Support is also provided by Candidate Subsidy Program Administrators (not NBPTS employees), who perform a variety of functions related to data collection and the disbursement of federal subsidies at the state level.

OCR for page 59
 THE ASSESSMENT PROGRAM available to candidates and rewards to those who earn board certification. The board’s website also provides an opportunity for board-certified teach- ers to network as well as a variety of information related to its work, in- cluding a list of research and other articles, press releases, advocacy pieces, links to other websites, and statistics, such as total board-certified teachers by state (http://www.nbpts.org/). A network affiliate program provides an avenue for board-certified teachers to work together and take on leader- ship roles in their districts and schools. Once they earn board certification, teachers are also encouraged to participate in scoring assessment exercises and to serve as mentors to other candidates. In addition, the board spon- sors a national conference and exposition every other year, which provides an opportunity for board-certified teachers, candidates, and others to at- tend educational sessions and develop connections with education leaders. Approximately 900 people attended the 2007 conference, of whom about two-thirds were board-certified teachers. With funding from federal and corporate sponsors, the board has also developed two programs designed to increase the diversity of the pool of board-certified teachers. The goal of the Targeted High Need Initiative pro- gram is to increase the number of board-certified teachers in high-poverty urban and rural schools. The Direct Recruiting Efforts to Attract Minorities Team highlights the importance of national board certification for minority teachers and their students. The NBPTS has begun offering another option for educators who are interested in pursuing certification. The Take One® program (http://www. nbpts.org/products_and_services/take_one) allows teachers and counselors who are interested but not ready to commit to the certification process to submit a single portfolio entry for scoring. The board presents it as an op- portunity to learn and grow as a teacher, as well as a way to sample the process. Teachers may later apply the scored portfolio entry to a complete application if they wish. APPLYING FOR BOARD CERTIFICATION To begin the process of earning board certification, a candidate must first establish that he or she is qualified to apply. The candidate must have completed a bachelor’s degree, have completed at least three full years of teaching or counseling before beginning the application process, and have had a valid teaching or counseling license throughout that period.2 After submitting documentation of eligibility, the candidate receives more 2 A candidate who does not have a license may be eligible if he or she has been teaching in a school in which licensure is not required that is “recognized and approved to operate by the state” (http://www.nbpts.org).

OCR for page 59
2 ASSESSING ACCOMPLISHED TEACHING detailed instructions about the procedures for completing the entire assess- ment. The candidate must also submit payment of the test fee, which is currently $2,500. The assessment consists of 10 exercises administered in two parts: a computer-based assessment of content knowledge that includes 6 constructed-response exercises, and a portfolio that consists of 4 exercises. Responses to the computer-based exercises and the portfolios are scored at designated times during the year, so the time that elapses between sub- mitting an application and receiving certification depends on the time of year the candidate initiates the process, as well as on how long he or she takes to complete the portfolios and schedule the computer-based assess- ment. National board staff estimate that the typical candidate spends up to 400 hours on the 10 exercises. Given the deadlines throughout the year and the volume of work to be included in the portfolios, most candidates complete the process in approximately 12 to 18 months, but the national board will allow candidates to take as long as 3 years to complete all of the requirements before voiding early scores and asking the candidate to start again. The board offers certification in 16 content areas of specialization covering 7 age groupings, for a total of 25 individual certificates, as shown in Table 4-1. The certificate is effective for 10 years and then must be renewed. THE CONTENT STANDARDS In Chapter 3 we described the development of the board’s five core propositions: 1. Teachers are committed to students and their learning. 2. Teachers know the subjects they teach and how to teach those subjects to students. 3. Teachers are responsible for managing and monitoring student learning. 4. Teachers think systematically about their practice and learn from experience. 5. Teachers are members of learning communities. Although there are detailed content standards for each of the 25 fields for which certification is available, all of the certification-specific standards and assessments are based on these five core propositions. The content standards for each certification area are described in detailed booklets that are posted on the NBPTS website. Each standards document begins with an overview of the board’s approach to determining and structuring its

OCR for page 59
 THE ASSESSMENT PROGRAM standards and then an introduction to the main issues pertaining to the certification area. All of the individual standards are presented in a two- part format: a definition of the standards, or standards statement, followed by elaboration. In the next sections, we briefly describe the standards for these two ar- eas of certification: middle childhood generalist and middle childhood and early adolescence mathematics (National Board for Professional Teaching Standards, 1998, 2001c). Standards for Middle Childhood Generalist There are 11 content standards for the middle childhood generalist certificate. Box 4-1 lists the main standards; the standards document for this certificate provides additional details (see http://www.nbpts.org/the_ standards/standards_by_cert?ID=27&x=40&y=7), including an elabora- tion of what is intended by each. The structure is consistent across fields: a standard statement describes key aspects of accomplished practice in the certificate area in terms of “observable actions of teacher that have an impact on their students” (National Board for Professional Teaching Stan- dards, 2001c). This statement is followed by elaboration, which provides an explanation of what accomplished teachers need to know and demonstrate in order to meet the standard. The elaboration of the first standard for this certificate—knowledge of students—explains that middle childhood generalists who meet this standard “understand and appreciate the ways in which each student is unique as well as the commonalities of middle school students” and are “keen observers of students.” The elaboration provides examples of how these attributes are demonstrated. For example, teachers who understand the developmental status of children in middle childhood “know the im- portance of working at a concrete level, providing material such as maps, timelines, manipulatives, and tools for organizing and interpreting data.” Recognizing the importance of opportunities both to evaluate and analyze and to learn meaningful facts, such teachers “directly teach techniques for locating, retaining, and using facts.” Similarly detailed elaboration is pro- vided for each of the 11 standards. The middle childhood generalist is expected to teach a variety of subjects so the elaboration for Standard 2—knowledge of content and curriculum—describes what such teachers need to know in six subjects (English/language arts, mathematics, science, social studies, the arts, and health) and also describes how they would be expected to use this knowl- edge. For example, an elaboration of this standard for English/language arts indicates that the accomplished teacher will “use a wide range of response activities, such as journals, dramatic productions, and stories, for the pur-

OCR for page 59
 TABLE 4-1 Areas of National Board Certification Early and Early Childhood Adolescence Early Adolescence Early Middle Middle Through Young Early and Young Through Young Childhood Childhood Childhood Adulthood Adolescence Adulthood Adulthood (ages 3-8) (ages 7-12) (ages 3-12) (ages 3-18+) (ages 11-15) (ages 14-18+) (ages 11-18+) Art x x Career and x technical education English as a x x new language English ● x language arts Exceptional x needs specialist Generalist x x Health x Education

OCR for page 59
Library media x Literacy: x reading- language arts Mathematics ● x Music x x Physical x x education School x counseling Science x x Social x x studies-history World x languages other than English SOURCE: See http://www.nbpts.org. 

OCR for page 59
 ASSESSING ACCOMPLISHED TEACHING BOX 4-1 Content Standards for Middle Childhood Generalist 1. Knowledge of Students. Accomplished teachers draw on their knowledge of child development and their relationships with students to understand their students’ abilities, interests, aspirations, and values. 2. Knowledge of Content and Curriculum. Accomplished teachers draw on their knowledge of subject matter and curriculum to make sound decisions about what is important for students to learn within and across the subject areas of the middle childhood curriculum. 3. Learning Environment. Accomplished teachers establish a caring, inclu- sive, stimulating, and safe school community where students can take intellectual risks, practice democracy, and work collaboratively and independently. 4. Respect for Diversity. Accomplished teachers help students learn to re- spect and appreciate individual and group differences. 5. Instructional Resources. Accomplished teachers create, assess, select, and adapt a rich and varied collection of materials and draw on other resources such as staff, community members, and students to support learning. 6. Meaningful Applications of Knowledge. Accomplished teachers engage students in learning within and across the disciplines and help students under- stand how the subjects they study can be used to explore important issues in their lives and the world around them. 7. Meaningful Paths to Knowledge. Accomplished teachers provide students with multiple paths needed to learn the central concepts in each school subject, explore important themes and topics that cut across subject areas, and build overall knowledge and understanding. 8. Assessment. Accomplished teachers understand the strengths and weak- nesses of different assessment methods, base their instruction on ongoing as- sessment, and encourage students to monitor their own learning. 9. Family Involvement. Accomplished teachers initiate positive, interactive relationships with families as they participate in the education of their children. 10. Reflection. Accomplished teachers regularly analyze, evaluate, reflect on, and strengthen the effectiveness and quality of their practice. 11. Contributions to the Profession. Accomplished teachers work with col- leagues to improve schools and to advance knowledge and practice in their field. SOURCE: National Board for Professional Teaching Standards (2001c). Reprinted with permission from the National Board for Professional Teaching Standards, http://www.nbpts.org. All rights reserved. pose of ongoing assessment” and will “incorporate their students’ language strategies and skills into other areas of the curriculum.” The elaboration for this standard is particularly extensive because it treats each field in turn at a similar level of detail.

OCR for page 59
 THE ASSESSMENT PROGRAM Standards for Middle Childhood and Early Adolescence Mathematics The 12 standards for middle childhood and early adolescence math- ematics follow the same format as those for the middle childhood generalist (the standards document is available at http://www.nbpts.org/the_standards/ standards_by_cert?ID=8&x=37&y=10). Because all of the standards derive from the board’s five core propositions, the standard statements, shown in Box 4-2, identify similar goals, such as knowledge of students, tailored to the subject matter and age range targeted. For example, Standard 2 for middle childhood and early adolescence mathematics is similar but not identical to the corresponding standard for middle childhood generalists. The elaboration of this standard for the mathematics certificate addresses such issues as “the differing ways in which students process information on their path to understanding mathematics.” The elaboration notes, for example, that accomplished mathematics teachers are able to “understand a student’s misconception, identify the underlying rationale, and clarify the student’s thinking,” and also understand the developmental changes that take place between the ages of 7 and 15 and are able to “factor this developmental knowledge into their instructional planning.” Such a teacher would recognize that students at the upper end of this range could tackle a complex, multistep problem, but that younger students may have more difficulty with this type of task. THE ASSESSMENT Assessment Center Exercises Candidates’ mastery of the content necessary for accomplished teach- ing in their field is assessed by means of a computer-based assessment con- sisting of six individual 30-minute exercises taken at a designated testing center. The board contracts with a vendor to establish centers throughout the country so that 90 percent of candidates do not need to travel more than 60 miles. Candidates can take the assessment center exercises between July 1 and June 15. The NBPTS posts scoring guides on its website to assist candidates in understanding what will be expected of them; these describe several sample exercises and provide discussion of how responses are scored (see http://www.nbpts.org/for_candidates/scoring?ID=27&x=39&y=9 for the middle childhood generalist scoring guide and http://www.nbpts.org/ for_candidates/scoring?ID=8&x=38&y=8 for the early adolescence math- ematics scoring guide). These assessment exercises focus primarily on the candidates’ knowl- edge of the content important for the area in which they seek certification, although some questions cover pedagogical strategies. For the middle child-

OCR for page 59
 ASSESSING ACCOMPLISHED TEACHING BOX 4-2 Content Standards for Middle Childhood Through Early Adolescence Mathematics CREATING A PRODUCTIVE LEARNING ENVIRONMENT 1. Commitment to Equity and Access Accomplished mathematics teachers value and acknowledge the individuality and worth of each student; they believe that all students can learn and should have access to the full mathematics curriculum; and they demonstrate these beliefs in their practice by systematically providing all students equitable and complete access to math. KNOWLEDGE OF STUDENTS, MATHEMATICS, AND TEACHING 2. Knowledge of Students Accomplished mathematics teachers recognize that students are shaped by a variety of educational, social, and cultural backgrounds and experiences that influence learning. They draw on knowledge of how students learn and develop in order to understand students and to guide curricula and instructional decision. 3. Knowledge of Mathematics Accomplished mathematics teachers draw on their broad knowledge of mathemat- ics to shape their teaching and set curricular goals. They understand significant connections among mathematics ideas and the application of those ideas not only with mathematics but also to other disciplines and the world outside of school. 4. Knowledge of Teaching Practice Accomplished mathematics teachers rely on their extensive pedagogical knowl- edge to make curricular decisions, select instructional strategies, develop instruc- tional plans, and formulate assessment plans. ADVANCING STUDENT LEARNING 5. The Art of Teaching Accomplished mathematics teachers create elegant and powerful approaches to instructional challenges. Their practice reflects a highly developed personal synthesis of their caring for students, their passion for teaching and math, un- derstanding of mathematics content, ability to apply math, and rich knowledge of established and innovative educational practices. 6. Learning Environment Accomplished mathematics teachers create stimulating, caring, and inclusive environments. They develop communities of involved learners in which students accept responsibility for learning, take intellectual risks, develop confidence and self-esteem, work independently and collaboratively, and value math.

OCR for page 59
 THE ASSESSMENT PROGRAM 7. Using Math Accomplished mathematics teachers help students develop a positive disposition for mathematics and foster the development of all students’ abilities to use math- ematics as a way to understand the world around them. They focus instruction on developing students’ mathematics power by providing opportunities for students to understand and apply mathematics concepts; investigate, explore, and discover structures and relationships; demonstrate flexibility and perseverance in solving problems; create and use mathematics models; formulate problems of their own; and justify and communicate their conclusions. 8. Technology and Instructional Resources Accomplished mathematics teachers are knowledgeable about and, where avail- able, use current technologies and other resources to promote student learning in math. They select, adapt, and create engaging instructional materials and draw on human resources from the school and the community to enhance and extend students’ understanding and use of math. 9. Assessment Accomplished mathematics teachers integrate assessment into their instruction to promote the learning of all students. They design, select, and employ a range of formal and informal assessment tools to match their educational purposes. They help students develop self-assessment skills, encouraging them to reflect on their performance. PROFESSIONAL DEVELOPMENT AND OUTREACH 10. Reflection and Growth Accomplished mathematics teachers regularly reflect on teaching and learning. They keep abreast of changes in mathematics and in mathematical pedagogy, continually increasing their knowledge and improving their practice. 11. Families and Communities Accomplished mathematics teachers work to involve families in their children’s education, help the community understand the role of mathematics and math- ematics instruction in today’s world, and, to the extent possible, involve the com- munity in support of instruction. 12. Professional Community Accomplished mathematics teachers collaborate with peers and other education professionals to strengthen the school’s program, promote program quality and continuity across grade levels, advance knowledge in the field of mathematics education, and improve practice within the field. SOURCE: National Board for Professional Teaching Standards (1998). Reprinted with permission from the National Board for Professional Teaching Standards, http://www.nbpts.org. All rights reserved.

OCR for page 59
0 ASSESSING ACCOMPLISHED TEACHING hood generalist certificate, the six exercises measure the ability to support reading skills, analyze student work, and integrate the arts, as well as knowledge of science, social studies, and health. For the middle childhood and early adolescence mathematics certificate, the six exercises measure candidates’ understanding of six areas (algebra and functions, connections, data analysis, geometry, number and operation sense, and technology and manipulatives) with an emphasis on the capacity to draw inferences and apply knowledge to real-world circumstances (National Board for Profes- sional Teaching Standards, 2006b,c). For example, in a sample exercise for the middle childhood generalist certificate, the candidate is asked to imagine that he or she is working with a group of third grade students of mixed ability and to read a passage and a transcription of a student’s oral reading of the passage. The candidate then responds to four prompts that explicitly ask him or her to identify errors, cite examples from the student’s text to support analysis of the student’s skills, describe strategies for addressing the errors, and provide a rationale for using these strategies. An example from the early adolescence mathematics assessments shows a similar assessment approach in a different context. The scoring guide for this exercise, which covers data analysis, explains that candidates must present: • a complete and accurate graphical representation of a given set of data; • a meaningful interpretation of the data based on the graphical representation; • an appropriate and accurate alternate graphical representation of the data; and • a meaningful, accurate, and distinct interpretation of the data based on its alternate graphical representation. For the exercise, the candidate is provided with some data on high- grossing movies. The candidate is asked to create a box-and-whisker plot to display the data, to discuss the skewing of the data, and then to produce an alternate representation of the data and answer a question about it. Portfolio Exercises The portfolio component of the assessment is intended to allow candi- dates to demonstrate their teaching practice and expertise in several ways. Most certificates require candidates to submit: • one classroom-based entry with accompanying student work,

OCR for page 59
 THE ASSESSMENT PROGRAM • two classroom-based entries that require videorecordings of inter- actions between candidates and their students, and • one documented accomplishments entry that provides evidence of the candidate’s accomplishments outside the classroom and how that work impacts student learning. The specific instructions for these portfolio exercises are as follows (National Board for Professional Teaching Standards, 2007, p. 19): Classroom-based entries: Each entry requires some direct evidence of practice as well as a commentary describing, analyzing, and reflecting on this evidence. This commentary requires the teacher to set the work in context and to provide the wider goals for that particular class; and provides an opportunity for the teacher to reflect critically on practice, thus, permitting the raters to see if what the teacher writes about goals or outcomes mirrors what is seen in the evidence of practice. The video, student work or other submitted artifacts, and the commentary support each other in providing evidence of a teacher’s in-class practice. These three entries are specifically designed for each certificate area, though there is significant commonality in the structure of entries from certifi- cate to certificate. Documented accomplishment entry: The entry requires that candi- dates illustrate partnerships with students’ families and community and development as a learner and collaborator with other professionals by submitting descriptions and documentation of activities and accom- plishments in those areas that impact student learning. All of the evidence to be included in the portfolio must be collected within the 12 months preceding the submission deadline. After initiating the application process, the candidate receives a portfolio kit that includes detailed instructions for videotaping lessons and preparing the portfolio, as well as labeling the submission, shipping, and handling other logistics. Candidates are encouraged to videotape many lessons, both to make sure they and their students are comfortable in the presence of the camera and to ensure that they have plenty of videos to choose from as they prepare the submission. The elements of the portfolio are designed as an integrated set of evi- dence both of how the candidate approaches teaching and of the ways in which he or she describes, analyzes, and reflects on that practice. Instruc- tions to candidates emphasize that the materials they submit should provide evidence of their mastery of the five core propositions, and specifically, the NBPTS standards for their certification area. Thus, one of the written en- tries gives the candidate an opportunity to demonstrate understanding of

OCR for page 59
2 ASSESSING ACCOMPLISHED TEACHING his or her students, the goals for the teaching shown in the accompanying videos, and the strategies he or she intended to use. Another is intended to elicit evidence of how thoughtfully the candidate reflects afterward on the teaching interactions he or she has chosen to submit. The NBPTS raters, who work to score the materials that teachers sub- mit, view the videos and evaluate the extent to which the candidate dem- onstrates the NBPTS standards for the certification area, particularly how well he or she knows the students and the subject matter, knows how to teach the subject matter, and can reflect with insight on and learn from his or her teaching experiences. These materials are supplemented with samples of students’ work related to the lesson shown and additional written com- mentary on what that student work demonstrates and how the candidate used and responded to it. Although the portfolios for all certification areas require the same kinds of exercises, the specific instructions to candidates are tailored for each certification area to elicit evidence that they meet the standards out- lined for that field. For example, the portfolio instructions for the middle childhood generalist certificate specify that the teacher must include a vid- eotaped lesson plus written commentary and instructional materials that demonstrate his or her ability to “sustain a classroom environment that supports students’ growth, learning, social and emotional development, and emerging abilities to understand and consider perspectives other than their own through a social/history theme, issue, or topic” in the context of a social studies lesson (National Board for Professional Teaching Standards, 2006a,e). The description of a Level 4 entry for this exercise is shown in Box 4-3. In another example, the first portfolio submission for the early adoles- cence mathematics certificate focuses on developing and assessing math- ematical thinking and reasoning (National Board for Professional Teaching Standards, 2006d). For this entry, candidates are instructed to “demon- strate how the design and implementation of an instructional sequence or unit of study works to inform you of students’ knowledge and furthers students understanding of a substantive idea in mathematics.” The instruc- tions ask them to present “evidence of your ability to plan and implement instruction to facilitate your students’ understanding of an important idea in mathematics” and go on to provide details about exactly what they should submit. The description of a Level 4 entry for this prompt is shown in Box 4-4. Scoring the Assessment Candidates are given detailed descriptions of the criteria the raters use in evaluating both the assessment center and the portfolio exercises. Each

OCR for page 59
 THE ASSESSMENT PROGRAM BOX 4-3 Level 4 Scoring Rubric for “Building a Classroom Community Through Social Studies” Portfolio Entry The Level 4 performance provides clear, consistent, and convincing evidence that the teacher is able to create a stimulating learning climate that supports students’ emerging abilities to understand and consider perspectives other than their own through a social studies/history theme, issue, or topic, and to assume responsibility for their actions. The Level 4 response provides clear, consistent, and convincing evidence that the teacher understands child development and knows the backgrounds, abili- ties, interests, aspirations, and values of her or his students, which is evidenced by the detailed descriptions of the students and the compelling rationale behind the strategies for encouraging students to consider a range of perspectives and to enable students to take responsibility for their own actions. There is clear, consis- tent, and convincing evidence that the strategies employed by the teacher foster students’ emerging abilities to understand and respect individual and group differ- ences, to consider a range of perspectives other than their own, and to assume intellectual and social responsibility. The Level 4 response provides clear evidence that the teacher can establish an equitable, accessible, and fair classroom com- munity where students can take intellectual risks and work collaboratively. The response contains clear, consistent, and convincing evidence that the teacher can plan, organize, and facilitate students’ active participation in a meaningful discussion that develops their expression of ideas and opinions, their consider- ation of others’ points of view, and their assumption of responsibility for their own actions. There is clear evidence of the teacher’s ability to engage in reflective thinking about her or his instructional practice, to support instructional decisions, to articulate a strong rationale for pedagogical actions, and to make decisions that will strengthen the quality of her or his future practice. Overall, there is clear, consistent, and convincing evidence that the teacher is able to create a stimulat- ing learning climate that supports students’ emerging abilities to understand and consider perspectives other than their own through a social studies/history theme, issue, or topic, and to assume responsibility for their actions. SOURCE: National Board for Professional Teaching Standards (2006b). Reprinted with permission from the National Board for Professional Teaching Standards, http://www.nbpts.org. All rights reserved. exercise is assigned a score ranging from 1 to 4; generally, a score of 3 represents passing performance, and a score of 2 is considered below the accomplished level. The scores on each exercise are weighted and combined as follows: each classroom-based entry exercise is weighted 16 percent, the documented accomplishment exercise is weighted 12 percent, and each of the six assessment center exercises is weighted 6.67 percent. The weighted

OCR for page 59
 ASSESSING ACCOMPLISHED TEACHING BOX 4-4 Level 4 Scoring Rubric for “Developing and Assessing Mathematical Thinking and Reasoning” Portfolio Entry The Level 4 performance provides clear, consistent, and convincing evidence that the teacher is able to design a sequence of learning experiences that builds on, and gives insight into, students’ conceptual understanding of a substantive idea in mathematics within the context of instruction that enhances students’ abili- ties to think and reason mathematically. The Level 4 performance provides clear, consistent, and convincing evidence that the teacher sets high, worthwhile, and appropriate learning goals for students based on detailed knowledge of students’ interest, abilities, and needs and that he or she connects the instructional sequence to these goals. The Level 4 response provides evidence that the instructional activities are placed in the larger context of instruction that is designed to enhance student learning in mathematics. The Level 4 response features clear, consistent, and convincing evidence that the instructional sequence includes activities that are sequenced and organized to develop understanding of a substantive mathematical idea as the sequence unfolds while building on students’ interest and prior knowledge. The featured activities clearly and consistently promote mathematical reasoning on the part of students and are effective in eliciting responses that can affect instruction. There is clear, consistent, and convincing evidence of the teacher’s deliberate intent to build students’ conceptual understanding through the strength of the connections between each of the featured activities and the substantive mathematical idea as well as the connection between the two featured activities. The Level 4 response provides clear, consistent, and convincing evidence that the teacher is able to in- scores are combined, producing a score scale that ranges from 1 to 400, with the passing score set at 275. Candidates who do not earn a passing score on their first attempt are allowed to persevere in their attempts to earn certification for up to 24 months after receiving the first score. These candidates may retain any scores and reattempt only those components on which they scored less than 2.75. All submissions are scored by trained raters, who are teachers in the field for which they are scoring (they are not required to be board certi- fied). The raters are trained using sample submissions scored in previous years, and they are overseen by experienced trainers who are also teachers in the field. The training covers not only the scoring rubric but also possible sources of bias—including, for example, the professional bias that might unconsciously influence a rater who has strong views about how particular material should be taught.

OCR for page 59
 THE ASSESSMENT PROGRAM tegrate assessment into instruction and use strategies to probe and push students’ mathematical thinking, particularly by providing feedback that includes targeted questions or instructive comments designed to encourage students to use and develop appropriate mathematical written communication, reasoning, and think- ing. The analysis of student responses is detailed, specific, and accurate, showing differentiated insight into individual students’ learning over time. The feedback and next steps provided to students are rich, detailed, and instructive, moving students toward greater understanding of the featured mathematical concept. There is clear and consistent evidence of the connections among the concept of study, the instructional activities, the analysis of student responses, and the appropriate feedback and next steps for students. There is clear, consistent, and convincing evidence of the teacher’s own knowledge of mathematics and mathematics pedagogy, as shown through the selection of the concept, the way it is taught, and the teacher’s analysis and response to student work. The Level 4 response offers clear, consistent, and convincing evidence that the teacher is able to accurately describe his or her own practice, analyze it fully and thought- fully, and reflect on its implications and significance for future practice. Overall, the Level 4 performance provides clear, consistent, and convincing evidence that the teacher is able to design a sequence of learning experiences that builds on, and gives insight into, students’ conceptual understanding of a substantive idea in mathematics within the context of instruction that enhances students’ abilities to think and reason mathematically. SOURCE: National Board for Professional Teaching Standards (2006b). Reprinted with permission from the National Board for Professional Teaching Standards, http://www.nbpts.org. All rights reserved. Committee Comments on Two Video Portfolios At our fifth meeting, committee members had the opportunity to view two samples of teachers’ submissions for the portfolio exercises required for board certification in the area of adolescence and young adulthood Eng- lish language arts. We reviewed both the written and videotaped materials submitted by the two teachers. To protect the teachers’ privacy, we viewed these materials under secure conditions, with each teacher’s identity, as well as any information that would reveal the identity of the students, ob- scured.3 The presentation was intended to help us understand the certifica- tion process for candidates, the scoring, and the nature of the assessment. 3 In arranging with the NBPTS to review these materials, we agreed to certain terms to protect teachers’ and students’ privacy. Specifically, the committee had access to these materi- als only during the secure review session, which was led by NBPTS staff. All materials were returned to the NBPTS at the end of the session, and any written notes were destroyed. Com- mittee members agreed not to disclose any information that would reveal teachers’ or students’ identity or would reveal secure information about the test.

OCR for page 59
 ASSESSING ACCOMPLISHED TEACHING Despite the information we had already reviewed about the assessment and the standards, we were struck by the richness of what was captured in the videos. Reading about the portfolio exercises and the way they are scored had not fully conveyed to us the kind of information that could be learned from the videos. Readers who have had the opportunity to view videos of teachers in action, perhaps in the context of the Third Interna- tional Mathematics and Science Study, may appreciate how informative they can be. Several aspects of the national board video portfolio process are worth highlighting. During the presentation, the committee saw excerpts from the submis- sions of two teachers. The candidates had been asked to describe in writ- ing the school in which they teach and the group of students shown in the taped lesson. They also discussed the purpose of the lesson, the strategies they had used, and their thoughts about how the lesson actually went. Be- cause candidates are encouraged to videotape many hours of lessons and are completely at liberty in choosing the lessons they would like assessors to see, the tape submitted is considered to be exemplary of the candidate’s practice and of the points he or she makes in the written pieces. In arranging the presentation, we asked to see the portfolio submis- sions of a teacher who had passed the assessment and one who had failed, although we asked not to be told which was which before we completed our reviews. Below we provide our observations about these materials and summarize the skills each teacher demonstrated. Out of consideration for teachers’ and students’ privacy and to abide by the confidentiality terms to which we agreed, we do not go into explicit detail about what we observed but instead try to give the reader a general sense of the dif- ferences between the lesson presented by Teacher A, who passed, and Teacher B, who did not. To the extent possible, we connect our obser- vations to the standards for the adolescence and young adulthood Eng- lish language arts certificate (standards are available at http://www.nbpts. org/the_standards/standards_by_cert?ID=2&x=17&y=13). Teacher A’s lesson focused on a contemporary play that involved in- terpreting abstract material and existential ideas. In his written materials, Teacher A described the students as being generally below grade level and noted that many had learning problems. He had made this assignment because the play was relatively easy to read, even though the ideas were sophisticated and abstract. The videotaped lesson demonstrated that he was able to engage them in an extended classwide discussion of substantive themes about the reading assignment. Teacher A demonstrated a clear com- mand of the material he was presenting to the students, and he deftly guided the conversation so that students identified the key themes in the play. He demonstrated skill at engaging all of the students in the discussion and ef- fectively used strategies to encourage all to participate. All of the students

OCR for page 59
 THE ASSESSMENT PROGRAM seemed attentive during the course of the lesson, and no off-task behaviors were evident. He stimulated students to draw on their own experiences in order to understand the messages in the play. Teacher A’s written materials discussing the lesson demonstrated his self-reflection skills. He provided a thorough analysis of the positive and negative aspects of the lesson. While the lesson appeared to have been quite effective, he identified ways in which it might have been improved. This teacher’s submission demonstrated his knowledge of the students and the subject matter, effective use of instructional strategies to engage students and help them learn, and the ability to create an inclusive and supportive learning environment. Teacher B’s lesson focused on a classic 19th-century novel that her stu- dents were in the process of reading. The lesson was intended to provide historical context for the novel and to tie that context to a discussion of the text. In her written submission, Teacher B had described her students as chatty but very academically able, yet the lesson seemed not to capitalize on their ability levels nor their willingness to converse. The video showed a lesson that appeared to be disjointed and uninteresting to students. Students were not engaged in the discussion, some seemed not to be on task, and many side conversations were occurring. Students did not willingly partici- pate in the discussion, and even when called upon, their comments were not insightful with regard to the novel. Most striking were the lack of structure and haphazardness the lesson displayed and the number of times a possible discussion dwindled for lack of effective encouragement by the teacher. The video showed that Teacher B did not seem to have a strong com- mand of the subject matter. For example, the observations she raised about 19th-century England were trivial, and she missed opportunities to expand on their possible relevance to the events in the novel, the creation of mood, or other important points. She did not demonstrate effective strategies in engaging her students or creating an environment conducive to learning. Her written materials demonstrated a lack of insight into the positive and negative aspects of the lesson and little in the way of self-reflection. For instance, she described the discussion as successful and noted that the les- son was an effective way to introduce contextual background important for understanding the story. However, the video portrayed a class of students who were not engaged in the discussion and did not demonstrate their un- derstanding of important points about characters in the novel. When these misunderstandings became evident—by something the students asked or stated—the teacher was not effective in correcting them, often because the student did not appear to be listening. These two excerpts were chosen to illustrate the two score points on either side of passing, and the committee found there were clear distinctions between the two teachers’ performance. The committee came away from

OCR for page 59
 ASSESSING ACCOMPLISHED TEACHING the presentation with an impression that the process is complex and multi- dimensional, and that it allows teachers to demonstrate a level of thinking and performance that one might expect of accomplished teachers. We were also impressed by the extent to which the video portfolios captured aspects of teaching that could not be assessed with a paper and pencil assessment, such as the effectiveness of their classroom interactions with students. We note, however, that prior to this demonstration, we were not able to get a full sense of the nature of the assessment. At the beginning of our study, no sample portfolio entries had been publicly released, and none was made available to us. Clearly the need to respect candidates’ and students’ confidentiality poses a challenge, but teachers who are considering apply- ing or in the process of preparing their submissions would benefit greatly from the opportunity we were afforded in order to fully understand what is expected of them. Moreover, our reaction to the presentation demonstrated the importance of conveying the nature of the assessment to school admin- istrators, policy makers, and others so that they better understand what is required of teachers who earn board certification.