3
The Advanced Placement Program

The Advanced Placement (AP) and International Baccalaureate (IB) programs are the two most widely known and nationally recognized models for advanced study. Students in AP and IB courses are provided opportunities for both acceleration and advanced study in high school mathematics and science. Both AP and IB are examination-based programs, designed for well-prepared and highly motivated students in their last 2 years of high school. Both certify a student’s level of knowledge with a score on an end-of-course assessment that is recognized and valued nationally and internationally. And both programs are being challenged by a rapid expansion in the number of participating schools and students and by uses of test scores that probably were unforeseen when the programs were first established nearly 50 years ago.

Despite their ostensible similarities, the two programs are actually quite different with respect to their goals, their organization, and the ways in which they assess student learning. This chapter presents a detailed review of the AP program; a similar review of the IB program is provided in Chapter 4, this volume. The first section gives an overview of the AP program; the sections that follow address in turn AP curriculum, instruction, assessment, and professional development.

OVERVIEW

The AP program is the predominant national model for advanced courses in high school. It was established in 1955 to challenge able and well-prepared secondary students with college-level work. Supported by the Educational Testing Service (ETS), the College Board develops AP course descriptions and yearly end-of-course examinations that represent an attempt to mirror the coverage typical of college-level introductory courses. On the basis of demonstrated achievement on the examinations, students may be



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools 3 The Advanced Placement Program The Advanced Placement (AP) and International Baccalaureate (IB) programs are the two most widely known and nationally recognized models for advanced study. Students in AP and IB courses are provided opportunities for both acceleration and advanced study in high school mathematics and science. Both AP and IB are examination-based programs, designed for well-prepared and highly motivated students in their last 2 years of high school. Both certify a student’s level of knowledge with a score on an end-of-course assessment that is recognized and valued nationally and internationally. And both programs are being challenged by a rapid expansion in the number of participating schools and students and by uses of test scores that probably were unforeseen when the programs were first established nearly 50 years ago. Despite their ostensible similarities, the two programs are actually quite different with respect to their goals, their organization, and the ways in which they assess student learning. This chapter presents a detailed review of the AP program; a similar review of the IB program is provided in Chapter 4, this volume. The first section gives an overview of the AP program; the sections that follow address in turn AP curriculum, instruction, assessment, and professional development. OVERVIEW The AP program is the predominant national model for advanced courses in high school. It was established in 1955 to challenge able and well-prepared secondary students with college-level work. Supported by the Educational Testing Service (ETS), the College Board develops AP course descriptions and yearly end-of-course examinations that represent an attempt to mirror the coverage typical of college-level introductory courses. On the basis of demonstrated achievement on the examinations, students may be

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools awarded college credit or advanced placement in higher-level college courses (see Chapter 2, this volume). A high school can elect to offer one or more AP courses simply by scheduling the courses and assigning a teacher. Individual students can elect to take any number of AP courses, as their high school allows, and there is no requirement from the College Board that students who take AP courses and receive the AP designation on their high school transcript take the examination.1 The College Board has developed an AP Diploma that is being offered in the academic year 2000–2001 in 20 school districts across the United States. To earn the diploma, students must complete five AP courses and receive a qualifying score (3) on each of the five examinations. The five courses must include at least one in each of the core areas of mathematics, science, language arts, and history, along with one AP elective course. The College Board plans to offer the AP Diploma in all schools in the 2001–2002 school year. During the past 45 years, the AP program has grown from a small program that served only the top students from largely suburban public and private high schools to one that now is available in a much more diverse group of approximately 62 percent of the nation’s high schools. The program experienced a decade of rapid growth during the 1990s as the number of students taking AP examinations increased from 206,000 to more than 760,000 a year. The number of examinations administered in a year rose from 277,000 in May 1990 to 1,277,000 in May 2000 (College Entrance Examination Board [CEEB], 2000c). In its report Access to Excellence, the Commission on the Future of the Advanced Placement Program (CFAPP) (CFAPP, 2001) describes significant challenges to the program that have accompanied this rapid growth. Since approximately 34 percent of students enrolled in AP courses do not take the AP examinations, the first challenge noted is how the program can maintain the quality of courses and examinations and the validity of the AP credential. Second, students from urban, rural, and poor districts are underrepresented among those who take the AP examinations, and minority students are less likely than other students to take AP courses when they are offered and to achieve success on the examinations. Therefore, the commission notes the critical importance of finding ways to increase the equity of access to AP courses. The report sets forth goals and recommendations that would broaden the aim of the program by positioning it to enable many more students to experience college-level courses and earn college credit while in high school. 1   Some states and school districts require that students who receive an AP credit on their transcript take the corresponding AP examination. This requirement is most common in districts where students receive financial support for taking the examinations.

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools At the same time, the commission emphasizes that the AP program needs to adhere to the following set of expectations to maintain program quality and oversight: While continuing to reflect college-level expectations and cooperation between schools and colleges, AP courses and examinations also should reflect changes in the disciplines and in pedagogy and provide models of effective instruction. The reliability of the AP examinations in measuring student achievement at the introductory college level must be ensured. Standards for AP programs in schools and school systems, for AP teachers, and for teachers’ professional development should be formulated and implemented. The program should focus on expanding access for underserved schools and populations by strengthening the preparation of students in courses that precede AP courses. AP CURRICULUM Consistent with the purpose of the program, AP courses are designed to be equivalent to general introductory college courses with respect to the range and depth of topics covered, the kinds of textbooks used, the kinds of laboratory work done by students, and the time and effort required of students (CEEB, 2001a). A course description for each AP course briefly outlines the topics that may be included on the end-of-course examination, describes the examination format, and provides sample questions. The course description and teacher’s guide for each AP subject area are available for a fee to teachers, who can use them as the basis for developing the curriculum for their own courses (course descriptions can now be downloaded from the College Board’s Web site for free). Individual teachers are given great leeway in structuring AP courses for their classrooms. Thus, the curriculum for AP courses varies from classroom to classroom in both design and implementation, including which topics are emphasized, how the topics are related, how the content is sequenced, and how much time is spent on laboratory activities. Development of AP Courses College faculty members in each discipline, along with experienced AP teachers, are recruited by the College Board to serve on a development committee for each AP course. The traditional strategy for determining the content and skills to be covered in an AP course and on the AP examination

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools has been to survey the department faculty from colleges and universities receiving the most AP score reports in a discipline. The faculty who serve on the subject-specific development committees synthesize the information from these surveys and construct a course description they believe best represents the consensus view of the college-level introductory course (CFAPP, 2001, p. 22). The development committee for a course develops both the topic outline and, with help from ETS experts, the course examinations. The committee prepares the course description for each subject and compiles a list of the textbooks used most frequently in the corresponding college course. Development committees for AP science courses may also recommend laboratory activities that are representative of work done by college students in the corresponding introductory course. This entire process is repeated every 5 to 6 years. One result of this process is that until curricular changes have become common in introductory college courses, those changes are not reflected in the AP course descriptions and examinations. Thus, some disciplinary leaders have contended that, with the growth in AP participation and the program’s influence on high school curricula, the AP course development process has the potential to slow the implementation of desirable curricular reforms. In contrast, the development of AP calculus during the 1990s provides a model for implementing rather than impeding curricular change. The AP Calculus Development Committee used a broader, more forward-looking strategy for revising the description and the examination specifications for this course by collaborating with experts in mathematics content, curriculum, and pedagogy. The committee became an active participant in the calculus reform movement at the college level and the National Council for Teachers of Mathematics–driven standards movement at the precollege level.2 According to CFAPP, “We believe that such strategies must be replicated whenever AP course descriptions are reviewed and revised. Leaders in the disciplines, pedagogy, and research must all play a role in order to create the highest quality curriculum possible” (CFAPP, 2001, p. 12). This committee concurs with that recommendation. Content of AP Courses Each AP course description, or acorn book to use the popular term, includes a topic outline. The major topics on the outline for biology, chemistry, and physics are accompanied by percentages. However, the percent- 2   More information about the AP calculus development process is available at AP Calculus for a New Century by Dan Kennedy, http://www.collegeboard.org/ap/calculus/new_century/index.html (November 26, 2001).

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools ages may have different meanings for different subjects. For example, in the course description for AP biology, the percentages for each major topic and subtopic to be covered in a course are indicated (CEEB, 2001a, pp 5–6). In the course description for AP physics, the “percentage goals for the examination” are given (CEEB, 2001d, pp.16–20). In AP chemistry, the percentages indicate “the approximate proportion of questions on the multiple-choice portion of the examination that pertain to that topic” (CEEB, 2001c, pp. 6–8). A listing of subtopics is provided for each major topic. The portion of the AP chemistry topic outline related to chemical bonding illustrates the level of detail provided by the topic outlines for each subject (CEEB, 2001c, p. 6): B. Chemical Bonding Binding forces Types: ionic, covalent, metallic, hydrogen bonding, van der Waals (including London dispersion forces) Relationships to states, structure, and properties of matter Polarity of bonds, electronegativities Molecular models Lewis structures Valence bond: hybridization of orbitals, resonance, sigma and pi bonds VSEPR [valence shell electron pair repulsion] Geometry of molecules and ions, structural isomerism of simple organic molecules and coordination complexes; dipole moments of molecules; relation of properties to structure The topic outline is intended to indicate the scope of the course, but not necessarily the order or depth in which topics are taught. Teachers are told that although the examination is based on the outline, they may wish to add further topics (CEEB, 2001b, p. 12). Laboratory Requirement for AP Science Courses Because each AP course is developed independently, information on laboratory work provided to teachers of AP science courses varies among disciplines. There is a common recommendation that at least one double period per week in AP chemistry and AP physics and two double periods per week in AP biology be spent in laboratory work. The AP physics course description does not recommend specific laboratory exercises, but rather presents survey results on laboratory emphases in typical introductory col-

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools lege physics courses and describes the diversity of approaches used. The May 2002–May 2003 course description for AP chemistry includes A Guide for the Recommended Laboratory Program for Advanced Placement Chemistry (CEEB, 2001c, pp. 36–51). The guide recommends 22 laboratory activities,3 with the qualification that it is unlikely that every student will complete all 22 activities while enrolled in an AP chemistry course (CEEB, 2001c, pp. 45–49). Students who have completed a first-year chemistry course will likely have been exposed to many of these activities prior to the AP course. For AP biology teachers, an AP biology student laboratory guide and accompanying teacher’s guide are available. Like the recommended chemistry activities, many of the biology laboratories will likely have been part of a first-year biology program. Biology teachers are encouraged to substitute alternative investigations as appropriate if they will enable students to meet the objectives stated in the guide. While the importance of including laboratory work in the curriculum is emphasized to teachers, the interrelationships among curriculum, assessment, and instruction are also highlighted. The AP physics course description suggests that laboratory experiences should help students understand the topics being considered in the course and may improve test performance overall (CEEB, 2001d, p. 8). The description for AP chemistry states, “Data show that student scores on the AP Chemistry Exam improve with increased time spent in laboratory. This correlation is expected to be even stronger now that a question concerned with laboratory experiences is included on the examination each year” (CEEB, 2001c, pp. 43–44). Recent guidelines given to the AP science course development committees include a charge to assess knowledge about laboratory skills and experimentation. Additional Guidance Provided to Teachers About Curriculum Development The College Board asks AP biology teachers to stress understanding of concepts rather than memorization of terms and technical details. The rationale provided is that students who understand the topics on a conceptual level will do better on the national examination. Teachers are told that the AP biology examination will assign increasingly less weight to specific facts, and that students should be encouraged to focus on understanding important relationships, processes, mechanisms, and potential extensions and applications of concepts. According to the College Board, questions on future 3   The recommended activities include some processes and procedures that can often be combined and incorporated into a single laboratory investigation.

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools examinations can be expected to test students’ ability to explain, analyze, and interpret biological processes and phenomena more than their ability to recall specific facts (CEEB, 1999a). With regard to the issue of depth vs. breadth, the AP physics course description states, “concentration on basic principles of physics and their applications through careful and selective treatment of well-chosen areas is more important than superficial and encyclopedic coverage of many detailed topics” (CEEB, 2001d, p. 4). Although the stated goals for AP courses emphasize the importance of depth of understanding of fundamental concepts, the materials provided to teachers give little specific and detailed advice for designing a coherent curriculum. The AP biology course description goes the furthest in articulating the principal concepts underlying the discipline and in organizing knowledge around them: “Emphasizing concepts over facts makes the content of a biology course less overwhelming and more meaningful. A biology course has more structure and meaning when the key concepts for each topic are placed in the broader context of unifying themes” (CEEB, 2001a, p. 4). To this end, the course description identifies three main subject areas that are crosscut by eight biological themes. The College Board presents a brief, open-ended framework for conceptual knowledge and encourages teachers to continue the articulation and organization of knowledge themselves (see Table 3-1). TABLE 3-1 Some Applications of the Energy Transfer Theme to the Three Main Subject Areas Theme I. Molecules and Cells II. Heredity and Evolution III. Organisms and Populations   Plants transform light energy into chemical energy. A cell must spend energy to transcribe and translate a gene because entropy decreases as monomers are organized into complex macromolecules. Energy flows from producers to consumers in an ecosystem. III. Energy Transfer A proton gradient across membranes powers the synthesis of ATP in mitochondria, chloroplasts, and prokaryotes. Energy released by the hydrolysis of ATP is used by cells in DNA synthesis, transcription, and translation. Ion pumps in membranes reestablish a transmembrane resting potential after a neuron fires an impulse or a muscle fiber contracts. SOURCE: Adapted from CEEB (2001a, p. 17). NOTES: ATP = Adenosine Tri-Phosphate. The College Board encourages teachers to continue identifying ways in which the eight major themes can be applied to the three major subject areas.

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools The teacher’s guide for AP biology (Schofield, 2000) encourages teachers to develop a curriculum for conceptual or thematic understanding. The guide illustrates a shift in emphasis of AP biology examination questions away from factual listings of information, as illustrated by the 1989 free-response question about cell energetics, and toward conceptual understanding, as illustrated by the 1995 question. The following examples are taken from that guide (Schofield, 2000, pp. 24–25). Factual free-response question about cellular energetics from the 1989 AP biology examination: Explain what occurs during the Krebs (citric acid) cycle and electron transport by describing the following: The location of the Krebs cycle and electron transport chain in the mitochondria The cyclic nature of the reactions in the Krebs cycle The production of ATP and reduced coenzymes during the cycle The chemiosmotic production of ATP during electron transport Conceptual-thematic free-response question about cellular energetics from the 1995 AP biology examination: Energy transfer occurs in all cellular activities. For three of the following five processes involving energy transfer, explain how each functions in the cell and give an example. Explain how ATP is involved in each example you choose: Cellular movement Active transport Synthesis of molecules Chemiosmosis Fermentation AP INSTRUCTION Little is known, other than anecdotally, about what actually happens in AP classrooms as teachers engage with students in the teaching–learning process. Teachers make decisions every day and in every class period about what to teach and how. Many factors influence these decisions, including teaching philosophies; experience with various teaching strategies; the teacher’s own educational background, experience, and familiarity with various topics in the discipline; and student outcomes. In typical honors courses,

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools teachers generally feel freer to follow their own interests and pursue topics within the broad course outline in which their content knowledge is most secure. In AP courses in which teachers focus on preparing their students for success on an AP examination, there is internal accountability if students take the external examination (Herr, 1993), although little can be said about accountability if large numbers of students in the course do not take the test. Differences among schools also influence how a course is implemented. Scheduling and length of class periods, available facilities and resources, and existing state standards and assessments for courses preceding the AP course all help shape a course differently in each locality. AP Course Descriptions and Teacher’s Guides AP program materials address instruction only in a very general way. For example, a letter to teachers from Gasper Caperton, President of the College Board, is included as a preface to the May 2002–May 2003 course descriptions for AP biology, calculus, chemistry, and physics. He states, “This AP Course Description provides an outline of content and description of course goals, while still allowing teachers the flexibility to develop their own lesson plans and syllabi, and to bring their individual creativity to the AP classroom.” Consistent with this message, for example, the May 2002–May 2003 AP physics course description features a message from the development committee encouraging the following broad instructional goals (CEEB, 2001d): Basic knowledge of the discipline of physics, including phenomenology, theories and techniques, and generalizing principles. Ability to ask physical questions and to obtain solutions to physical questions by use of qualitative and quantitative reasoning and by experimental investigation. Fostering of important student attributes including appreciation of the physical world and the discipline of physics, curiosity, creativity and reasoned skepticism. Understanding connections of physics to other disciplines and to societal issues. (p. 3) The course descriptions in the other subjects also provide statements of goals and emphases that suggest a focus for instruction, such as the value of active learning, but do not address strategies or specific models for instruction in any detail. Instructional strategies that might be employed effectively in achieving the instructional goals specified for AP courses may be among the teaching

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools strategies and techniques included with the sample course outlines presented in the AP teacher’s guides. However, the following two examples of strategies provided by teachers in the teacher’s guide for AP biology (Schofield, 2000) are indicative of the degree of variability in instruction that might be found in AP biology classrooms: “The primary goal of the class is to ensure that students leave having experienced an intense course of college-level biology. I do this mainly through lectures and discussions, during which time students are able to ask plenty of questions.” (p. 45) “I try to provide my students with a variety of teaching techniques that encourage both independent and group activities.” The strategies described by this teacher include student discussion of journal and newspaper articles, student presentations of course topics, use of computer simulation software and a yearlong independent experimental research project. (pp. 58–59) The section of the May 2002–May 2003 course description for AP chemistry entitled A Guide for the Recommended Laboratory Program for Advanced Placement Chemistry (CEEB, 2001c, pp. 36–51) clearly states goals for the laboratory program that include inquiry and student experimentation: The program of laboratory investigations should be seen as a cyclic continuum of inquiry rather than a linear sequence of steps with a beginning and an end … the ideal program should not only allow students to gain experience with traditional laboratory exercises, … but also provide opportunities for students to carry out novel investigations. (CEEB, 2001c, pp. 39–40). The stated goals for laboratory work (CEEB, 2001c) encourage students to: Think analytically and to reduce problems to identifiable, answerable questions. Design and carry out experiments that answer questions. Draw conclusions and evaluate their validity. Propose questions for further study. Communicate accurately and meaningfully. The teacher’s guide for AP chemistry (Mullins, 1994) describes a variety of laboratory programs, but does not address the context in which they might be effective in meeting the stated goals. Most of the programs described in the guide are based on typical published manuals for college-level introductory chemistry laboratory programs that prescribe both a purpose and a step-by-step procedure for students and allow few opportunities

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools for inquiry or student-designed investigations. Some course descriptions include laboratory components that may be more effective in providing these opportunities, including “small-scale inquiry labs” (Mullins, 1994, p. 74) and preparation of a botanically based universal indicator (p. 86). Similarly, the stated goal for the 12 biology laboratory investigations is to encourage higher-level thinking and provide opportunities for students to learn concepts, acquire skills, and engage in problem solving. The manuals, however, while providing important information to teachers about materials and procedures that are essential features of a biology laboratory program, specify the questions to be investigated, as well as step-by-step procedures. They provide no guidance to teachers about establishing a laboratory program that would enable student inquiry and no support for the goal stated in the AP Biology course description that students gain personal experience in scientific inquiry. The report from the CFAPP is also relatively silent about instruction. It recommends only that the College Board set high standards for AP course delivery and provide demonstrations via video and other media of “best practice” teaching approaches (CFAPP, 2001, p. 10). Messages About Instruction Conveyed by AP Examinations Recent guidelines given to the development committees for AP science courses include a charge to assess knowledge about laboratories and experimentation. One of the free-response questions on each AP science examination will be a laboratory question. The format of the second section of the chemistry examination was changed in 1999 with the introduction of a required laboratory-based question. The 1999 question referred to a traditional experiment designed to determine the molar mass of an unknown gas. Commentary on the question in the 1999 released exam for AP chemistry states that the goal of the question is to determine whether students understand how an experiment works, as well as the chemistry behind it. Students having performed the experiment (or a similar one involving the determination of moles of a gas collected over water) in the laboratory may have had an advantage, as the question asked them to explain the purpose of steps in the experiment and list measurements that must be made. However, the question also could have been answered on the basis of knowledge of variables needed in the calculation, without a student’s having actually completed the laboratory. And although one of the stated goals for the AP chemistry laboratory program is for students to design and carry out experiments, students have not yet been asked to design an experiment on an AP chemistry examination. Thus the program materials may be sending conflicting messages,

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools since the examination questions indicate that experience with designing experiments is not necessary for success. AP ASSESSMENT The AP examinations, administered nationally in May of each year, provide the foundation for curriculum and instruction in AP courses. They are timed examinations, with about 50 percent of the total time devoted to multiple-choice questions and the rest to free-response, essay, or problem-solving questions.4 Students can elect to have their examination scores reported to their choice of colleges, where the scores may be considered in decisions about placement in a college course, awarding of credit for the introductory course in the subject, or both (see Chapter 2, this volume). AP courses are intended to represent general introductory-level college courses. The AP examinations are designed to allow students to demonstrate mastery of the concepts and skills learned in the course, enabling some students to undertake, as freshmen, second-year work in the sequence at their institution or to register for courses in other fields for which the general course is a prerequisite.5 Consequently, AP examinations must be valid and reliable measures of student achievement at the college level. “Continued acceptance by colleges and universities of the validity of the content of AP courses, the validity and reliability of the AP Examinations, and the integrity of the scoring process is critical to AP’s success” (CFAPP, 2001, p. 6). How AP Examinations Are Developed Content specifications for AP examinations are determined during the development of AP courses. The development committee for each AP course is responsible for deciding the general content of the examination and the ability level to be tested. The examination is constructed using the topic percentages from the AP course descriptions as a guideline for the distribution of questions. The development committee helps write and review test questions, as well as materials (including the AP course descriptions) that 4   These percentages do not necessarily reflect the weighting of scores as a final examination grade is determined. 5   The policies of colleges and universities vary widely with respect to the score they will accept to award credit or placement and how that score translates into college credit or placement. Indeed, individual departments within an institution of higher education often have very different policies and expectations. This issue is considered in greater detail in Chapter 2 this volume.

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools are distributed to schools. The committees also help create and give final approval for each examination.6 The committees, representing both secondary- and college/university-level teachers, work closely with ETS content specialists and psychometricians to ensure that the examination scores will mean the same thing to from year to year and from student to student. AP validity studies are designed to validate the use of AP for college credit by measuring the comparability of student knowledge of content and processes with that of students in introductory-level college courses (see Chapter 10, this volume, for a discussion of these studies). Most of the multiple-choice questions are written by committee members and pretested in college classes to obtain some estimate of the degree of difficulty and comparability with college courses. For optimal measurement, the AP development committee endeavors to design a multiple-choice section such that the average raw score is between 40 and 60 percent of the maximum possible raw score. Questions at varying levels of difficulty are included. Using many questions of medium difficulty ensures that clear distinctions will be made between students earning grades of 2 and 3 on the one hand and 3 and 4 on the other (CEEB, 2000b, p. 1). Some previously administered questions are included to link the current form of the examination to previous forms, thus maintaining reliability from year to year and examination to examination. The committees write, select, review, and refine free-response questions. One important aspect of test development is determining which item type and format are best for assessing a given topic or skill area. AP development committee members work with AP content experts and ETS statisticians to make this determination. Free-response questions are designed so that students will have to use analytical and organizational skills to solve problems, predict the products of chemical reactions, and formulate answers to questions (CEEB, 2000b, p. 2). The College Board does not, however, employ systematic research to determine the validity of test items in measuring cognitive processes.7 As questions are being written and refined, the development committees propose preliminary scoring standards that are based on consistent criteria from year to year. The committees also develop a formula for assigning composite scores based on differential weights for the multiple-choice and free-response questions. 6   see http://www.collegeboard.org/ap/techman/whois/apdevcom.htm (November 26, 2001). 7   The committee has been informed that an effort is presently under way by the College Board to begin the collection of validity data.

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools TABLE 3-2 College Board Recommendations Regarding Students Qualifications for Receiving College Credit, Placement, or Both AP Grade Qualification 5 Extremely well qualified 4 Well qualified 3 Qualified 2 Possibly qualified 1 No recommendation   SOURCES: CEEB (2001a, 2001b, 2001c, 2001d). How AP Examinations Are Scored In scoring, the number of correct and incorrect answers in the multiple-choice section of an examination is determined, and a correction for guessing is applied. The procedures for scoring the free-response sections are similar to those used by other testing programs to score essays or constructed-response items. Faculty consultants score the free-response questions during AP readings, which are held at various sites (usually college campuses) throughout the United States each June.8 Because it is essential that students’ responses be scored consistently, “a great deal of attention is paid to the creation of detailed scoring guidelines, the thorough training of all faculty consultants, and various ‘checks and balances’ applied throughout the AP Reading.”9 Composite scores are created using formulas developed by each development committee. AP uses a five-point scale for awarding final grades on the examinations. The qualification for college credit that is described for each AP grade is shown in Table 3-2. As indicated, those students who earn a score of 3, 4, or 5 are described by the College Board as qualified for credit and/or enrollment in advanced courses at colleges and universities. Boundaries for awarding AP grades are reset annually at a grade-setting session for each examination. Participants in these sessions usually include the chief faculty consultant, the AP director or associate director from the ETS, the College Board director or associate director of the AP program, ETS content experts for the discipline, and an AP program statistician. Grade distribution charts for each examination are available on the subject pages of the AP Web site.10 8   All of the examinations from a subject area are graded at the same site. 9   See http://www.collegeboard.org/ap/techman/chap3/scorefc.htm (February 11, 2002). 10   See http://apcentral.collegeboard.com/courses/overviews/ (February 11, 2002).

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools How Examination Results Are Reported AP grade reports are sent in July to the college(s) designated by the student, to the student’s secondary school, and to the student. The report includes the grade earned and an interpretation of that grade for each AP exam taken by the student in the current year. A report to AP teachers is sent for each examination taken by five or more students at participating schools. The report compares these students’ performance with that of the national candidate group. Mean scores for the multiple-choice questions grouped by four or five major topics and for each free-response section are compared. The free-response booklets of a school’s senior AP students are released to the schools after October 15 of the year in which the exam is taken if the booklets have not already been requested by a postsecondary institution. The free-response questions on the AP examinations are released to students and teachers after the exam has been administered. They are also posted on the AP Web site, along with scoring guidelines and student samples with scoring commentary, after the examinations have been graded. Entire released examinations, including both the multiple-choice and free-response sections, are published about every 5 years for each subject.11 The publication provides the following: Multiple-choice questions and answer key, and the percentage of candidates who answered each question correctly. Free-response questions and scoring guidelines. Sample student responses with commentary (the actual scores these students received, with a brief explanation). Statistical information about students’ performance on the examination. How the College Board Determines Whether AP Examinations Accomplish Their Purpose AP examinations must be valid and reliable measures of students’ achievement at the college level. Colleges need to know that the AP grades they receive for their incoming students represent a level of achievement equivalent to that of students who take the corresponding introductory course in college. The College Board conducts college comparability studies to confirm that the AP grade scale is properly aligned with current college standards. Periodically, instructors from some of the 200 colleges receiving the largest number of AP grades for an examination administer the AP examina- 11   Free-response questions are released annually.

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools tion or portions thereof to college students who are completing the corresponding college course. The performance of the college students on the examination is compared with that of the AP students. When AP grades are set, the composite score cut points are established so that the lowest composite score for a grade of 5 is roughly equal to the average composite score of college students earning a grade of A. The lowest composite score for a grade of 4 is roughly equal to the average composite score for students with a grade of B. The average composite score of students receiving a grade of C is used to set the lowest AP grade of 3. Similar logic is used in setting the lowest composite score for a grade of 2 (Morgan and Ramist, 1998). AP PROFESSIONAL DEVELOPMENT What Is Known About the Preparation and Credentials of AP Teachers Very little is known about the teaching experiences and preparation of teachers who are teaching AP courses. The College Board is currently conducting a survey of AP teachers with the goal of describing their characteristics and identifying teacher attributes that affect the success of students.12 Until the results of this study are available, information about the teachers involved with the AP program must be based on descriptions provided in program materials. The College Board (CFAPP, 2001, p. 6) states, “AP is an enterprise that relies on a culture of dedication, volunteerism, and altruism among AP teachers.” The College Board (CFAPP, 2001, p. 2) estimates that there were approximately 100,000 teachers teaching AP classes in the United States during the 1999–2000 school year. Assigning teachers to AP courses is usually a school-level decision; the College Board does not certify teachers and has relatively little to say about the matter of teacher qualifications. School administrators attending an AP workshop reported that arbitrary assignment of teachers to AP courses is used only when absolutely necessary and that in many cases, teachers apply specifically to take an AP assignment (Burton, Bruschi, Kindig, and Courtney, 2000). Professional Development Experiences of AP Teachers Although the College Board does not require participation in AP professional development activities, each of the AP teacher’s guides encourages such participation. Stipends and travel support may come from local schools 12   See http://www.collegeboard.org/ap/research/abstract24.html (February 11, 2002).

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools or school districts or from the states and the federal government, or a teacher may receive no support at all. Consequently, many of the teachers who participate in AP professional development activities do so on their own time and at their own expense. The two principal types of professional development available to AP teachers nationally are 1- and 2-day AP workshops offered by the College Board’s regional offices and AP summer institutes, usually 5 days long and offered by a variety of independent agencies. During the 1999–2000 academic year, more than 56,000 AP teachers (about 56 percent of the approximately 100,000 teachers teaching AP classes in the various disciplines) participated in professional development workshops and summer institutes (CFAPP, 2001, p. 2). Teachers who participate may have widely differing professional development experiences, depending on which workshop or summer institute they attend. Recognizing the variability in content and uneven quality of the workshops and summer institutes, the College Board recently developed and implemented quality standards for AP workshop consultants and AP summer institutes (College Board, 2000, 2001). Generally, teachers, guidance counselors, and school administrators attend the 1- or 2-day workshops to learn the rudiments of teaching an AP course and to be apprised of the latest developments and expectations for each course. Major topics covered include the following: Advice on preparing students for the AP examinations. Practical information on starting or revising an AP course. Insights into the structure, content, and grading of the AP examinations. New technology applications. Updates from the College Board and the ETS. AP teachers attending the summer institutes explore a variety of instructional strategies that can be employed both to help their students learn and understand the material and to prepare them for success on the national examination. The strategies discussed range from covering every topic to emphasizing concepts, principles, and a “scientific attitude,” and from teacher-centered, content-based lecture delivery to a variety of student-centered pedagogies (Burton et al., 2000). AP science teachers attending summer institutes often perform and discuss laboratory activities with the guidance of experienced AP instructors. They follow actual laboratory procedures; learn to handle equipment and materials; and discuss time frames, pitfalls, possible shortcuts, reasonable laboratory results, good follow-up questions, and ways of integrating the laboratory experience most profitably into the classroom setting (Schofield, 2000, p. 13). Fifty summer institutes responded to a 1999 survey in which directors were asked to provide evaluation data

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools about their workshops. Workshop participants from these 50 programs indicated that sharing ideas and experiences, networking, and having good discussions with other teachers were of the activities of most value to them. Many of the participating AP teachers reported feeling isolated and unsupported after they left the institutes and workshops, as most of them returned to schools where they were the only AP teacher in a discipline (Burton et al., 2000). Experienced AP teachers often cite the experience of serving as a faculty consultant for the annual AP reading as a highly valued professional development experience. In June 2000, approximately 5,000 college faculty and high school teachers participated in the 1-week sessions. The College Board cites three of the most common reasons given by readers for applying to return after their first experience:13 High school and college faculty exchange ideas on an equal playing field. Networks are established among participants. Practice gained in establishing and applying rubrics makes participants better teachers. The CFAPP recommends that teacher professional development become the area of greatest emphasis for technological investment and development: “Technology can supplement face-to-face workshops and institutes, providing ongoing support to teachers throughout the year” (CFAPP, 2001, p. 11). Services currently provided to teachers by the College Board include an AP electronic mailing list for each discipline—AP Central; the College Board’s online site for the AP program, with information about AP courses, publications, workshops, and summer institutes; and videoconferences with the teachers who develop the AP courses and examinations. The College Board presents no information about local professional development activities available for AP teachers, but AP teacher’s guides for the individual disciplines refer teachers to their respective professional organizations, which often sponsor sessions on AP at state and national conventions. The CFAPP report warns that the current models for AP professional development are and will continue to be insufficient. An estimated 100,000 new AP teachers will be needed between now and the end of the decade, and many of them will be teaching AP for the first time. The commission states: “The College Board, working in cooperation with schools, school systems, colleges and universities, governments and others, must make an unconditional commitment to teacher professional development. Without 13   See http://www.collegeboard.org/ap/readers/index.html (November 26, 2001).

OCR for page 64
Learning and Understanding: Improving Advanced Study of Mathematics and Science in U.S. High Schools this commitment, access to AP will not improve and quality will decline” (CFAPP, 2001, p. 10). Specific recommendations include (1) developing and implementing a mentoring system using experienced and retired AP teachers as mentors, (2) identifying successful strategies for preparing teachers to teach AP, and (3) evaluating and incorporating new models of professional development that support instructional and curricular changes. The commission also recommends that the College Board expand the development and implementation of the AP Vertical Teams program. This program focuses on collaborative planning workshops at which teachers in grades 7–12 are involved in redesigning middle school and high school curricula to give students good preparation for AP and other advanced courses.