Schools, 2000.⁵ The panel found surprisingly few other data on AP and IB on which to base its evaluation.⁶

For example, little is known, except anecdotally, about how either program is implemented in U.S. high schools, including the instructional strategies and resources used in individual classrooms, the structure of the syllabi in different schools, the quantity and quality of the facilities available, the preparation of teachers who teach the courses, and the ways in which students are prepared prior to enrolling in AP calculus or advanced IB mathematics courses. Information about the AP and IB assessments is also limited. The IBO currently conducts no systematic research addressing the validity,⁷ reliability,⁸ or comparability⁹ of its assessments across administrations (Pook, 2001). The senior examiners, not psychometricians, make determinations about the degree to which each administration is a valid and reliable measure of student achievement. The College Board, on the other hand, has gathered considerable data to demonstrate the reliability and comparability of student scores from one administration to the next and from one student to another. However, neither program has a strong program of validity research, and neither has gathered data to document that the test items on its examinations measure the skills and cognitive processes they purport to measure. To fully analyze or evaluate the AP and IB assessments, it is necessary to know, for example, that test items intended to measure problem solving do in fact tap those skills and do not just elicit memorized solutions or procedures.

Further, little evidence is available for evaluating the long-term effects of the AP and IB programs. For instance, the panel could not find systematic data on how students who participate in AP and IB fare in college mathematics relative to other students. Nor could the panel find studies that examined the effects on postsecondary mathematics programs of the ever-increasing numbers of students who are entering college with credit or advanced standing in mathematics. While the College Board and a few colleges that receive IB students have conducted some isolated studies addressing how AP or IB students perform in college (see, for example, Morgan and Ramist, 1998), the inferences that can accurately be drawn from the findings of these studies are ambiguous (see Chapter 10 of the parent committee’s report).

Because empirical evidence about the programs’ quality and effectiveness is lacking, the panel focused its analysis on what the programs say they do, using available program materials.