robust, then the committee was also asked to “develop recommendations about the design of a subsequent project that could result in the generation of more reliable and valid data for evaluating these materials.”

In response to our charge, the committee finds that:

Therefore, according to our charge, we recommend that:

The committee emphasizes that we did not directly evaluate the materials. We present no analysis of results aggregated across studies by naming individual curricular programs because we did not consider the magnitude or rigor of the database for individual programs substantial enough to do so. Nevertheless, there are studies that provide compelling data concerning the effectiveness of the program in a particular context. Furthermore, we do report on individual studies and their results to highlight issues of approach and methodology. To remain within our primary charge, which was to evaluate the evaluations, we do not summarize results on the individual programs.

The second part of our charge was to present the criteria and framework for reviewing the evidence. To do so, we have developed a set of definitions of key terms which draw on a framework for evaluating the effectiveness of mathematics curricula. Using these definitions and the framework, we were able to undertake a review of the major categories of evaluation studies. We briefly review the definitions and the framework.

To guide our review of evaluations of mathematics curricula, the committee developed a “Framework for Evaluating Curricular Effectiveness” (see Figure 3-2). This framework emerged from deliberations of the committee following the testimony of experts in the field at two workshops held during 2003, motivated by the need to find common ways to examine different types of evaluations. It permitted the committee to compare evaluations and consider how to identify and distinguish among the variety of methodologies they employed. The committee recommends that individuals

Front Matter (R1-R16)

Executive Summary (1-10)

1 The Challenge and the Charge (11-17)

2 Committee Procedures and Characteristics of Studies Reviewed (18-35)

3 Framework for Evaluating Curricular Effectiveness (36-64)

4 Content Analysis (65-95)

5 Comparative Studies (96-166)

6 Case Studies and Synthesis Studies (167-184)

7 Conclusions and Recommendations (185-205)

References (206-218)

Appendix A: Biographic Sketches (219-222)

Appendix B: Bibliography of Studies Included in Committee Analysis (223-258)

Appendix C: Outcomes Measures (259-260)

Index (261-272)