or teams charged with curriculum evaluations conduct studies that make use of the following framework:

1. The program materials and author’s design principles;

2. The quality, extent, and means of curricular implementation components; and

3. The effects on the quality, breadth, type, and distribution of outcomes of student learning over time.

The quality of an evaluation depends on how well it connects these components into a chain of reasoning, evidence, and argument to show the effects of curricular use, and to demonstrate their connection to the treatment under investigation. Studies could also include systematic variation to explore which features of curricula are context dependent and which are context independent.

In applying the framework, one needs to distinguish two different aspects of determining curricular effectiveness. First, a single study should demonstrate that it has obtained a level of scientific validity. Then, for a curricular program to be established as effective, a set of scientifically valid studies should be aggregated and synthesized to yield a judgment of effectiveness. We address each of these aspects in turn.

Based on the framework, the committee identified a set of methodological categories of evaluations. For each category, the committee developed a set of methodological expectations for conducting that type of study. This permitted us to define a scientifically valid study as follows:

For a single curricular evaluation to be scientifically valid, it should address the components identified in the “Framework for Evaluating Curricular Effectiveness.” In addition, it should conform to the methodological expectations of the appropriate category of evaluation as discussed in the report (content analysis, comparative study, or case study). Other designs are possible but would have to address both the theoretical and methodological considerations specified in the framework.

Front Matter (R1-R16)

Executive Summary (1-10)

1 The Challenge and the Charge (11-17)

2 Committee Procedures and Characteristics of Studies Reviewed (18-35)

3 Framework for Evaluating Curricular Effectiveness (36-64)

4 Content Analysis (65-95)

5 Comparative Studies (96-166)

6 Case Studies and Synthesis Studies (167-184)

7 Conclusions and Recommendations (185-205)

References (206-218)

Appendix A: Biographic Sketches (219-222)

Appendix B: Bibliography of Studies Included in Committee Analysis (223-258)

Appendix C: Outcomes Measures (259-260)

Index (261-272)