Given this conclusion, the committee turned to the second part of its charge, developing recommendations for future evaluation studies. To do so, the committee developed a framework for evaluating curricular effectiveness. It permitted the committee to compare evaluations and consider how to identify and distinguish among the variety of methodologies employed.

The committee recommends that individuals or teams charged with curriculum evaluations make use of this framework. The framework has three major components that should be examined in each curriculum evaluation: (1) the program materials and design principles; (2) the quality, extent, and means of curricular implementation; and (3) the quality, breadth, type, and distribution of outcomes of student learning over time.

The quality of an evaluation depends on how well it connects these components into a research design and measurement of constructs and carries out a chain of reasoning, evidence, and argument to show the effects of curricular use.

The committee distinguished two different aspects of determining curricular effectiveness. First, each individual study should demonstrate that it has obtained a level of scientific validity. In the committee’s view, for a study to be scientifically valid, it should address the components identified in the framework and it should conform to the methodological expectations of the appropriate category of evaluation as discussed in the report (content analysis, comparative study, or case study).

Defining scientific validity for individual studies is an essential element of assuring valid data about curricular effectiveness. However, curricular effectiveness cannot be established by a single scientifically valid study; instead a body of studies is needed, which is the second key aspect of determining effectiveness. Curricular effectiveness is an integrated judgment based on interpretation of a number of scientifically valid evaluations that combine social values, empirical evidence, and theoretical rationales.

Furthermore, a single methodology, even replications and variations of a study, is inadequate to establish curricular effectiveness, because some types of critical information will be lacking. For example, a content analysis is important because, through expert review of the curriculum content, it provides evidence about such things as the quality of the learning goals or topics that might be missing in a particular curriculum. But it cannot determine whether that curriculum, when actually implemented in classrooms, achieves better outcomes for students. In contrast, a comparative study can

Front Matter (R1-R16)

Executive Summary (1-10)

1 The Challenge and the Charge (11-17)

2 Committee Procedures and Characteristics of Studies Reviewed (18-35)

3 Framework for Evaluating Curricular Effectiveness (36-64)

4 Content Analysis (65-95)

5 Comparative Studies (96-166)

6 Case Studies and Synthesis Studies (167-184)

7 Conclusions and Recommendations (185-205)

References (206-218)

Appendix A: Biographic Sketches (219-222)

Appendix B: Bibliography of Studies Included in Committee Analysis (223-258)

Appendix C: Outcomes Measures (259-260)

Index (261-272)