The following HTML text is provided to enhance online
readability. Many aspects of typography translate only awkwardly to HTML.
Please use the page image
as the authoritative form to ensure accuracy.
On Evaluating Curricular Effectiveness: Judging the Quality of K-12 Mathematics Evaluations
achievement when curricula are implemented in classrooms. Learning is taking place in a complex dynamical system, and one should consider whether the correction of current curricular deficiencies, a process that obviously must be carried out, is the key pressure point for changing the educational system. Will curricular content changes by themselves actually reduce the disparities in learning that have been documented in many studies? Answers to analogous questions exist in other fields. For example, although detailed understanding of the immunology of HIV (resulting in the development of a robust vaccine) may be the key in the long run, as of today, human behavior along with cultural practices and norms are the drivers (pressure points) of the HIV epidemic. Efforts to alter behavior and cultural practices and norms, at this point, are most likely to have immediate impact on HIV transmission dynamics. Furthermore, changes in the social landscape will have a beneficial impact on the transmission dynamics and control of many sexually transmitted diseases. Analogously, in examining curricular effectiveness the role of content analyses, while critical, is not the only pressure point in the system and may interact in fundamental ways with other pressure points. Therefore it is essential to consider methods and approaches that take into account the possible impact and implications of changing and evolving landscapes on a curriculum’s implementation and effectiveness. The landscape shift illustrated in the 5-year longitudinal study by Carroll (2001) of Everyday Mathematics highlights some of these challenges. The next two chapters, on comparative studies and case studies, add indirect consideration of other influences on the curricular effectiveness.
Nonetheless, as we transition to these, we emphasize the importance of content analysis in relation to comparative and case studies. For example, content analyses can offer insights into the designs of comparative analyses. A thorough content analysis provides a clear articulation of the program theory from one point of view. From a set of such reviews, researchers can identify contentious issues that merit further basic research. One could examine questions such as, Do clear, concise materials used as primary texts lead to stronger conceptual development than engaging, active challenges and tasks? or Does the introduction of formal definitions facilitate further and deeper understanding and mastery of new material? How and when does this work? Research is needed to determine whether analyses of the intended curricula are validated by the empirical outcomes of the enacted curricula. Content analyses are also valuable to inform the conduct of comparative studies. A content analysis can help an evaluator to select appropriate outcome measures, to measure particularly important content strands, and to concentrate on the essential aspects of implementation and professional development. For these multiple reasons, careful and increasingly sophisticated content analysis will make important contributions to the evaluation of the effectiveness of a curricular program.