Does it address important individual and societal needs?
Does the program make a measurable difference in student learning?
AAAS’s Project 2061 (http://www.project2061.org) developed and used a methodology to review middle grades curricula and subsequently algebra materials. In describing their methods, Kulm and colleagues (1999) outlined the training and method. After receiving at least three days of training before conducting the review, each team rated a set of algebra or middle grades standards in reference to the Curriculum and Evaluation Standards for School Mathematics from the National Council of Teachers of Mathematics (NCTM) (1989). In algebra, each review encompassed ideas from one area of algebra: functions, operations, and variables. Similarly in middle grades, particular topics were specified. Each team used the same idea set to review a total of 12 textbooks or sets of textbooks over 12 days. Two teams reviewed each book or set of materials.
The content analysis procedure is outlined on the AAAS website and summarized below:
Identify specific learning goals to serve as the intellectual basis for the analysis, particularly to select national, state, or local frameworks.
Make a preliminary inspection of the curriculum materials to see whether they are likely to address the targeted learning goals.
Analyze the curriculum materials for alignment between content and the selected learning goals.
Analyze the curriculum materials for alignment between instruction and the selected learning goals. This involves estimating the degree to which the materials (including their accompanying teacher’s guides) reflect what is known generally about student learning and effective teaching and, more important, the degree to which they support student learning of the specific knowledge and skills for which a content match has been found.
Summarize the relationship between the curriculum materials being evaluated and the selected learning goals.
The focus in examining the learning goals is to look for evidence that the materials meet the following objectives:
Have a sense of purpose.
Build on student ideas about mathematics.
Engage students in mathematics.
Develop mathematical ideas.
Promote student thinking about mathematics.