being rejected. In designing experiments, they tend to seek information that confirms a hypothesis, change too many variables at one time, or manipulate variables irrelevant to the hypothesis. Frequent problems in the interpretation of data include confirming the hypothesis regardless of what the data indicate and difficulty in interpreting graphs (Roberts et al., 1997). Teachers benefit from assessing their students’ initial ideas about what it means to conduct an investigation and think scientifically and how these ideas and their skills change over time.
It is easy to say that students should not simply learn isolated facts or definitions without understanding. It is harder to say what the understanding of a concept looks like or how students should produce evidence of their understanding. In the New Standards Project, in which several states and urban districts are working together to develop an assessment system based on the Standards, conceptual understanding is described as follows:
The student demonstrates conceptual understanding by using a concept accurately to explain observations and make predictions and by representing a concept in multiple ways (through words, diagrams, graphs, or charts, as appropriate). Both aspects of understanding — explaining and representing — are required to meet this standard (New Standards, 1997, p. 133).
Similarly, the AAAS Assessment Blueprint (AAAS, 1998) suggests posing questions that stress reflective thinking, requiring the integration of information, rather than reflexive thinking, where a memorized response is called for. As the Blueprint puts it, “Students should be asked to address questions such as, ‘How do we know this?’ and ‘What difference does it make?’ rather than being asked to reproduce memorized vocabulary items or the like.”
Again, many of these strategies were apparent in the vignettes in Chapter 3. For example, when Mrs. Flores wanted to assess her students’ understanding of the idea of a fair test, she had them evaluate whether a design they had not previously encountered was fair. Ms. Flores also gave her students rich and open tasks such as designing soda bottle homes for their worms based on their observations of the places where they found worms naturally.
However, many of the assessments in Chapter 3 guided the actual day-to-day evolution of lessons, making those assessments susceptible to general judgments and off-the-top evaluations of competence. For assessments that carry stakes, whether of passing courses or assigning grades, “standardized” ways of evaluating knowledge and abilities are needed, prefer-