Developing Assessments for the Next Generation Science Standards (2014) / Chapter Skim
Currently Skimming:
3 Assignment Design and Validation
3 Assignment Design and Validation
Pages 47-82
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 47... ...
, will require a careful and thoughtful approach to assessment design. This chapter focuses on strategies for designing and implementing assessment tasks that measure the intended practice skills and content understandings laid out in the Next Generation Science Standards (NGSS)
|
From page 48... ...
in laying out an approach to assessment design that makes use of the fundamentals of cognitive research and theory and measurement science. We first discuss assessment as a process of rea soning from evidence and then consider two contemporary approaches to assess ment development -- evidence-centered design and construct modeling -- that we think are most appropriate for designing individual assessment tasks and collec tions of tasks to evaluate students' competence relative to the NGSS performance expectations.1 We provide examples of each approach to assessment task design.
|
From page 49... ...
For example, assessment tasks that engage students in applying the three-dimen sional learning (described in Chapter 2) could possibly yield information about how students use or apply specific practices, crosscutting concepts, disciplinary core ideas, or combinations of these.
|
From page 50... ...
to the development of tasks or problems for them to answer. Given the complexities of the assessment design process, it is unlikely that such a process can lead to a quality assessment without 50 Developing Assessments for the Next Generation Science Standards
|
From page 51... ...
As we noted in Chapter 2, assessment tasks that comprise multiple interrelated questions, or components, will be needed to assess the NGSS performance expectations. Further, a range of item formats, including construct-response and performance tasks, will be essential for the assessment of three-dimensional learning consonant with the framework and the NGSS.
|
From page 52... ...
Figure 3-2 shows these three essential components of the overall process. The process starts with defining as precisely as possible the claims that one wants 52 Developing Assessments for the Next Generation Science Standards
|
From page 53... ...
The evidence statements associated with given sets of claims capture the features of work products or performances that would give substance to the claims. This evidence includes which features need to be present and how they are weighted in any evidentiary scheme (i.e., what matters most and what matters least or not at all)
|
From page 54... ...
, as well as the references cited above. 54 Developing Assessments for the Next Generation Science Standards
|
From page 55... ...
The first building block is specification of the construct, in the form of a construct map. Construct maps consist of working definitions of what is to be measured, arranged in terms of consecutive levels of understanding or complexity.4 The second building block is item design, a description of the possible forms of items and tasks that will be used to elicit evidence about students' knowledge and understanding as embodied in the constructs.
|
From page 56... ...
56 Developing Assessments for the Next Generation Science Standards
|
From page 57... ...
The subsequent steps in the approach build on the arguments sketched out in domain modeling and represent ed in the design patterns, including designing tasks to obtain the relevant evidence, scoring performance, and reporting the outcomes. The specific design pattern selected for this task supports the writing of storyboards and items that address scientific reasoning and process skills in planning and conducting experimental investigations.
|
From page 58... ...
. 58 Developing Assessments for the Next Generation Science Standards
|
From page 59... ...
Questions 1 and 2 out of 12 Progress bar Click the button to play the 1. As the plunger is pulled back and animation of the Pinball Car race.
|
From page 60... ...
60 Developing Assessments for the Next Generation Science Standards
|
From page 61... ...
___ have on the potential be used in the Pinball Car race. and kinetic energy of the spring?
|
From page 62... ...
62 Developing Assessments for the Next Generation Science Standards
|
From page 63... ...
Question 12 out of 12 Progress bar Experiment 1 and 2 Results Your hypothesis is: [FILL IN PREVIOUS ANSWER] Springs with ___ (more coils/less Spring 6 coils/thicker wire/thinner wirer)
|
From page 64... ...
64 Developing Assessments for the Next Generation Science Standards
|
From page 65... ...
The practices listed in the tables were used in the TABLE 3-1 Assessment Targets for Example 3 (Measuring Silkworms) and the Next Generation Science Standards (NGSS)
|
From page 66... ...
R02484 FIG3-12.eps 66 Developing Assessments for the Next Generation Science Standards
|
From page 67... ...
The third element, outcome space, is a general guide to the way students' responses to items developed in relation to a particular construct map will be valued. The more specific guidance developed for a particular item is used as the actual scoring guide, which is designed to ensure that student responses can be interpreted in light of the construct map.
|
From page 68... ...
68 Developing Assessments for the Next Generation Science Standards
|
From page 69... ...
The third display shows how some students used the measurement scale and counts of cases, but because of difficulties they experienced with arranging the display on paper, they curtailed all counts greater than 6: see Figure 3-16. This display represents student proficiency at Level 4 of the construct map The displays that students developed reveal significant differences in how they thought about and represented their data.
|
From page 70... ...
During this conversation, the students critiqued how space was used in the displays to represent lengths of the larvae and began to 70 Developing Assessments for the Next Generation Science Standards
|
From page 71... ...
. The teacher also led a conversation about the mathematics of display, including the use of order, count, and interval and measurement scale to create different senses of the shape of the data.
|
From page 72... ...
72 Developing Assessments for the Next Generation Science Standards
|
From page 73... ...
are where a student is estimated to have a probability of 0.50 of 9The logit scale is used to locate both examinees and assessment tasks relative to a common, underlying (latent) scale of both student proficiency and task difficulty.
|
From page 74... ...
; see Figure 3-18. 74 Developing Assessments for the Next Generation Science Standards
|
From page 75... ...
3. Meta-representational competence, which is closely related to DaD, proposes keystone performances as students learn to harness varied representations for making claims about data and to consider tradeoffs among representa tions in light of these claims.
|
From page 76... ...
. Validation involves evaluation of the proposed inter pretations and uses of the assessment results, using different kinds of evidence, 76 Developing Assessments for the Next Generation Science Standards
|
From page 77... ...
For the examples discussed in this report, validation would include analysis of the processes and theory used to design and develop the assessment, evidence that the respondents were indeed thinking in the ways envisaged in that theory, the internal structure of the assessment, the relationships between results and other outcome measures, and whether the consequences of using the assessment results were as expected, and other studies designed to examine the extent to which the Assessment Design and Validation 77
|
From page 78... ...
. For 78 Developing Assessments for the Next Generation Science Standards
|
From page 79... ...
It is important to keep in mind that attributes of tasks that may seem unimportant can cause differential performance, often in ways that are unexpected and not predicted by assessment designers. There are processes for bias and sensitivity reviews of assessment tasks that can help identify such problems before the assess Assessment Design and Validation 79
|
From page 80... ...
The endeav or needs to be guided by theory and research about science learning to ensure 80 Developing Assessments for the Next Generation Science Standards
|
From page 81... ...
RECOMMENDATION 3-1 To ensure that assessments of a given performance expectation in the Next Generation Science Standards provide the evidence necessary to support the intended inference, assessment designers should follow a systematic and principled approach to assessment design, such as evidence-centered design or construct modeling. In so doing, multiple forms of evidence need to be assembled to support the validity argument for an assessment's intended interpretive use and to ensure equity and fairness.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.