Testing Service (ETS). The developers of NAEP assessments have taken advantage of advances in educational measurement and have been early users of new item types and modes of assessment, such as constructed-response formats, items that require the use of calculators, items that require students to interact with objects such as atlases or scientific specimens, and the like. The NAEP Arts assessment, for example, has demonstrated the willingness of NAEP’s sponsors and assessment developers to explore new ways of gaining information about students’ knowledge and skills. With examples such as these, NAEP has motivated many states to consider and adopt more creative testing approaches than they had been using (National Research Council, 2000a).
Given this record, the committee believes that NAEP has a responsibility to take the lead as well on the complex issues surrounding the assessment of students with disabilities and English language learners. NAEP has published one study on the validity of accommodations that uses an external criterion variable (Weston), and several other studies are underway. We encourage NAEP to continue to pursue research that follows this model. More generally, we hope that the recommendations in this report will spur NAEP’s sponsors to set an example of forward-looking research and practice in these areas.
Although most of the committee’s messages have been directed to those who develop policy for NAEP and who are responsible for its technical soundness, the issues raised in this report have important implications for state and local assessment programs as well. Points that are relevant to states have been mentioned in several places, and the committee has offered one specific recommendation to states (Recommendation 4-5). However, we close the report with a discussion of the broader implications of our findings and conclusions for states because it is the states that are struggling most immediately with the technical challenges of assessing students with disabilities and English language learners.
The No Child Left Behind legislation has been the source of considerable urgency for states’ efforts to collect data about students’ academic achievement. As we have noted in earlier chapters, the requirement for assessing students and managing the resulting data under this law is steep. Yet previous NRC committees and others have already documented evidence that the collection and reporting of assessment results for these two groups were insufficient even before the No Child Left Behind Act took effect (e.g., National Research Council, 1997a, 1997b, 1999a; Thurlow et al., 2002), and the need for such data does not arise solely because of that law.
With regard to students with disabilities, an NRC committee specifically noted that large-scale studies of education issues frequently fail to include these students in their samples or include them in ways that are not systematic (National Research Council, 1997a, pp. 209-210). That committee called for data on how