National Academies Press: OpenBook

Keeping Score for All: The Effects of Inclusion and Accommodation Policies on Large-Scale Educational Assessments (2004)

Chapter: 7 Implications of the Committee's Conclusions and Recommendations

« Previous: 6 Articulating Validation Arguments
Suggested Citation:"7 Implications of the Committee's Conclusions and Recommendations." National Research Council. 2004. Keeping Score for All: The Effects of Inclusion and Accommodation Policies on Large-Scale Educational Assessments. Washington, DC: The National Academies Press. doi: 10.17226/11029.
×

7
Implications of the Committee’s Conclusions and Recommendations

IMPLICATIONS FOR NAEP

The work of this committee was stimulated by concerns about including students with disabilities and English language learners in the National Assessment of Educational Progress (NAEP) and about the effects that accommodations they receive may have on their performance. The National Center for Education Statistics, which initiated the study, asked the NRC to examine NAEP’s policies regarding the inclusion of students with disabilities and English language learners in its assessments, the use of accommodations for these students, and a variety of questions related to these issues. In response, the committee has focused much of its attention on reasons why NAEP officials should review their policies in both of these areas, compare them with those of the states, and work toward greater consistency between NAEP’s policies and those used in the states.

We have recommended that NAEP continue to pursue the goal of maximizing the participation of all students, and that NAEP officials be mindful of the requirements states must meet under the No Child Left Behind Act, recognizing that even though no formal comparisons of state scores to NAEP results are required, NAEP is frequently viewed as an informal benchmark for state results. We hope that this report makes evident that the committee believes it is very important that the sometimes stark differences between the policies in effect for NAEP and those used by states be recognized, and, to the extent possible, reduced.

At the same time, NAEP has in many ways been a model for the assessment community. Much groundbreaking research has been conducted by NAEP’s sponsors and others associated with the NAEP program, such as Educational

Suggested Citation:"7 Implications of the Committee's Conclusions and Recommendations." National Research Council. 2004. Keeping Score for All: The Effects of Inclusion and Accommodation Policies on Large-Scale Educational Assessments. Washington, DC: The National Academies Press. doi: 10.17226/11029.
×

Testing Service (ETS). The developers of NAEP assessments have taken advantage of advances in educational measurement and have been early users of new item types and modes of assessment, such as constructed-response formats, items that require the use of calculators, items that require students to interact with objects such as atlases or scientific specimens, and the like. The NAEP Arts assessment, for example, has demonstrated the willingness of NAEP’s sponsors and assessment developers to explore new ways of gaining information about students’ knowledge and skills. With examples such as these, NAEP has motivated many states to consider and adopt more creative testing approaches than they had been using (National Research Council, 2000a).

Given this record, the committee believes that NAEP has a responsibility to take the lead as well on the complex issues surrounding the assessment of students with disabilities and English language learners. NAEP has published one study on the validity of accommodations that uses an external criterion variable (Weston), and several other studies are underway. We encourage NAEP to continue to pursue research that follows this model. More generally, we hope that the recommendations in this report will spur NAEP’s sponsors to set an example of forward-looking research and practice in these areas.

IMPLICATIONS FOR STATES

Although most of the committee’s messages have been directed to those who develop policy for NAEP and who are responsible for its technical soundness, the issues raised in this report have important implications for state and local assessment programs as well. Points that are relevant to states have been mentioned in several places, and the committee has offered one specific recommendation to states (Recommendation 4-5). However, we close the report with a discussion of the broader implications of our findings and conclusions for states because it is the states that are struggling most immediately with the technical challenges of assessing students with disabilities and English language learners.

The No Child Left Behind legislation has been the source of considerable urgency for states’ efforts to collect data about students’ academic achievement. As we have noted in earlier chapters, the requirement for assessing students and managing the resulting data under this law is steep. Yet previous NRC committees and others have already documented evidence that the collection and reporting of assessment results for these two groups were insufficient even before the No Child Left Behind Act took effect (e.g., National Research Council, 1997a, 1997b, 1999a; Thurlow et al., 2002), and the need for such data does not arise solely because of that law.

With regard to students with disabilities, an NRC committee specifically noted that large-scale studies of education issues frequently fail to include these students in their samples or include them in ways that are not systematic (National Research Council, 1997a, pp. 209-210). That committee called for data on how

Suggested Citation:"7 Implications of the Committee's Conclusions and Recommendations." National Research Council. 2004. Keeping Score for All: The Effects of Inclusion and Accommodation Policies on Large-Scale Educational Assessments. Washington, DC: The National Academies Press. doi: 10.17226/11029.
×

students with disabilities compare with other students on many variables related to their schooling and educational achievement. They pointed out the need for research to investigate the relationship between accommodations and validity, not only to document the effects of various accommodations on test scores but also to develop criteria for deciding which accommodations will preserve the validity and usefulness of test results. This committee also noted the need for research to support the development of reliable and valid alternate assessments, and the equating and scaling of such alternatives.

There are even fewer data available for English language learners. For example, simply finding out how many fourth graders across the country are English language learners cannot readily be done with available data. National demographic data on limited English proficient students have been gathered for decades in the U.S. Department of Education’s Survey of the States’ Limited English Proficient Students and Available Education Programs and Services of Education. However, the quality of the data is uncertain because different states have used different criteria for the identification of students of limited English proficiency (summary reports available from National Clearinghouse on English Language Acquisition: http://www.ncela.gwu.edu). An NRC report on the schooling of language minority children identified specific research needs related to the assessment of these children (National Research Council, 1997b, pp. 128-131). It noted in particular the need for research that links assessment strategies to the knowledge base regarding language acquisition, and that can assist in the development of guidelines for when and how to assess these students, and for the development of accurate and consistent means of scoring the work of English language learners.

There is in general, as we have discussed, a lack of research findings that could be used to help policy makers, administrators, and others make decisions about when and how to offer accommodations to students with different kinds of needs. There is a particular need for research into the validity of interpretations made from accommodated scores, and we have described the direction we think this kind of research should take. We would encourage local education officials and schools to undertake such research, to take part in such research when it is conducted by others, and to provide data to researchers engaged in such research. We believe that such research is a crucial element in the effort to build an education system that strives to meet the needs of all students. In order that resources be adequately managed and that the needs of all students be recognized and addressed, those responsible for meeting these goals must have accurate information about all students.

Suggested Citation:"7 Implications of the Committee's Conclusions and Recommendations." National Research Council. 2004. Keeping Score for All: The Effects of Inclusion and Accommodation Policies on Large-Scale Educational Assessments. Washington, DC: The National Academies Press. doi: 10.17226/11029.
×
Page 123
Suggested Citation:"7 Implications of the Committee's Conclusions and Recommendations." National Research Council. 2004. Keeping Score for All: The Effects of Inclusion and Accommodation Policies on Large-Scale Educational Assessments. Washington, DC: The National Academies Press. doi: 10.17226/11029.
×
Page 124
Suggested Citation:"7 Implications of the Committee's Conclusions and Recommendations." National Research Council. 2004. Keeping Score for All: The Effects of Inclusion and Accommodation Policies on Large-Scale Educational Assessments. Washington, DC: The National Academies Press. doi: 10.17226/11029.
×
Page 125
Next: References »
Keeping Score for All: The Effects of Inclusion and Accommodation Policies on Large-Scale Educational Assessments Get This Book
×
Buy Paperback | $49.00 Buy Ebook | $39.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

U.S. public schools are responsible for educating large numbers of English language learners and students with disabilities. This book considers policies for including students with disabilities and English language learners in assessment programs. It also examines the research findings on testing accommodations and their effect on test performance.

Keeping Score for All discusses the comparability of states’ policies with each other and with the National Assessment of Educational Progress (NAEP) policies and explores the impact of these differences on the interpretations of NAEP results. The book presents a critical review of the research literature and makes suggestions for future research to evaluate the validity of test scores obtained under accommodated conditions. The book concludes by proposing a new framework for conceptualizing accommodations. This framework would be useful both for policymakers, test designers, and practitioners in determining appropriate accommodations for specific assessments and for researchers in planning validity studies.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!