Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 147
Measuring What Counts: A Conceptual Guide for Mathematics Assessment COMMISSIONED PAPERS EFFECTS OF MANDATED TESTING ON INSTRUCTION DESIGN INNOVATIONS IN MEASURING MATHEMATICS ACHIEVEMENT LEGAL AND ETHICAL ISSUES IN MATHEMATICS ASSESSMENT
OCR for page 148
Measuring What Counts: A Conceptual Guide for Mathematics Assessment This page in the original is blank.
OCR for page 149
Measuring What Counts: A Conceptual Guide for Mathematics Assessment EFFECTS OF MANDATED TESTING ON INSTRUCTION LYNN HANCOCK JEREMY KILPATRICK UNIVERSITY OF GEORGIA The past two decades have seen a striking increase in the use of testing in the United States by school officials and legislators attempting to determine whether funds invested in schools are yielding an educated citizenry. Testing is viewed as the major instrument for holding schools accountable for the resources they have received. It has become a vital tool of state and federal education policy. Governments and local school authorities have mandated the administration of tests, usually at the end of major phases of schooling but sometimes at the end of each grade, in the belief that test scores provide critical information on how well students are learning and how effective instruction has been. Testing of all types seems to be on the rise in the United States, but the increase in mandated testing has been especially dramatic. In 1990, 46 states had mandated testing programs as compared with 29 in 1980. As the school population increased 15 percent from 1960 to 1989, revenues from the sales of standardized tests increased 10 times as fast.1 More than a third of the elementary school teachers in a recent survey2 saw the emphasis on standardized testing in U.S. education as strong and getting stronger. Somewhat fewer saw the same increasing emphasis in their school district, and even fewer saw it in their own school. Almost no
OCR for page 150
Measuring What Counts: A Conceptual Guide for Mathematics Assessment teachers, however, said that the emphasis on standardized testing was weak or diminishing. As the amount of instructional time lost to mandated testing increases, teachers and other educators have begun to express concern about the effects of such testing on instruction. Because most of the standardized tests used in mandated testing programs are of the multiple-choice variety, particular attention has been given to the argument that these tests promote a narrow approach to teaching, passive and low-level forms of learning, and a fragmented school curriculum. The amount of available research to address these concerns and arguments, however, is quite sparse. Much of this research consists of surveys of teachers' appraisals of the effects of mandated testing rather than direct observation or independent judgments of these effects. The findings from the research are often inconclusive and sometimes conflicting. The purpose of this paper is to review the literature on the effects of mandated testing on school instruction. Because the climate of educational testing in the United States has changed so rapidly over the past decade, we give special attention to the most recent studies. Furthermore, although the research is not confined to mandated testing in mathematics, we have tried to draw conclusions of particular relevance to the mathematics education community. EFFECTS ON CURRICULUM Resnick and Resnick3 portray the process by which state legislatures and departments of education use accountability programs to control curriculum content and standards of performance as follows. Tests of desired educational objectives are mandated and administered, and the scores are widely disseminated. Because of the attention given to test results, teachers gradually adapt their instruction to the test objectives and format. Adaptation of the curriculum takes place as teachers who administer a test every year have the opportunity to see test forms and compare test content with the content they are teaching. The result is that "you get what you assess, and you do not get what you do not assess."4
OCR for page 151
Measuring What Counts: A Conceptual Guide for Mathematics Assessment Others have also described this process. The term WYTIWYG—what you test is what you get—was coined by Burkhardt et al.5 to describe the effects of public examination systems on the curriculum. Burkhardt6 claims that desirable changes in the mathematics curriculum can be brought about through modest, carefully planned changes in examinations. In this way, WYTIWYG can serve as a lever of educational reform. Some contend that the power of that lever depends on the importance that has been placed on the test results. Popham7 used the expression measurement-driven instruction to describe classroom practices motivated by consequences, or stakes, attached to the test results. He identified two types of high stakes for tests. One type is characterized by the use of scores to make important decisions about students, such as promotion to the next grade, reward of course credit, or qualification for a high school diploma. The other type of high stakes is associated with news media reports of school or district test results. Thus, high-stakes tests draw their power from educators' concerns for students' welfare and for their own standing in the community. One of Madaus's principles of measurement-driven instruction8 is that high-stakes tests have the power to transfer what was once local control over the school curriculum to the agency responsible for the examination. We begin, therefore, with an examination of evidence that externally mandated tests are influencing school mathematics curricula and, if so, what the nature of that influence is. CHANGES IN CONTENT Several recent studies have looked at the effects that mandated testing programs have on curriculum content. They show that, to various degrees, the WYTIWYG phenomenon is at work in classrooms. According to Stake and Theobold,9 the most frequently reported change in school conditions that teachers attributed to the increased emphasis on testing is greater pressure to teach stated goals. Darling-Hammond and Wise10 collected data from in-depth interviews with 43 randomly selected teachers from three large school districts in three mid-Atlantic states. When asked what impact standardized tests had upon their classroom behavior, the most common response was that they changed their curriculum emphasis. Some teachers reported that the emphasis on standard-
OCR for page 152
Measuring What Counts: A Conceptual Guide for Mathematics Assessment ized tests has caused them "to teach skills as they are tested instead of as they are used in the real world."11 Nature of the effects As part of a study of the impact of the minimum competency testing program that the state of Kansas implemented in 1980 at grades 2, 4, 6, 8, and 10, Glasnapp et al.12 solicited the opinions of school board members, superintendents, principals, and teachers. A different questionnaire was used for each group, with 1,358 teachers participating in the 1982 survey, 816 in 1983, and 1,244 in 1987. The data show that as the test objectives were more widely distributed and as the teachers reported increased encouragement to direct their instruction to the state objectives, there was a corresponding increase in teachers' reports that the test was influencing their instruction. Over half the teachers surveyed in 1987 reported that the test objectives were valuable for identifying what needed to be taught and that they had given those objectives increased emphasis in their instruction. Nearly half said they used the state-distributed minimum competency objectives to plan instructional activities, up from 38 percent in 1983 and 23 percent in 1982. The Kansas teachers also reported that the state minimum competency testing reduced the time they spent teaching skills that the tests did not cover. Smith and Rottenberg13 interviewed 19 elementary school teachers and then observed the classes of four of these teachers for an entire semester, during which externally mandated tests were administered. The researchers noted a definite trend, which they attributed to time constraints and a packed curriculum, to neglect topics not included on the standardized tests and to focus on those that were. Mathematics beyond what was to be covered on the tests was given very little attention. Romberg et al.14 undertook a study of eighth-grade teachers' perceptions of the impact that their state- or district-mandated testing programs had on mathematics instruction. A national sample of 552 teachers responded to the survey questionnaire. Of the 252 respondents who said they administered a state-mandated test, 34 percent reported placing a greater emphasis on the topics emphasized on the test and 16 percent reported placing less emphasis on topics not emphasized. In reaction to their state testing programs, 23 percent of the teachers were placing a greater emphasis on paper-and-pencil computation whereas only 1 percent were de-
OCR for page 153
Measuring What Counts: A Conceptual Guide for Mathematics Assessment creasing that emphasis. As for familiarity with their state tests, 46 percent said they looked at the test to see whether the topics were those they were teaching whereas 33 percent reported that they did not examine the test at all. Strength of the effects Clearly, these studies demonstrate that mandated testing is having an impact on the content of mathematics instruction, but the strength of that impact is another question. Porter et al.,15 in a review of several studies of elementary school teachers' decisions about instructional content, found little evidence that standardized tests given once a year significantly influence the choices made about what to teach. These studies did not, however, take into account the importance the elementary school teachers attached to the tests, either for their students or for themselves. Recent studies seem to bear out the claim of Madaus16 and Popham17 that the higher the stakes attached to the test results, the greater the impact of the testing program on the curriculum. In interviews conducted by Darling-Hammond and Wise,18 teachers typically reported that when tests are used to measure teacher effectiveness or student competence, incentives are created to teach the precise test content instead of underlying concepts or untested content. Corbett and Wilson19 studied the effects of state-mandated minimum competency testing programs in Maryland and Pennsylvania. At the time of the study, Maryland students needed passing scores on the reading and mathematics tests in order to receive a high school diploma. In Pennsylvania the purpose of the minimum competency tests in language and mathematics was to identify students in need of remedial instruction. Thus the Maryland test was considered by the researchers to have higher stakes than the Pennsylvania test. The study results consistently showed that the Maryland testing program had the more powerful influence on the school curriculum. For example, in Maryland 53 percent of the educators surveyed reported a major or total change in class content resulting from their state testing program. In Pennsylvania only 7 percent reported a major change in their instructional content. A national survey of teachers on the influence of mandated testing on mathematics and science teaching was conducted as part
OCR for page 154
Measuring What Counts: A Conceptual Guide for Mathematics Assessment of a larger study by the Center for the Study of Testing, Evaluation, and Educational Policy.20 The survey findings were based on over 1800 responses from teachers whose classes were given mandated tests in mathematics. The results showed that teachers with high-minority classes (greater than 60 percent minority students) perceived standardized tests to be of greater importance than did teachers with low minority classes (less than 10 percent minority students). Teachers of high-minority mathematics classes were more likely to use mandated test scores to place students in special services, to recommend students for graduation, and to evaluate student progress. These teachers also felt more pressure to improve their students' scores on mandated mathematics tests. Two thirds of the high-minority classroom teachers said their students' scores on mandated mathematics tests were below their districts' expectations, compared with only one fifth of the teachers of low minority classes. Three quarters of the teachers of high-minority classes agreed that they felt pressure from their districts to improve their students' scores on mandated mathematics tests. Asked about the influences that mandated standardized tests have on their instructional practice, teachers of high-minority classes indicated stronger curriculum effects than did teachers of low-minority classes. Teachers of high-minority classes were more likely to be influenced by mandated tests in their choice of topics and in the emphasis they gave those topics in their mathematics classes. Direction of the effects Although some researchers have tried to determine whether mandated testing is causing a shift in curriculum content, others have tried to discern the direction of the shift. Shepard and Smith21 reported, from interviews with and observations of kindergarten and first-grade teachers, that standardized tests at third and sixth grades have served to fix requirements for the end of the first grade. In a position paper on appropriate guidelines for curriculum content and assessment programs, the National Association for the Education of Young Children and the National Association of Early Childhood Specialists in State Departments of Education22 point to the overemphasis on test results as causing a downward shift in content, so that what used to be taught in first grade is now taught in kindergarten. The impact of such testing has even trickled down into programs for 3- and 4-year-old children. Many of the state testing directors interviewed by Shepard23 emphasized that it is the conscious purpose of state testing pro-
OCR for page 155
Measuring What Counts: A Conceptual Guide for Mathematics Assessment grams to ensure that essential skills are taught. Several recent studies indicate that the state programs are achieving some success in steering instruction towards basic skills. Stake and Theobold24 surveyed 285 teachers in Illinois, Indiana, Minnesota, South Dakota, Washington, Maryland, and North Carolina. When asked for a summary judgment on formal standardized testing, 36 teachers indicated that testing is helpful in many ways and 173 said it is a generally positive factor that is more helpful than harmful. Asked for the single most positive contribution that testing makes in their school, they most often cited the increased time spent teaching basic skills. Corbett and Wilson25 found that 85 percent of the Maryland educators and 30 percent of the Pennsylvania educators surveyed perceived at least a moderate spread of basic skills instruction throughout the curriculum as a result of their state minimum competency testing programs. Nearly 80 percent of the teachers surveyed by Lomax et al.26 either agreed or strongly agreed that mandated testing influences teachers so that they spend more instructional time in mathematics classes on basic skills. Some have concluded from these studies that mandated standardized tests are causing school curricula to move towards an emphasis on basic skills. Archbald and Porter,27 however, are not so sure. They contend that mandated testing, rather than causing instruction to focus on basic skills, is merely consistent with the instructional practice that would take place in any case. Their skepticism is supported by research findings indicating that teachers have a positive view of teaching basic skills. Research by Glasnapp et al.28 found that 89 percent of the Kansas teachers surveyed were satisfied or extremely satisfied with their district's emphasis on basic skills instruction. Even though 86 percent of the teachers who participated in the study by Romberg et al.29 characterized state tests as primarily tests of basic or essential skills, only 31 percent said they placed a greater emphasis on basic skills than they would otherwise. Whether mandated testing programs are the cause or merely a contributing factor, the important point is that the resulting emphasis on basic skills is certainly far from the mathematics curriculum called for by the National Council of Teachers of Mathematics (NCTM) in their Curriculum and Evaluation Standards.30 There is also evidence that an emphasis on problem solving and critical thinking, which is in line with the NCTM Standards, is on
OCR for page 156
Measuring What Counts: A Conceptual Guide for Mathematics Assessment the rise as well. Most of the teachers surveyed by Stake and Theobold,31 215 of 285, report that an increase in emphasis on problem solving and critical thinking has taken place in their schools over the last year or two. When asked what changes they thought were at least partly caused by an emphasis on testing, one of the three changes most frequently noted was a gain in emphasis on problem solving and critical thinking. In the study by Romberg et al.,32 81 percent of the teachers reported that they knew problem-solving items were on the state test. Whereas 20 percent reported placing a greater emphasis on problem solving because of the state test, only 8 percent reported less emphasis. The researchers suspected, however, that the "teachers who consider problem solving to be on the test are probably thinking of simple word problems"33 and do not hold the broader conception of problem solving called for in the NCTM Standards. CURRICULUM ALIGNMENT Leinhardt and Seewald34 referred to the extent to which instructional content matches test content as overlap. They pointed out that teachers are well aware of the notion that the greater the test overlap in their instructional emphasis, the higher their students' test scores are likely to be. The result, according to Resnick and Resnick,35 is that "school districts and teachers try to maximize overlap … by choosing tests that match their curriculum. When they cannot control the test, … they strive for overlap by trying to match the curriculum to the tests, that is, by 'curriculum alignment.'"36 Though no recent studies directly address the extent to which teachers recognize and strive for overlap, various research methods have been used to measure overlap indirectly. For example, Freeman et al.37 conducted year-long case studies of several fourth-grade teachers to analyze their styles of textbook use and to determine how the different styles affected content overlap between the mathematics textbook used and five standardized tests of fourth-grade mathematics. The researchers defined five models of textbook use on the basis of their classroom observations. In every case, a substantial proportion of the problems presented during the teachers' lessons dealt with tested topics.
OCR for page 157
Measuring What Counts: A Conceptual Guide for Mathematics Assessment As noted before, the majority of the Kansas teachers who participated in a 1987 survey38 found the state's minimum competency test objectives to be a valuable guide for their curriculum and reported changing their instructional emphasis accordingly. As teachers strive to maximize overlap, some observers have expressed concern that the curriculum will eventually narrow until instruction and learning are focused exclusively on what is tested.39 However, there is little in the way of research to support the claim that the curriculum is actually narrowing in response to mandated testing programs. Only 16 percent of those same Kansas teachers had seen indications that the school curriculum was being narrowed as a result of the state minimum competency tests.40 In fact, according to Stake and Theobold,41 199 out of 285 teachers surveyed reported that a general broadening of the curriculum had taken place in their schools over the last few years. Perhaps these differences in perspective can be attributed to the different ways in which researchers and teachers interpret "narrowing". To researchers, narrowing refers to teaching to the test. Some teachers, however, appear to interpret "narrowing" as teaching fewer topics. That only a small fraction of Kansas teachers believed that their curriculum had narrowed can perhaps be explained by two other statistics: 45 percent of them reported adding lessons or units to the curriculum as a result of the tests, whereas only 17 percent reported sacrificing instruction in other areas or skills to teach to the state objectives. In response to pressures to alter content, teachers have demonstrated a greater willingness to add topics to the curriculum than to delete them.42 The Corbett and Wilson research43 suggests that a narrower curriculum may be welcomed by some teachers. Even though 64 percent of the Maryland educators surveyed said that there had been at least a moderate narrowing of their school curriculum as a result of the state minimum competency test, 56 percent also reported at least a moderate improvement in the curriculum. In follow-up interviews, Maryland educators said the curriculum was "structured, coordinated, more focused, more defined, sequentially ordered, more systematic, consistent, and created a consciousness (about what was being taught)."44 For these Maryland teachers, narrowing was associated with bringing an unwieldy curriculum under control.
OCR for page 214
Measuring What Counts: A Conceptual Guide for Mathematics Assessment without prior consent from the family.59 The statute was passed in an era in which Congress' privacy concerns were focused on such issues as student grade transcripts, letters of reference, or school psychologist's evaluation reports and the extent to which such information was being used outside a student's school to make potentially damaging judgments about the student. But the language and intent of FERPA apply to certification of mathematics proficiency and perhaps also to the information that may be used as the basis for granting certification, such as a student's performance on a particular assessment task. Given the interests of potential employers in obtaining access to such information, clear privacy protections must be in place. Related to these privacy issues is a concern about the possibility of challenges on religious grounds to the content and assessment of standards. Policymakers should be prepared for the fact that some religious groups will have bona fide objections that standards or assessment techniques interfere with the free exercise of their religious rights or their free exercise of speech. The U.S. Supreme Court recognized exemptions from certain public education requirements for the Amish on the basis of this religious freedom argument.60 Such a challenge could be raised against some of the more fundamental and objective skills, such as higher-level mathematics or technology, as well as some of the more subjective assessment techniques. Assessments could conceivably be designed to identify a student's attitudes toward individual responsibility, sociability, or integrity that embodies religious or cultural biases. It is also possible that assessments could be implemented in a way that curtails an individual's opportunity to engage in free speech, such as might occur if the assessments of the SCANS "sociability" or "works with diversity" skills were applied to favor behavior that is in the currently popular terminology, "politically correct''. The primary goal of the free speech clause of the First Amendment to the Constitution is, after all, the protection of all expressions of a point of view, even the most politically unpopular.61 EQUITY AND THE GOVERNANCE OF EDUCATION With the proposal from some to establish a national assessment system that would truly be national, not federal, current reform initiatives acknowledge the long-standing tradition of state control of education. At the same time, however, the national reform move-
OCR for page 215
Measuring What Counts: A Conceptual Guide for Mathematics Assessment ment encourages conformity in curriculum content, performance goals, and standards of assessment across states and localities. The Tenth Amendment to the U.S. Constitution requires that "the powers not delegated to the United States by the Constitution, nor prohibited by it to the states, are reserved to the states respectively, or to the people." Because education is not a power specifically given to the federal government, this doctrine of enumeration may be seen as barring efforts to create a federally mandated system of standards for states and localities. However, one power which is explicitly given the federal government is the power to regulate interstate and foreign commerce. A proposal to use assessment to reform the educational preparation of workers who will participate in interstate commerce may fall within the purview of the federal government's constitutional powers to regulate commerce. Given the breadth and depth of political enthusiasm for national education reform, the so-called "state's rights" concerns may be minimal, particularly in comparison to some of the other issues set forth in this paper. Further, there are any number of federal initiatives that have withstood scrutiny under the Tenth Amendment and have been vigorously enforced by the federal courts against the states under the terms of other constitutional provisions. For example, the Fourteenth Amendment's Due Process and Equal Protection Clauses have been used numerous times to enforce a national policy goal. In the education context, the most notable of these were the school desegregation cases, many of which inquired deeply into matters of local school curriculum and instruction. The failure to adopt a federal system of curriculum standards and assessments presents the potential for fifty different sets of issues concerning validity, reliability, and fairness, with the ensuing possibility of fifty different sets of legal problems. If implementation is local, rather than at the state level, each local district could confront its own set of potential legal difficulties. These legal problems are accentuated whenever assessments are used for high-stakes decisions related to high school graduation, college admission, continuing education, and certification for employment. The technical problems inherent in a proposed system of high-stakes assessment are substantial. The potential legal difficulties and the enormity of the policy questions concerning such a proposal would urge great caution on the part of the proponents of such a system. A laudable goal such as that of the National Council on Education Standards and Testing (NCEST) to create a system of "tests worth teaching
OCR for page 216
Measuring What Counts: A Conceptual Guide for Mathematics Assessment to"62 can be lost among all of the other possible goals for the program, including: improving classroom instruction; improving learning outcomes for all students; informing students, parents, and teachers about student progress; measuring and holding students, schools, school districts, states, and the nation accountable for educational performance; assisting education policy-makers with programmatic decisions; certifying for future employment; credentials for college admission; etc.63 The pursuit of multiple reform goals puts multiple pressures on both the psychometricians and preventive law specialists contemplating the manner in which the potential for challenges to such endeavors might be mounted. The possibility of having a separate set of practices in each governmental entity choosing to implement the program (not to mention any employer participating in the use of any resulting certification), creates the possibility for numerous legal challenges. Further, in any such legal challenge, the national or federal bodies with whom the localities are working might also face the risk of involvement. In short, it may be much wiser in the long run, particularly given that a truly national approach to these problems is being sought, to simply make this a federal effort and abandon the pretext of state and local control. Several commentators have already noted that all of the America 2000 initiatives are moving us inexorably toward a national curriculum.64 EQUITY AND ECONOMICS Another set of potential legal issues centers around school finance and the current inequities from district to district and building to building in financial resources for education. Related to this is the potential impact of assessment information as a part of the inquiry in discussions of state takeovers of low-performing or educationally bankrupt school districts. In the past several years, state governments have become more willing to expand their exercise of responsibility for oversight of local education efforts. Some states, such as New Jersey, have implemented receiverships for certain low-performing districts. If mathematics assessment information or other educational accountability reports begin to inform state-level reviews of local district educational achievement, then such variables as the mathematics assessment will come to have very high stakes consequences not only for students but also for local school districts. As a result, such efforts might be subject to
OCR for page 217
Measuring What Counts: A Conceptual Guide for Mathematics Assessment local district challenge on federal constitutional bases concerning due process and equal protection; they would, in addition, invite a broad array of state constitutional and statutory challenges concerning financing of education. Local district or individual school challenges to the use of assessment results might also be mounted under any state or federal school choice scheme in which assessment data could be used to limit a student's opportunity to attend a particular school. Another practical issue, and one fraught with legal difficulties of another sort, is the disturbing question of how to pay for these ambitious initiatives. In his powerful reflection Savage Inequalities, Jonathan Kozol warns that the decision in Brown v. Board of Education "did not seem to have changed very much for children in the schools I saw, not, at least, outside of the Deep South"65 and that "the dual society, at least in public education, seems in general to be unquestioned."66 Further, most of the urban and less-affluent suburban schools he visited were untouched by school reform initiatives and, in the few instances where some reform initiatives had been tried, they amounted to little more than ''moving around the same old furniture within the house of poverty… In public schooling, social policy has been turned back almost one hundred years."67 At the core of all of these inequities, he finds, is a system of public finance of education which subsidizes and perpetuates these gross denials of educational opportunity. The Implementation Task Force of the National Council for Education Standards and Testing suggests that equitable distribution of resources among districts and among schools within districts is a critical component for implementation at each level of government.68 That group recognized that equity in funding is a key factor in the success of the endeavor69 and will become a major issue in all of the states.70 Federal programs in the past have been critical in providing assistance for the educationally disadvantaged. Such endeavors will need to continue but should be linked tightly to the common content and performance standards.71 NCEST in some respects seems to dismiss problems related to fiscal equity, hoping instead that national standards can create targets toward which educators can strive.72 NCEST argues that states and local districts could work together to overcome deficiencies in resources.73 Given the substantial difficulties that even one state, Texas, has had attempting to arrive at an equalization formula to
OCR for page 218
Measuring What Counts: A Conceptual Guide for Mathematics Assessment address constitutional deficiencies with school funding, this seems an excessively optimistic position.74 Participants in the reform debate must maintain constant awareness of the possibility of unintended legal consequences. Once government defines minimum educational outcomes for all students and creates a presumption that sufficient educational services will be provided so that all students can meet this level of proficiency, then it may create an entitlement to an education that the federal courts have never previously been in a position to recognize for constitutional protection. In San Antonio School District v. Rodriguez,75 the U.S. Supreme Court refused to recognize that education is a fundamental right under the Constitution; however, if a fundamental right is, in essence, created as the result of the creation of an entitlement, then the level of judicial scrutiny of a governmental practice may be subject to the burdensome "strict scrutiny" level of analysis of practices that work to deny citizens' fundamental interests, a burden nearly impossible for government to meet. A related issue concerns the fact that the government will have created a legitimate expectation on the part of students that school attendance will result in attainment of a certain level of mathematics skills. This also creates a need to assess whether the doors previously closed by state court judges to claims of "educational malpractice" may be wedged open again as a result of the new national standards and goals.76 William Clune identifies four generic problems confronting efforts to enhance student achievement: poor understanding of effective practice (weak technology, that is, lack of understanding of which practices produce improved learning); serious problems of policy implementation (central control can do little to affect the activities of millions of teachers and learners across the nation); serious problems of political organization and policy formation (effective educational policy must be carefully designed and tightly coordinated); and significant cost constraints (massive infusions of new capitol would be needed to subsidize major change).77 One cannot hope, Clune asserts, to successfully pursue strong educational goals through the use of weak policy instruments; he views efforts at reform through the use of educational indicators and assessments as requiring, in particular, further development of
OCR for page 219
Measuring What Counts: A Conceptual Guide for Mathematics Assessment assessments that are technically defensible and efforts to influence instructional content and practice as requiring tighter systems to guide instruction, perhaps including teacher education and systems to maximize teacher participation, enthusiasm, and responsibility, and greater focus on curriculum to promote higher-order thinking and problem solving. Each of these concerns has an analog in the legal issues discussed above. Without a satisfactory solution to each of these problems, the legal consequences could be substantial. In particular, specific attention must be paid to the impact of these proposals on educationally disadvantaged students. From a policy perspective, issues of equity should be of the utmost importance. From a legal perspective, it may be those who have traditionally been the most educationally disadvantaged who will be able to bring the most successful legal challenges to the endeavor. From an economic perspective, a failure to effectively address the needs of all students will have devastating consequences for the future economic welfare of the entire nation. CONCLUSION This paper provides a brief summary of the principal legal and policy issues that might arise in challenges to a mathematics assessment initiative by members of protected groups traditionally underserved by the nation's schools, by any student who performs poorly on an assessment, or by individual school districts. Enhanced educational attainment in mathematics is a goal with which few could disagree. However, educators and public policymakers should take care that all schools are provided sufficient resources to allow them to effectively meet that goal and that all students, no matter their race, ethnicity, language, background, or handicapping condition are given a fair opportunity to learn and a fair opportunity to demonstrate their learning through assessments. Finally, without an adequate system of financing mathematics education and assessment in all schools, no effort at education reform will succeed.
OCR for page 220
Measuring What Counts: A Conceptual Guide for Mathematics Assessment ENDNOTES 1 This paper was considerably influenced by a previous paper by the author commissioned by the Secretary's Commission on Achieving Necessary Skills of the U.S. Department of Labor. 2 Brown v. Board of Education, 347 U.S. 483 (1954). 3 Debra P. v. Turlington, 474 F. Supp. 244 (M. D. Fla. 1979); aff'd in part, rev'd in part, 644 F. 2d 397 (5th Cir. 1981); reh. en banc den. 4 474 F. Supp. at 247. 5 Id. at 249. 6 42 U.S.C. 2000d. 7 474 F. Supp. at 252. 8 Id. citing Armstead v. Starkville Municipal Separate School District, 461 F. 2d 276 (5th Cir. 1972). 9 644 F. 2d 397, at 400. 10 Id. at 402. 11 See text accompanying endnotes 2-6. 12 644 F. 2d 397, 406-407. 13 644 F. 2d at 407. 14 Although the latter is not a basis for a legal claim in most circumstances, it correlates with race and ethnicity and may thus result in a basis for a legal challenge. 15 An item from the February 1991 Maryland School Performance Assessment Program Grade 8 Mathematics Assessment teacher's guide involves a task asking students to develop a survey plan to collect information on potential respondents to assist a developer's efforts to build a new restaurant. The lowest-scoring sample student answer is "1 would ask people in the rich part of the county." Without doubt, that response lacks a richness of detail that reflects much understanding of sampling methodology even at the eighth grade level, but for a low-income student who could never contemplate having the opportunity to be a developer, the sample answer says it all. 16 508 F. 2d 1017 (5th Cir. 1975). 17 See also, Hobson v. Hansen, 269 F. Supp. 401 (D.D.C 1967), aff'd sub nom Smuck v. Hansen, 408 F. 2d 175 (D.C. Cir. 1969) (en banc). 18 Lorry P. v. Riles, 343 F. Supp. 1306 (N. D. Cal. 1972), aff'd 502 F. 2d 963 (9th Cir. 1974). 19 There are also issues concerning both education and employment of persons with limited English proficiency (LEP); these issues are not addressed here on the assumption (perhaps erroneous) that courts will find English proficiency requirements quite acceptable for the nation's future workplaces. However, even if this assumption is true, there is another set of legal issues, unaddressed here, concerning the rights of LEP students to education that meets their special needs.
OCR for page 221
Measuring What Counts: A Conceptual Guide for Mathematics Assessment 20 What Work Requires of Schools: A SCANS Report for America 2000. The Labor Secretary's Commission on Achieving Necessary Skills, U.S. Department of Labor (hereinafter SCANS), 1991, p. 24. 21 SCANS, p. 25. 22 SCANS, pp. 26-27. 23 SCANS, p. 27. 24 42 U.S.C. 2000e - 2(a)(1). 25 42 U.S.C. 2000e - 2(h). 26 Griggs v. Duke Power Co., 401 U.S. 424 at 431 (1971). 27 See P. Patterson, "Employment Testing and Title VII of the Civil Rights Act of 1964" in Gifford and O'Connor, pp. 93-95. 28 Albemarle Paper Co. v. Moody, 422 U.S. 405 at 425 (1975). 29 Griggs and Albemarle. 30 Albemarle, at 431. 31 Albemarle, op. cir. 32 See B. Schlei and P. Grossman, Employment Discrimination Law (1983), pp. 98-161 and 1985 Supp. p. 18; See Test Policy and the Politics of Opportunity Allocation: The Workplace and the Law, B. Gifford, ed, Klover, Boston (1989). 33 422 U.S. at 434. 34 422 U.S. at 435. 35 Note also that to the extent that proposals may be implemented in a manner not analogous to a scored test, but rather as a less uniform assessment according to subjective criteria, Title VII is applicable and the Griggs standard is followed. See Schlei, B. L. & Grossman, P. (1983). Employment Discrimination Law (2nd ed.). Washington, DC: Bureau of National Affairs, Inc. pp. 162-190, and 1993-84 Cum-Supp., pp. 21-23. 36 P.L. 102-166. 37 Senate sponsors of the law, including Senators Danforth, Kennedy, and Dole, and the administration created a specific legislative history for the law stating that the terms "business necessity" and "job-relatedness" are intended to reflect the concepts enunciated by the Supreme Court in Griggs v. Duke Power Co., 401 U.S. 424 (1971), and in the other Supreme Court decisions prior to Wards Cove Packing Co. v. Atonio, 490 U.S. 642 (1989). When a decision-making process includes particular, functionally integrated practices that are components of the same criterion, standard, method of administration, or test, such as the height and weight requirements designed to measure strength in Dothard v. Rawlinson, 433 U.S. 321 (1971), the particular, functionally integrated practices may be analyzed as one employment practice." 38 P.L. 102-166, Sec. 106. 39 29 U.S.C. 701 et. seq. 40 42 U.S.C. 12101 et. seq.
OCR for page 222
Measuring What Counts: A Conceptual Guide for Mathematics Assessment 41 20 U.S.C. 1400 et. seq. 42 20 U.S.C. 1681-1687, as amended by the Civil Rights Restoration Act of 1987, codified at 20 U.S.C. 1687. 43 34 C.F.R. 86.1-86.70. 44 See, for example, Massachusetts General Laws Ann. ch. 76, sec. 5. 45 42 U.S.C. 2000e. 46 See K. Connor and E. Vargyas, "The Legal Implications of Gender Bias in Standardized Testing," Berkeley Women's Law Journal, (1992), pp. 13-89 for an excellent analysis of gender discrimination law as it applies to testing. 47 Id. 48 Franklin v. Gwinnett County Public Schools, 112 S. Ct. 1028 (1992). 49 644 F. 2d at 403, emphasis in original. 50 644 F. 2d 397, 404. 51 See Meyer v. Nebraska, 262 U.S. 390 (1923). 52 See Wisconsin v. Yoder, 406 U.S. 205 (1972). 53 See West Virginia State Board of Education v. Barnette, 319 U.S. 624 (1943). 54 Brookhart v. Peoria, 697 F. 2d 182 (7th Clr. 1982). See Anderson v. Banks, 520 F. Supp. 472 (S. D. Ga., 1981), appeal from subsequent order dismissed sub nom. Johnson v. Sikes, 730 F. 2d 644 (11th Cir. 1984). 55 E. Baker and R. Stites, "Trends in Testing in the USA" in The Politics of Curriculum and Testing, S. H. Fuhrman and B. Malen (eds.) (1991), pp. 148-149. 56 "Court Orders Testing Service to Release Disputed Scores": The Chronicle of Higher Education, September 2, 1992. 57 Raising Standards for American Education: A Report to Congress, the Secretary of Education, the National Education Goals Panel, and the American People. The National Council on Education Standards and Testing (hereinafter NCEST), Washington D.C. 1992, p. 7. 58 Merriken v. Cressman, 364 F. Supp. 913 (E. D. PIL. 1973). 59 20 U.S.C. 1232g et. seq.; 34 C.F.R. Part 99. 60 Wisconsin v. Yoder, 406 U.S. 205 (1971). 61 L Tribe, American Constitutional Law (2nd ed.) (1988), Mineola, NY: The Foundation Press, Inc. pp. 785-1061; M. Yudof, D. Kirp, T. VanGeel, and B. Levin, Educational Policy and the Law (2nd ed.), (1982), Berkeley, CA: McCutchan Publishing. pp. 205-212. 62 NCEST, p. 6. 63 NCEST, pp. 5, 6. 64 E. Baker and R. Stites (1991). Trends in testing in the USA. Politics of Education Association yearbook 1990. (p. 152) London: Taylor & Francis.
OCR for page 223
Measuring What Counts: A Conceptual Guide for Mathematics Assessment 65 J. Kozol, Savage Inequalities: Children in America's Schools. New York, Crown Publishers, Inc. (1991), p. 3. 66 Id. p. 4. 67 Id. 68 NCEST Implementation Task Force, p. G-7. 69 Id. p. G-13. 70 Id. 71 NCEST Standards Task Force Report, p. E-13. 72 Id. p. E-15. 73 Id. 74 Lonnie Harp, "Texas Finance Bill Signed Into Law, Challenges Anticipated," Education Week, 9 june 1993; Lonnie Harp, "Impact of Texas Finance Law, Budget Increase Gauged," Education Week, 16 June 1993; Millicent Lawton, "Alabama Judge Sets October Deadline for Reform Remedy," 23 June 1993. 75 411 U.S. 1 (1972). 76 See e.g., E. T. Connors, Educational Tort Liability and Malpractice, 1981, pp. 148-158. 77 W. Clune, "Educational policy in a situation of uncertainty; or, how to put eggs in different baskets," in Fuhrman and Malen, op. cir., pp. 132-133.
OCR for page 224
Measuring What Counts: A Conceptual Guide for Mathematics Assessment This page in the original is blank.
Representative terms from entire chapter: