Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 1
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Executive Summary Americans have adopted a reform agenda for their schools that calls for excellence in teaching and learning (Goals 2000: Educate America Act of 1994, P.L. 103–227; Improving America’s Schools Act, 1994, P.L. 103–328; Council of Chief State School Officers, 1998). School officials across the nation are hard at work targeting instruction at high levels for all students. Gaps remain, however, between the nation’s educational aspirations and student achievement (Jencks and Phillips, 1998; Rothstein, 1998; U.S. Department of Education, 1996b, 1997a, 1997b, 1997c, 1998b, 1999a). To address these gaps, policy makers have recently focused on the qualifications of teachers and the preparation of teacher candidates (National Commission on Teaching and America’s Future, 1996; U.S. Department of Education, 1999b). Increasingly, states are testing candidates who want to become teachers. Forty-two states require teacher candidates to pass one or more tests to earn a license. States use licensure tests for admission to teacher education, as a condition of graduation, and for the initial licensure of teachers. Failure rates on current tests are not insubstantial, particularly for racial/ethnic minority candidates. These tests have significant consequences for teacher candidates and potentially for America’s students and schools. Teacher licensure tests also play a critical role in the recently enacted Teacher Quality Enhancement Grants for States and Partnerships (Title II). The law seeks to improve teacher quality. One of the goals of the new law is to use teacher licensure tests to hold higher education institutions and states accountable for the quality of teacher preparation and licensing. The law requires states to issue report cards on their tests and licensure policies, to identify low-performing
OCR for page 2
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality teacher preparation programs, to report statewide passing rates on licensure tests for their teacher candidates, and to report the numbers of individuals teaching who have waivers of state licensure requirements. States are also required to report the passing rates of candidates at each teacher education institution and, based on these, to rank teacher education programs or place them in quartiles. The law requires teacher education institutions to report on the quality of their teacher preparation programs, to report their students’ passing rates on state teacher licensure tests, to compare the institutions’ passing rates to the state’s average passing rates, and to indicate whether their programs have been designated as low performing. The law’s requirements create a mechanism that could limit federal funding to teacher preparation programs from which states have withdrawn approval or funding because the programs are low performing. The law prohibits these programs from enrolling students who receive federal financial aid under Title IV of the Higher Education Act. The institutions also are ineligible to receive professional development funds under the law. The law’s provisions have met with notable resistance from some states and higher-education institutions that question whether its requirements provide a sound basis for determining the quality of teacher preparation and licensure programs. THE COMMITTEE’S CHARGE The importance of efforts to improve teaching quality and the difficulty of this work recently led the U.S. Department of Education to request that the National Academy of Sciences empanel the Committee on Assessment and Teacher Quality under the aegis of the Board on Testing and Assessment. In early 1999 the committee was asked to examine the appropriateness and technical quality of teacher licensure tests currently in use and to consider alternatives for developing and assessing beginning teacher competence. The committee also examined the merits of using licensure test results to hold institutions of higher education and states accountable for the quality of teacher preparation and licensure. CONCLUSIONS AND RECOMMENDATIONS Do Current Tests Measure Beginning Teacher Competence Appropriately and in a Technically Sound Way? Defining Competent Beginning Teaching Definitions of what teachers should know and be able to do have changed over time as society’s values have changed, and they will continue to do so. The job of teaching students to learn and use new information, develop and apply
OCR for page 3
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality skills, and think critically is highly complex and demanding. Teachers need to motivate and engage all students, including students from varied backgrounds and those with different learning and language needs. In addition to being responsible for student learning, teachers are expected to provide safe and nurturing classrooms, to serve as good role models, and to engage parents and the community in the business of their schools. Teachers need a wide range of knowledge, skills, abilities, and dispositions to perform these many complex tasks. There is no single agreed-upon definition of what competencies a beginning teacher should have. Different professional organizations and many states have recently developed standards for teachers. The fact that different states have affiliations with these national and regional standards development efforts suggests some agreement between states about standards for teacher competence. Given that states have different educational standards for students, have teacher candidate pools with different characteristics, and that licensing of teachers is a state responsibility, it is not surprising that there is some variation in the knowledge and skills that states seek for beginning teachers. Designing Tests for Initial Licensure The primary goal of licensing beginning teachers is to ensure that all students have competent teachers. Teacher licensing is under the authority of individual states. There are 51 unique licensure systems in the United States; they share some commonalties, however. As in other professions, teacher licensing relies on more than tests to judge whether candidates have the knowledge, skills, abilities, and dispositions to practice responsibly. Teacher candidates generally must fulfill education requirements, successfully complete practice teaching, and receive the recommendations of their preparing institutions. These requirements help ensure that a broad range of competencies are considered in licensing new teachers. Initial teacher licensure tests are designed to identify candidates with some of the knowledge and skills needed for minimally competent beginning practice. The tests currently used measure basic skills, general knowledge, content knowledge, and knowledge of teaching strategies. They are designed to separate teacher candidates who are at least minimally competent in the areas assessed from those who are not. Initial teacher licensure tests do not provide information to distinguish moderately qualified from highly qualified teacher candidates nor are they designed to test all of the competencies relevant to beginning practice. States decide whether and what tests to use to license beginning teachers. Each of the 42 states that require tests uses a different combination of them, uses them at different points in a candidate’s education, and sets its own passing scores. Several hundred different initial licensure tests are currently in use. Two
OCR for page 4
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality test developers—Educational Testing Service (ETS) and National Evaluation Systems (NES)—develop the vast majority of these tests. Conclusions Because a teacher’s work is complex, even a set of well-designed tests cannot measure all of the prerequisites of competent beginning teaching. Current paper-and-pencil tests provide only some of the information needed to evaluate the competencies of teacher candidates. States have gradually adopted tests for teacher licensure, and test developers have made various tests available over time. Therefore, it is not surprising that states have adopted a variety of tests to license beginning teachers. Appropriate, technically sound tests are difficult and costly to develop. Collaborations among states participating in the Interstate New Teacher Assessment and Support Consortium and other states, professional associations, and test developers bring the intellectual and financial resources of several organizations to this difficult work. Recommendations It is crucial that states use multiple forms of evidence in making decisions about teacher candidates. Licensure systems should be designed to rely on a comprehensive but parsimonious set of high-quality indicators. States, test developers, and professional organizations should continue exploring joint development of initial teacher licensing tests for the knowledge and skill areas they have in common. Federal and state governments and private organizations should appropriate funds to support this kind of collaboration. Making Decisions About Candidates Based on Licensure Tests States set passing scores on licensure tests based on judgments about the levels of knowledge and skill needed for minimally competent beginning teaching. Although many states rely on commonly used standard-setting procedures, there is little documentation about these procedures and how states actually use this information in arriving at a final decision about passing scores. In attempts to raise teacher standards, some states have recently raised their passing scores on particular tests. Some report having set passing scores that are higher than those of other states. On all of the tests the committee reviewed, minority candidates had lower passing rates than nonminority candidates on their initial testing attempts. Though differences between the passing rates of candidate groups eventually decrease because many unsuccessful test takers retake and pass the tests, eventual passing rates for minority candidates are still lower than those for nonminority test takers.
OCR for page 5
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Initial licensure tests are only one factor influencing the supply of new teachers. The quality and size of the pool of new teachers depend on many things, including recruiting efforts, other licensing requirements, labor market forces, licensing reciprocity, teacher salaries, and the conditions under which teachers work. Conclusions States differ in how high they set passing scores. The committee does not know the extent to which this variation in passing scores reflects differences among states in standard-setting methods; state teaching and learning standards; the characteristics of applicant pools; or different concerns about measurement error, teacher quality, or teacher supply. To the extent that the tests provide accurate measurements, setting higher passing scores would be expected to increase the proportion of teacher candidates in the hiring pool who are competent in the knowledge and skills measured by the tests, although higher passing scores will tend to lower the number of candidates who pass the test. To the extent that test scores have measurement error, setting higher passing scores could eliminate competent candidates. Reducing the number of new licensed teachers could require districts to make difficult choices, such as hiring uncredentialed teachers, increasing class sizes, or increasing salaries to attract licensed teachers from other districts and states. The lower passing rates for minority teacher candidates on current licensure tests pose problems for schools and districts in seeking a qualified and diverse teaching force. Setting substantially higher passing scores on licensure tests is likely to reduce the diversity of the teacher applicant pool, further adding to the difficulty of obtaining a diverse school faculty. Recommendations States should follow professionally accepted standard-setting methods and document the methods they use to set passing scores on initial licensure tests. This documentation should describe the work of standard-setting panels and the basis on which policy decisions were made by the officials setting the final passing scores. Documentation should be publicly available to users and other interested parties. If states raise passing scores as a way to increase the competence of new teachers, they should examine not only the impact on teacher competence but also the effects of raising passing scores on applications to teacher education programs, on the supply of new teachers, and on the diversity of the teaching force.
OCR for page 6
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Evaluating Licensing Tests Solid technical characteristics and fairness are key to the effective use of tests. The work of measurement specialists, test users, and policy makers suggests criteria for judging the appropriateness and technical quality of initial teacher licensure tests. The committee drew on these to develop criteria it believes users should aspire to in developing and evaluating initial teacher licensure tests. The committee used these evaluation criteria to evaluate a sample of five widely used tests produced by ETS. The tests the committee reviewed met most of its criteria for technical quality, although there were some areas for improvement. The committee also attempted to review a sample of NES tests. Despite concerted and repeated efforts, though, the committee was unable to obtain sufficient information on the technical characteristics of tests produced by NES and thus could draw no conclusions about their technical quality. Conclusions The committee’s criteria for judging test quality include the following: tests should have a statement of purpose; systematic processes should be used in deciding what to test and in assuring balanced and adequate coverage of these competencies; test materials should be tried out and analyzed before operational decisions are made; test administration and scoring should be uniform and fair; test materials and results should be protected from corruptibility; standard-setting procedures should be systematic and well documented; test results should be consistent across test forms and scorers; information about tests and scoring should be available to candidates; technical documentation should be accessible for public and professional review; validity evidence should be gathered and presented; costs and feasibility should be considered in test development and selection; and the long-term consequences of licensing tests should be monitored and examined. The profession’s standards for educational testing say that information sufficient to evaluate the appropriateness and technical adequacy of tests should be made available to potential test users and other interested parties. The committee considers the lack of sufficient technical information made available by NES and the states to evaluate NES-developed tests to be problematic and a concern. It is also significant because NES-developed tests are administered to very large numbers of teacher candidates. The initial licensure tests currently in use rely almost exclusively on content-related evidence of validity. Few, if any, developers are collecting evidence about how test results relate to other relevant measures of candidates’ knowledge, skills, and abilities. It is important to collect validity data that go beyond content-related
OCR for page 7
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality validity evidence for initial licensing tests. However, conducting high-quality research of this kind is complex and costly. Examples of relevant research include investigations of the relationships between test results and other measures of candidate knowledge and skills or on the extent to which tests distinguish candidates who are at least minimally competent from those who are not. The processes used to develop current tests, the empirical studies of test content, and common-sense analyses suggest the importance of at least some of what is measured by these initial licensure tests. Beginning teachers should know how to read, write, and do basic mathematics; they should know the content areas they teach. Little research has been conducted on the extent to which scores on current teacher licensure tests relate to other measures of beginning teacher competence. Much of the research that has been conducted suffers from methodological problems that interfere with making strong conclusions about the results. This makes it hard to determine what effect licensure tests might have on improving the actual competence of beginning teachers. Recommendations States should strive to use the committee’s or similar evaluation criteria when developing and evaluating tests for use in initial teacher licensure systems. When states are selecting from among existing tests for initial teacher licensure, they should obtain and carefully consider field test and any available operational data regarding reliability, validity, cost/feasibility, and fairness as part of their decision-making process. When states are developing licensing tests, they should collect and weigh this evidence in making decisions about the final form of the test and its use. The degree of disparate impact should be an important consideration when states are deciding which licensure test to use for various decisions about candidate competence. States and test developers should provide technical documentation to users, scholars, and the public about the reliability, validity, and disparate impact of their tests. Field test or any available operational data should be used to document the technical quality of tests. The technical information used in deciding on a test normally should be made publicly available before tests are actually used to make decisions about individuals. If technical data are not provided by the end of the first administration year, states should not use information from these tests to make licensing decisions. State agencies contracting for test development should specify the technical information they require. Because of the importance of these data
OCR for page 8
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality for the technical evaluation of the tests, state agencies should request sufficient technical data and timely delivery of documentation. State agencies should ensure their clear authority to release this information to users and others for the purpose of objectively evaluating the technical quality of the tests, the resulting scores, and the interpretations based on them. States should arrange for independent evaluations of their current tests and teacher licensure systems and make the results of these independent examinations of their systems available for outside review. The committee encourages the federal government and others to conduct research on the extent to which teacher licensure tests distinguish between beginning teachers who are at least minimally competent and those who are not regarding the knowledge and skills the tests are intended to measure. This research should include evidence on a broad range of teacher competencies. Such research is likely to improve the development of teacher licensure tests. Within the limits of privacy law, states should make their raw data available to the research community to facilitate development and validity research on initial teacher licensure tests. Should Teacher Licensure Tests Be Used to Hold States and Institutions of Higher Education Accountable for the Quality of Teacher Preparation and Licensure? Holding Programs Accountable A new federal law called the Teacher Quality Enhancement Grants for States and Partnerships (Title II) was enacted in 1998 to achieve four goals: to improve student achievement; to improve the quality of the current and future teaching force by improving the preparation of prospective teachers and enhancing professional development activities; to hold institutions of higher education accountable for preparing beginning teachers to have the necessary teaching skills and to be highly competent in the academic content areas in which they plan to teach; and to recruit highly qualified individuals, including individuals from other occupations. Conclusions It is reasonable to hold teacher education institutions accountable for the quality of their teacher preparation programs. By their design and as currently used, initial teacher licensure tests fall short of the intended policy goals for their use as accountability tools and as levers for improving teacher preparation and licensing programs. The public reporting and accountability provisions of Title II may encourage erroneous conclusions about the quality of teacher preparation.
OCR for page 9
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Although the percentage of graduates who pass initial licensure tests provides an entry point for evaluating an institution’s quality, simple comparisons among institutions based on their passing rates are difficult to interpret for many reasons. These include the fact that institutions have different educational missions and recruiting practices, their students have different entry-level qualifications, teacher education programs have different entry and exit testing requirements, and programs have different procedures for determining the institutional affiliations of their candidates. By themselves, passing rates on licensure tests do not provide adequate information on which to judge the quality of teacher education programs. Simple comparisons of passing rates across states are misleading. Many states use different tests for initial licensure or set different passing scores on the same tests. States have different policies about when a test is given or what decisions it supports. To fairly and accurately judge the quality of teacher education programs, federal and state officials need data on a wide variety of program characteristics from multiple sources. Other indicators of program quality might include assessment data for students in relation to course and program benchmarks, employer evaluations, and district or state evaluations of beginning teaching. Other indicators might include information on course requirements and course quality, measures of the amount and quality of field experiences, and evidence of opportunities to work with students with special learning needs and students with diverse backgrounds. Data on the qualifications of program faculty, the allocation of resources, and the adequacy of facilities might be considered. The qualifications of students at entry to teacher education programs also should be included. Recommendations States should not use passing rates on initial licensure tests as the sole basis for deciding whether their teacher education programs are low performing. States should report multiple indicators of the quality of teacher preparation programs to federal officials in complying with Title II. The federal government should not use passing rates on initial teacher licensing tests as the sole basis for comparing states or teacher education programs or for withholding funds, imposing other sanctions, or rewarding teacher education programs. Federal officials should continue to collect the state and school data required by Title II but should not withhold funds from, otherwise sanction, or reward programs until a study is mounted of the multiple and varied data that might be used to judge the quality of teacher preparation and licensure.
OCR for page 10
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality How Can Innovative Measures of Beginning Teacher Competence Help Improve Teacher Quality? Examining New Assessments Several new and developing teacher assessment systems use a variety of testing and assessment methods, including assessments of teaching performance. These include multiple measures of candidates’ knowledge, skills, abilities, and dispositions. In these systems, assessments are integrated with professional development and with ongoing support of prospective or beginning teachers. Conclusion New and developing assessment systems warrant investigation for addressing the limits of current initial teacher licensure tests and for improving teacher licensure. The benefits, costs, and limitations of these systems should be investigated. Recommendations Research and development of broad-based indicators of teacher competence, not limited to test-based evidence, should be undertaken; indicators should include assessments of teaching performance in the classroom, of candidates’ ability to work effectively with students with diverse learning needs and cultural backgrounds and in a variety of settings, and of competencies that more directly relate to student learning. When initial licensure tests are used, they should be part of a coherent developmental system of preparation, assessment, and support that reflects the many features of teacher competence.
Representative terms from entire chapter: