Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 163
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality 9 Conclusions and Recommendations In this report the committee describes recent efforts by teacher educators, state officials, and federal policy makers to improve teacher preparation and strengthen initial teacher licensure. The committee notes that states are increasingly testing candidates for their ability to become teachers and the federal government is looking to licensure tests for leverage in changing teacher education and improving teacher quality. In the preceding chapters, the committee reviews programs that use written tests to determine whether entry-level teachers have the skills and knowledge minimally necessary for beginning teaching. The report describes a new federal law called the Teacher Quality Enhancement Grants for States and Partnerships that uses teacher licensure tests to hold teacher education programs and states accountable for the quality of their preparation and licensure systems. In this report the committee examines three questions: Do current tests measure beginning teacher competence appropriately and in a technically sound way? Should teacher licensure tests be used to hold states and institutions of higher education accountable for the quality of teacher preparation and licensure? How can innovative measures of beginning teacher competence help improve teacher quality? In this last chapter, the committee reiterates the conclusions of its analysis and presents a series of recommendations for policy makers, teacher testers, and licensure officials. The first set of recommendations speaks to the use of the
OCR for page 164
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality current tests. The second set is about evaluating teacher education, and the final recommendations encourage new and innovative assessments. DO CURRENT TESTS MEASURE BEGINNING TEACHER COMPETENCE APPROPRIATELY AND IN A TECHNICALLY SOUND WAY? The committee examined four types of data in response to this question; it looked at practice and research data on the knowledge, skills, abilities, and dispositions that competent teachers demonstrate; information about current licensing systems and licensing tests; evaluation data for several widely used teacher licensure tests; and an investigation of the extent to which tests can and cannot improve teacher competence and supply. Defining Competent Beginning Teaching Definitions of what teachers should know and be able to do have changed over time as society’s values have changed, and they will continue to do so. The job of teaching students to learn and use new information, develop and apply skills, and think critically is highly complex and demanding. Teachers need to motivate and engage all students, including students from varied backgrounds and students with different learning and language needs. In addition to being responsible for student learning, teachers are expected to provide safe and nurturing classrooms, to serve as good role models, and to engage parents and the community in the business of their schools. Teachers need a wide range of knowledge, skills, abilities, and dispositions to perform these many complex tasks. The quality of teaching in a school depends on more than just teacher quality. Quality teaching depends on a number of factors, including the amount and quality of instructional resources available, teacher professional development, staffing, and support from administrators and parents. There is no single agreed-upon definition of what competencies a beginning teacher should have. Different professional organizations and many states have recently developed standards for teachers. The fact that different states have affiliations with these national and regional standards development efforts suggests some agreement between states about standards for teacher competence. Given that states have different educational standards for students, have teacher candidate pools with different characteristics, and that licensing of teachers is a state responsibility, it is not surprising that there is some variation in the knowledge and skills that states seek for their beginning teachers.
OCR for page 165
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Designing Tests for Initial Licensure The primary goal of licensing beginning teachers is to ensure that all students have competent teachers. Teacher licensing is under the authority of individual states. There are 51 unique licensure systems in the United States; they share some commonalties, however. As in other professions, teacher licensing relies on more than tests to judge whether candidates have the knowledge, skills, abilities, and dispositions to practice responsibly. Teacher candidates generally must fulfill education requirements, successfully complete practice teaching, and receive the recommendations of their preparing institutions. These requirements help ensure that a broad range of competencies are considered in licensing new teachers. Initial teacher licensure tests are designed to identify candidates with some of the knowledge and skills needed for minimally competent beginning practice. The tests currently used measure basic skills, general knowledge, content knowledge, and knowledge of teaching strategies. They are designed to separate teacher candidates who are minimally competent in the areas assessed from those who are not. Initial teacher licensure tests do not provide information to distinguish moderately qualified from highly qualified teacher candidates nor are they designed to test all of the competencies relevant to beginning practice. States decide whether and what tests to use to license beginning teachers. Each of the 42 states that requires tests uses a different combination of them, uses them at different points in a candidate’s education, and sets its own passing scores. States use initial licensure tests for admission to teacher education, as a prerequisite for student teaching, as a condition of graduation, and/or as a licensure requirement. Several hundred different initial licensure tests are currently in use. Two test developers—Educational Testing Service (ETS) and National Evaluation Systems (NES)—develop the vast majority of these tests. Conclusions Because a teacher’s work is complex, even a set of well-designed tests cannot measure all of the prerequisites of competent beginning teaching. Current paper-and-pencil tests provide only some of the information needed to evaluate the competencies of teacher candidates. States have gradually adopted tests for teacher licensure, and test developers have made various tests available over time. Therefore, it is not surprising that states have adopted a variety of tests to license beginning teachers. Appropriate, technically sound tests are difficult and costly to develop. Collaborations among states participating in the Interstate New Teacher Assessment and Support Consortium and other states, professional asso-
OCR for page 166
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality ciations, and test developers bring the intellectual and financial resources of several organizations to this difficult work. Recommendations It is crucial that states use multiple forms of evidence in making decisions about teacher candidates. Licensure systems should be designed to rely on a comprehensive but parsimonious set of high-quality indicators. States, test developers, and professional organizations should continue exploring joint development of initial teacher licensing tests for the knowledge and skill areas they have in common. Federal and state government and private organizations should appropriate funds to support this kind of collaboration. Making Decisions About Candidates Based on Licensure Tests States set passing scores on licensure tests based on judgments about the levels of knowledge and skill needed for minimally competent beginning teaching. Although many states rely on commonly used standard-setting procedures, there is little documentation about these procedures and how states actually use this information in arriving at a final decision about passing scores. In attempts to raise teacher standards, some states have recently raised their passing scores for particular tests. Some report having set passing scores that are higher than those of other states. On all of the tests the committee reviewed, minority candidates had lower passing rates than nonminority candidates did on their initial testing attempts. Though differences between the passing rates of candidate groups eventually decrease because many unsuccessful test takers retake and pass the tests, eventual passing rates for minority candidates are still lower than those for nonminority test takers. Initial licensure tests are only one factor influencing the supply of new teachers. The quality and size of the pool of new teachers depend on many things, including recruiting efforts, other licensing requirements, labor market forces, licensing reciprocity, teacher salaries, and the conditions under which teachers work. Conclusions States differ in how high they set passing scores. The committee does not know the extent to which this variation in passing scores reflects differences among states in standard-setting methods; state teaching and learning standards; the characteristics of applicant pools; or different concerns about measurement error, teacher quality, or teacher supply.
OCR for page 167
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality To the extent that the tests provide accurate measurements, setting higher passing scores would be expected to increase the proportion of teacher candidates in the hiring pool who are competent in the knowledge and skills measured by the tests, although higher passing scores will tend to lower the number of candidates who pass the test. To the extent that test scores have measurement error, setting higher passing scores could eliminate competent candidates. Reducing the number of new licensed teachers could require districts to make difficult choices, such as hiring uncredentialed teachers, increasing class sizes, or increasing salaries to attract licensed teachers from other districts and states. The lower passing rates for minority teacher candidates on current licensure tests pose problems for schools and districts in seeking a qualified and diverse teaching force. Setting substantially higher passing scores on licensure tests is likely to reduce the diversity of the teacher applicant pool, further adding to the difficulty of obtaining a diverse school faculty. Recommendations States should follow professionally accepted standard-setting methods and document the methods they use to set passing scores on initial licensure tests. This documentation should describe the work of standard-setting panels and the basis on which policy decisions were made by the officials setting the final passing scores. Documentation should be publicly available to users and other interested parties. If states raise passing scores as a way to increase the competence of new teachers, they should examine not only the impact on teacher competence but also the effects of raising passing scores on applications to teacher education, on the supply of new teachers, and on the diversity of the teaching force. Evaluating Licensing Tests Solid technical characteristics and fairness are key to the effective use of tests. The work of measurement specialists, test users, and policy makers suggests criteria for judging the appropriateness and technical quality of initial teacher licensure tests. The committee drew on these to develop criteria that it believes users should aspire to in developing and evaluating initial teacher licensure tests. The committee used these evaluation criteria to evaluate a sample of five widely used tests produced by ETS. The tests the committee reviewed met most of its criteria for technical quality, although there were some areas for improvement. The committee also attempted to review a sample of NES tests. Despite concerted and repeated efforts, though, the committee was unable to obtain suffi-
OCR for page 168
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality cient information on the technical characteristics of tests produced by NES and thus could draw no conclusions about their technical quality. Conclusions The committee’s criteria for judging test quality include the following: tests should have a statement of purpose; systematic processes should be used in deciding what to test and in assuring balanced and adequate coverage of these competencies; test materials should be tried out and analyzed before operational decisions are made; test administration and scoring should be uniform and fair; test materials and results should be protected from corruptibility; standard-setting procedures should be systematic and well documented; test results should be consistent across test forms and scorers; information about tests and scoring should be available to candidates; technical documentation should be accessible for public and professional review; validity evidence should be gathered and presented; costs and feasibility should be considered in test development and selection; and the long-term consequences of licensing tests should be monitored and examined. The profession’s standards for educational testing say that information sufficient to evaluate the appropriateness and technical adequacy of tests should be made available to potential test users and other interested parties. The committee considers the lack of sufficient technical information made available by NES and the states to evaluate NES-developed tests to be problematic and a concern. It is also significant because NES-developed tests are administered to very large numbers of teacher candidates. The initial licensure tests currently in use rely almost exclusively on content-related evidence of validity. Few, if any, developers are collecting evidence about how test results relate to other relevant measures of candidates’ knowledge, skills, and abilities. It is important to collect validity data that go beyond content-related validity evidence for initial licensing tests. However, conducting high-quality research of this kind is complex and costly. Examples of relevant research include investigations of the relationships between test results and other measures of candidate knowledge and skills or on the extent to which tests distinguish candidates who are at least minimally competent from those who are not. The processes used to develop current tests, the empirical studies of test content, and common-sense analyses suggest the importance of at least some of what is measured by these initial licensure tests. Beginning teachers should know how to read, write, and do basic mathematics; they should know the content areas they teach.
OCR for page 169
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Little research has been conducted on the extent to which scores on current teacher licensure tests relate to other measures of beginning teacher competence. Much of the research that has been conducted suffers from methodological problems that interfere with making strong conclusions about the results. This makes it hard to determine what effect licensure tests might have on improving the actual competence of beginning teachers. Recommendations States should strive to use the committee’s or similar evaluation criteria when developing and evaluating tests for use in initial teacher licensure systems. When states are selecting from among existing tests for initial teacher licensure, they should obtain and carefully consider field test and any available operational data regarding reliability, validity, cost/feasibility, and fairness as part of their decision-making process. When states are developing licensing tests, they should collect and weigh this evidence in making decisions about the final form of the test and its use. The degree of disparate impact should be an important consideration when states are deciding which licensure test to use for various decisions about candidate competence. States and test developers should provide technical documentation to users, scholars, and the public about the reliability, validity, and disparate impact of their tests. Field test or any available operational data should be used to document the technical quality of tests. The technical information used in deciding on a test normally should be made publicly available before tests are actually used to make decisions about individuals. If technical data are not provided by the end of the first administration year, states should not use information from these tests to make licensing decisions. State agencies contracting for test development should specify the technical information they require. Because of the importance of these data for the technical evaluation of tests, state agencies should request sufficient technical data and timely delivery of documentation. State agencies should ensure their clear authority to release this information to users and others for the purpose of objectively evaluating the technical quality of the tests, the resulting scores, and the interpretations based on them. States should arrange for independent evaluations of their current tests and teacher licensure systems and make the results of these independent examinations of their systems available for outside review. The committee encourages the federal government and others to conduct research on the extent to which teacher licensure tests distinguish be-
OCR for page 170
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality tween beginning teachers who are at least minimally competent and those who are not regarding the knowledge and skills the tests are intended to measure. This research should include evidence on a broad range of teacher competencies. Such research is likely to improve the development of teacher licensure tests. Within the limits of privacy law, states should make their raw data available to the research community to facilitate development and validity research on initial teacher licensure tests. SHOULD TEACHER LICENSURE TESTS BE USED TO HOLD STATES AND INSTITUTIONS OF HIGHER EDUCATION ACCOUNTABLE FOR THE QUALITY OF TEACHER PREPARATION AND LICENSURE? Making Decisions About Programs Based on Licensure Tests Title II of the Higher Education Act was enacted to achieve four goals: to improve student achievement; to improve the quality of the current and future teaching force by improving the preparation of prospective teachers and enhancing professional development activities; to hold institutions of higher education accountable for preparing beginning teachers to have the necessary teaching skills and to be highly competent in the academic content areas in which they plan to teach; and to recruit highly qualified individuals, including individuals from other occupations. Conclusions It is reasonable to hold teacher education institutions accountable for the quality of their teacher preparation programs. By their design and as currently used, initial teacher licensure tests fall short of the intended policy goals for their use as accountability tools and as levers for improving teacher preparation and licensing programs. The public reporting and accountability provisions of Title II may encourage erroneous conclusions about the quality of teacher preparation. Although the percentage of graduates who pass initial licensure tests provides an entry point for evaluating an institution’s quality, simple comparisons among institutions based on their passing rates are difficult to interpret for many reasons. These include the fact that institutions have different educational missions and recruiting practices, their students have different entry-level qualifications, teacher education programs have different entry and exit testing requirements, and programs have different procedures for determining the institutional affiliations of their candidates. By themselves, passing rates on licensure tests do not provide
OCR for page 171
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality adequate information on which to judge the quality of teacher education programs. Simple comparisons of passing rates across states are misleading. Many states use different tests for initial licensure or set different passing scores on the same tests. States have different policies about when a test is given or what decisions it supports. To fairly and accurately judge the quality of teacher education programs, federal and state officials need data on a wide variety of program characteristics from multiple sources. Other indicators of program quality might include assessment data for students in relation to course and program benchmarks, employer evaluations, and district or state evaluations of beginning teaching. Other indicators might include information on course requirements and course quality, measures of the amount and quality of field experiences, evidence of opportunities to work with students with special learning needs and students with diverse backgrounds. Data on the qualifications of program faculty, the allocation of resources, and the adequacy of facilities might be considered. The qualifications of students at entry to teacher education programs also should be included. Recommendations States should not use passing rates on initial licensure tests as the sole basis for deciding whether their teacher education programs are low performing. States should report multiple indicators of the quality of teacher preparation programs to federal officials in complying with Title II. The federal government should not use passing rates on initial teacher licensing tests as the sole basis for comparing states and teacher education programs or for withholding funds, imposing other sanctions, or rewarding teacher education programs. Federal officials should continue to collect the state and school data required by Title II but should not withhold funds from, otherwise sanction, or reward programs until a study is mounted of the multiple and varied data that might be used to judge the quality of teacher preparation and licensure. HOW CAN INNOVATIVE MEASURES OF BEGINNING TEACHER COMPETENCE HELP IMPROVE TEACHER QUALITY? Several new and developing teacher assessment systems use a variety of testing and assessment methods, including assessments of teaching performance. These include multiple measures of candidates’ knowledge, skills, abilities, and
OCR for page 172
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality dispositions. In these systems, assessments are integrated with professional development and with ongoing support of prospective or beginning teachers. Conclusion New and developing assessment systems warrant investigation for addressing the limits of current initial teacher licensure tests and for improving teacher licensure. The benefits, costs, and limitations of these systems should be investigated. Recommendations Research and development of broad-based indicators of teacher competence, not limited to test-based evidence, should be undertaken; indicators should include assessments of teaching performance in the classroom, of candidates’ ability to work effectively with students with diverse learning needs and cultural backgrounds and in a variety of settings, and of competencies that more directly relate to student learning. When initial licensure tests are used, they should be part of a coherent developmental system of preparation, assessment, and support that reflects the many features of teacher competence.
Representative terms from entire chapter: