Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 34
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality 3 Testing and Licensing Beginning Teachers Teacher licensure is under the authority of individual states. The goal of initial teacher licensure is to ensure that all students have competent teachers. Given the complexity of teaching and differences in states’ current efforts to improve teaching and learning, it is not surprising that the landscape for teacher licensing is complex. States impose numerous and varied requirements on candidates for licensure. States require candidates to fulfill education and supervised teaching requirements, pass required tests, provide evidence of good character, and meet other licensure requirements. Furthermore, states use many different licensure tests in different ways. This chapter describes initial teacher licensing—the licensing decisions made before a teacher enters the classroom on an unsupervised basis for the first time. This chapter begins with a brief discussion of licensing in other professions and of the differences between teacher licensing and licensing in other fields. The committee then describes teacher licensure systems, the tests used by states, and the decisions about candidates they support. The committee closes with examples of teacher preparation, testing, and licensure in seven states. Here and elsewhere in the report the committee distinguishes between tests and assessments. The committee defines tests as paper-and-pencil measures of knowledge and skill; tests are evaluated and scored using standardized processes. Assessments are broader. While assessments can include standardized paper-and-pencil measures, they may also include performance-based evidence, such as portfolios, videotapes, and observation records. Assessments can be collections of different sorts of evidence, systematically gathered over time, to inform particular decisions or interpretations.
OCR for page 35
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality PROFESSIONAL LICENSING Many professions use licensing systems to select individuals into their fields and to prevent those considered incompetent from practicing. As defined in 1971 by the U.S. Department of Health, Education, and Welfare, licensure is “the process by which an agency of government grants permission to persons to engage in a given profession or occupation by certifying that those licensed have attained the minimal degree of competency necessary to ensure that the public health, safety, and welfare will be reasonably well protected” (p. 7). Licensure is a state function. States regulate more than 500 professions, from real estate appraisers to electricians to architects. Licensure requirements for a given profession may vary across states, and states may vary in the professions they regulate. Professions generally promote quality practice in three ways: through professional accreditation of preparation programs, through state licensing of applicants to the profession, and through certification of practitioners. Professions use accreditation to examine their preparation programs and to attempt to ensure that they provide high-quality instruction and practice opportunities. Licensing serves as a gateway to the profession, allowing only those who have met minimum standards of competence to practice. For some professions, certification is granted to those who demonstrate exemplary knowledge and skill. For example, in the medical profession, certification generally is professional recognition of higher standards of accomplishment and typically is associated with advanced study and practice. A national professional body, such as the American Board of Internal Medicine, the American Board of Emergency Medicine, or the National Board for Professional Teaching Standards usually grants certification. Tests often play an important role in licensure systems and in what candidates study and learn to prepare for licensure and practice (Darling-Hammond et al., 1999). Licensing tests are designed “to provide the public with a dependable mechanism for identifying practitioners who have met particular standards” (American Educational Research Association et al., 1999:156). Typically, panels of professionals determine the knowledge and skills that are critical for safe and effective performance, with an emphasis on the knowledge and skills that should be mastered prior to entering the profession (Stoker and Impara, 1995; American Educational Research Association et al., 1999). The test specifications for licensure examinations make explicit at least part of what professions consider worth knowing and how it should be known and demonstrated. Specifications for licensure tests may also recognize advances in professional knowledge. Licensing exams are viewed as a means for ensuring that advances in professional knowledge are incorporated into professional education programs. For example, one way that fields such as medicine, engineering, and psychology ensure that new research knowledge gets used is by including it on licensing examinations, specialty board examinations, and accreditation guidelines for professional schools, hospitals, or other training sites. Although tests
OCR for page 36
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality are prominent in licensing systems, it is important to recognize that they are only one part of the overall quality control system that most professions have developed (National Association of State Directors of Teacher Education and Certification, 2000b). Professional regulations generally address requirements in three areas: education, experience, and testing. The education component is intended to ensure that candidates have encountered the broad base of knowledge they will need to draw on when making decisions in professional practice. The supervised experience component allows candidates to learn the complex art of applying knowledge to specific problems of practice, to make judgments, to weigh and balance competing considerations, and to develop practical skills and put them to use. Professions generally include education, experience, and testing components in their licensing requirements but vary in the amounts of education and experience they require and in the sequence in which the requirements must be met. In deciding on specific education, experience, and testing requirements, professions evaluate the extent to which each element is likely to lead to public protection. The required levels of education, experience, and testing are intended to reflect the knowledge and skills needed for entry-level practice and not to be so high as to be unreasonably limiting (Schmitt, 2000; American Educational Research Association et al., 1999). To learn more about licensing in other professions, the committee commissioned a paper on licensure requirements in seven fields. The goal was to learn (1) how other professions handle licensing requirements; (2) what other professions require with regard to education, experience, and testing; and (3) whether the requirements of other professions suggest ways to improve teacher licensure. The committee focused on professions that generally require a bachelor’s degree for entry into the profession because these professions were expected to offer useful analogies to education. Table 3–1 compares licensure requirements for seven professions: architects, certified public accountants (CPAs), professional engineers, land surveyors, physical therapists, registered nurses, and social workers. For each profession the state is the licensing agent. For each profession there is also variability in the education and experience that states require for licensure. For example, there are four categories of practice for social workers: Basic, Intermediate, Advanced, and Clinical. States differ in the level of practice that they regulate. The 34 states that offer licenses to social workers with bachelor’s degrees (the Basic level) have 19 different combinations of education and experience requirements (Schmitt, 2000). For CPA candidates, over half of the states require two years of practical experience for a license and a few require three years (American Institute of Certified Public Accountants and National Association of State Boards of Accountancy, 1998). For architects about a quarter of states require graduation from an accredited five-year architecture program; other states accept alternative ways for satisfying the education requirements <www.ncarb.org>. These differ-
OCR for page 37
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality ences are analogous to the different state education and experience requirements for teachers. There is a difference, however, between states’ licensure testing requirements for the seven professions and the licensure testing requirements for teaching. For each of the seven professions examined, the same test or series of tests are used by all states (although some states augment a national examination with additional state components). For six of these professions (all but social work), the passing standards on the test(s) are the same across states (Schmitt, 2000). The situation is very different in education where testing requirements differ by state, subject area, and grade level and the passing standards vary from state to state. Table 3–1 also provides data on alternate paths into the seven professions. A number of the professions studied have varying degree requirements for alternate routes to licensing. Some accept experience in lieu of the education requirements. About half the states allow engineers to substitute experience for an engineering degree, although the experience requirement is substantial (e.g., over 10 years; National Council of Examiners for Engineering and Surveying, 2000). Prospective architects can also substitute experience for education. Over half the states allow architect candidates to be licensed with a bachelor’s degree or a high school diploma, in combination with experience/training (although typically candidates must have a combination of eight years’ experience and education <www.ncarb.org>. Several states accept various combinations of education and experience in lieu of a bachelor’s degree in order for prospective accountants to sit for the licensing exam (American Institute of Certified Public Accountants and National Association of State Boards of Accountancy, 1998). This is another area in which teaching is similar to the other professions studied. Some of the seven professions allow individuals to practice without a license, but they place limitations on what unlicensed individuals are allowed to do. Unlicensed architects can design certain structures, although only licensed architects can seal a design. Unlicensed individuals can work as accountants, although they cannot use the CPA designation. In some jurisdictions, registered nurses and physical therapists who have completed the education requirements can receive temporary licenses prior to fulfilling the examination requirements (Schmitt, 2000). Teaching is similar to other professions in this regard. Unlicensed teachers can work in private schools in some states (National Association of State Directors of Teacher Education and Certification, 2000b; U.S. Department of Education, 2000b). Also, many states allow unlicensed individuals to teach in public schools for a fixed period when there are too few licensed teachers to staff existing classes. TEACHER LICENSURE The teaching profession and individual states impose standards through program approval, teacher education admission and course requirements, testing, and initial licensure to promote quality practice.
OCR for page 38
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality TABLE 3–1 Comparison of Licensing Requirements for Teaching and Other Professionsa Requirements Profession Education Experience Examination Teaching All states require a bachelor’s degree in teacher education or a content area; most have post-baccalaureate alternative routes to enter teaching. Most states require student teaching; length of student teaching experience varies from 9 to 18 weeks.b Over 600 exams in use; most states require one or more tests of basic skills, general knowledge, subject matter knowledge, or pedagogical knowledge. Architectse About one-quarter of the states require a five-year degree from an NCARB-accredited program; others accept alternative ways of satisfying the education requirements. Most states require that the Intern Development Program standards be met (5,600 hours of defined experience) One exam with nine sections: predesign; general structures, lateral forces, mechanical and electrical systems, materials and methods; construction documents and services, site planning, building planning, and building technology. Certified public accountantsf Most states require a bachelor’s degree. Most require 150 hours of education prior to taking the test. Most states require experience, generally between one and three years. Four exams: Auditing, Financial Accounting and Reporting, Business Law and Professional Responsibility, and Accounting and Reporting; must pass all parts; over half the states require an exam or professional course in ethics. Professional engineersg Most states require a bachelor’s degree in engineering. All states require experience along with engineering degree, generally four years. Two-part exam; Fundamentals of Engineering (FE) is first step and can be taken before degree completion; Principles of Engineering (PE) is the second step and is usually taken after degree completion; about half the states also require jurisprudence, ethics, or specialty exams.
OCR for page 39
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Exam Format/Cost Experience Accepted in Place of Education Requirements Practice Without Completing All Licensing Requirements Continuing Education (CE) Requirements Primarily multiple-choice and constructed-response. B.A. is a minimum requirement for all states. Experience and education requirements for states’ alternative licensure programs vary widely. Most states allow unlicensed individuals to waive one or more licensing requirements to teach temporarilyc Most states require CE.d Computer simulation; six multiple-choice sections; three vignette sections with graphics problems; $981; sections can be taken at different times. Less than one-quarter accept training in lieu of education; typically need combination of 8 years’ education and experience. Unlicensed individuals can work as architects, but limitations are placed on the types of work permitted (e.g., size of structure, type of structure); unlicensed individuals cannot seal a design or use the title. Not required in most states; a few require 12 hours per year. Multiple-choice, essay, matching, short-answer, fill-in the-blank; 15 hours, two days; test fee varies from state to state. A few states allow a combination of experience and education to be substituted for the bachelor’s degree in order to sit for the exam. Unlicensed individuals can work as accountants but cannot use the CPA designation or perform the attest function. Most states require 40 hours CE per year. Multiple-choice and problems; 16 disciplines; eight hours per part; open book for PE exam but restrictions vary across states; $50 to $75 for FE; $100+ for PE. About half the states allow experience to be substituted for an engineering degree; most require over 10 years. Unlicensed individuals can work as engineers, but limitations are placed on the types of work permitted; unlicensed individuals cannot seal a drawing or use the title. Less than half of the states require CE requirements.
OCR for page 40
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Requirements Profession Education Experience Examination Land surveyorsg Most require a bachelor’ degree in land surveying. Over half the states require those with a degree from an accredited program to also have field experience. Two tests: Fundamentals of Land Surveying (FLS) and Principles and Practice of Land Surveying (PLS); most states also require a jurisprudence exam. Physical therapistsh All states require graduation from an accredited PT program; after 2002, only postbaccalaureate programs will be accredited. A few states require clinical practice. One exam; a few states require oral or practical exam prior to licensure. Registered nursesj Most states require either an associate’s degree, a bachelor of science in nursing, or graduation from NY Regents external degree program. No states have experience requirements. One exam. Social workersk Four categories of practice; states vary in the level(s) of practice they regulate. Requirements vary according to level of practice. Most states require at least a bachelor’s in social work for the basic level of practice, although some accept nonsocial work degrees. Experience requirements vary according to the level of practice. Four levels of exams: Basic, Intermediate, Advanced, and Clinical; candidates take only one exam for each level of licensure. aThis table summarizes information in a paper prepared for the committee by Kara Schmitt. The full electronic version of the paper can be obtained by contacting the author at email@example.com. bFrom National Association of State Directors of Teacher Education and Certification (2000b:Table B-22). cFrom Education Week, Quality Counts (2000:52). dFrom National Association of State Directors of Teacher Education and Certification (2000b:E-1). eFrom National Council of Architectural Registration Boards (1998); NCARB member board requirements. Washington, DC: National Council of Architectural Registration Boards; and <www.ncarb.org>. fFrom American Institute of Certified Public Accountants and National Association of State Boards of Accoun
OCR for page 41
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Exam Format/Cost Experience Accepted in Place of Education Requirements Practice Without Completing All Licensing Requirements Continuing Education (CE) Requirements Multiple-choice; eight hours for FLS, $175; six hours for PLS, $265. A few states accept experience in place of education. Unlicensed individuals can work as surveyors, but limitations are placed on the types of work permitted; unlicensed individuals cannot seal a drawing or use the title. About half the states require CE, generally 6 to 15 hours per year. Computer-based multiple-choice; four hours; $285. Experience cannot be substituted for education. About half of the states allow individuals who have completed the educational requirements to practice (temporarily) prior to passing the test. About half the states require CE (one to four CE units). Computer-adaptive; multiple-choice; five hours; $120 No states accept experience in lieu of education. About half of the states allow individuals who have completed the educational requirements to practice (temporarily) prior to passing the test; some states allow individuals to take the LPN exam while training for RN and to practice as an LPN until becoming an RN. About half of the states require CE for license renewal. Computer-based multiple-choice; four hours; $110 per level Most states do not accept experience in lieu of education. Four levels of practice that define a progression of education/experience requirements. Most states require CE, usually 15 to 20 hours every two years. tancy (1998). Digest of state accountancy laws and state board regulations. New York: American Institute of Certified Public Accountants and <www.aicpa.org>. gFrom National Council of Examiners for Engineering and Surveying (2000) <www.ncees.org>. hFrom American Physical Therapy Association. (1997). State licensure reference guide. Washington, DC: American Physical Therapy Association and <www.fsbpt.org>. jFrom Yocom et al. (1999) <www.ncsbn.org>. kFrom American Association of State Social Work Boards (1998) <www.aswb.org>.
OCR for page 42
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Accreditation and Approval of Teacher Education Programs As in other fields, accreditation of teacher education programs is a mechanism for examining and attesting to the quality of programs (National Council for the Accreditation of Teacher Education, 1997). However, unlike other professions, national professional accreditation is not required in teacher education. The majority of states recognize only teacher preparation programs from institutions that are regionally accredited. Fewer than 40 percent of teacher education programs are nationally accredited (U.S. Department of Education, 2001). All states require teacher preparation programs to obtain state approval based on policies and standards set by them. These standards establish criteria that, if met, authorize the program to prepare and recommend teacher candidates for state licensure. The teacher preparation approval standards often also incorporate specific state-required courses or competencies necessary to obtain a state license. These approval standards and license requirements are unique to each state. Fourteen states have established independent professional standards boards or commissions with responsibility for establishing licensure standards, and all but three of these boards have authority to approve teacher preparation programs (National Association of State Directors of Teacher Education and Certification, 2000b). The process for approving programs varies by state. Some states conduct a paper review of the curriculum. Others do an on-site review based on adopted standards, primarily process measures organized around students, faculty, and program resources (e.g., number of faculty, degree status, student admission criteria, diversity of students and faculty, professional development funds). Some states (such as Indiana, Connecticut, Ohio, North Dakota, and Minnesota) have approval criteria that are performance or competency based; they examine how a program endeavors to ensure that teacher candidates acquire specific knowledge and skills. Sometimes they also examine program outcomes, such as teachers’ and employers’ perceptions of their adequacy of the programs, graduation rates, and job placement rates. In over 40 states, teacher preparation programs in colleges and universities may obtain both state approval and national accreditation or may substitute national accreditation by the National Council for the Accreditation of Teacher Education (NCATE) for state approval. NCATE began accrediting teacher education programs in 1954 (NCATE, 2000a). It represents 33 specialty professional associations of teachers, teacher educators, content specialists, and local and state policy makers. NCATE introduced new outcome-based accreditation standards in 2000 (see Chapter 2 and Appendix B). Currently, 12 states require accreditation using the NCATE standards and more than 40 states have partnerships that encourage professional accreditation. A new organization called the Teacher Education Accrediting Council (TEAC) has proposed to take a different approach to accreditation. TEAC proposes to conduct academic audits against institutions’ own standards using evidence collected by the institution and to use those standards to judge program
OCR for page 43
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality quality. TEAC has applied to the U.S. Department of Education and the Council for Higher Education Accreditation for recognition as an accrediting agency but has not yet received this recognition. State-Specified Admission Requirements for Teacher Education Some states also address teacher quality by specifying standards for admission to teacher education programs. Thirty-five states specify entrance requirements for their teacher preparation programs (National Association of State Directors of Teacher Education and Certification, 2000b). Almost all include meeting a basic skills standard. Additional requirements may include minimum grade point averages, subject area majors, and certain coursework. State-Specified Course Requirements in Teacher Education States specify the coursework their teacher candidates should take and the competencies they must demonstrate. Thirty-seven states specify some required coursework in English, humanities, social sciences, natural sciences, and mathematics, although the specific course requirements may differ for elementary or multiple-subject teachers and single-subject teachers (National Association of State Directors of Teacher Education and Certification, 2000b). Some states require that a subject area major be completed prior to entering teacher preparation. Most teacher preparation programs include a mix of courses and field experiences (National Association of State Directors of Teacher Education and Certification, 2000b). The coursework usually includes teaching strategies and methodology, social foundations, development and learning, curriculum and instruction, classroom management, and student assessment. Some states also include the teaching of reading, use of technology, cultural diversity, school organization, and school improvement course requirements. Instead of specifying course requirements, some states have identified competencies or performance standards to be demonstrated by program completers (Alaska, Alabama, California, Minnesota, North Dakota, Texas, West Virginia, Connecticut and Ohio; National Association of State Directors of Teacher Education and Certification, 2000b). These competencies generally follow the tenets of the states’ teaching standards or the Interstate New Teacher Assessment and Support Consortium (INTASC) and other nationally developed standards statements. Although most states also specify the type and amount of supervised teaching experience candidates must complete, there is variability in state requirements. Some states require some experience in schools before the student teaching experience begins; others do not. Thirty states have field experience requirements before student teaching. For student teaching, states require between 9 and 18 weeks of supervised teaching (National Association of State Directors of Teacher Educa
OCR for page 44
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality tion and Certification, 2000b). Most professions, including medicine, architecture, psychology, and engineering, require more extensive supervised clinical experience (Darling-Hammond et al., 1999). Tests As noted earlier, in addition to these requirements, 42 states require candidates to pass one or more tests of basic skills, general knowledge, subject matter knowledge, or teaching knowledge (National Association of State Directors of Teacher Education and Certification, 2000b). Among states that use tests, the type and number of required tests run the gamut from one test of basic skills (e.g., Alabama) to four different types of tests, including basic skills, subject matter knowledge, pedagogical content knowledge and pedagogical knowledge tests (e.g., Michigan, Colorado). Within types, more than one test may be required. For example, California requires two subject matter tests for high school teachers who have not completed an approved subject matter program. In some states, licensing tests are tied to the granting of a degree; in others they are not. In states where tests are not required for graduation, candidates can successfully complete a teacher preparation program and graduate, but if they fail a state’s test(s), they cannot get a standard teaching license. In states where tests are required for degree conferral, candidates can successfully complete all of the institutional requirements for graduation but leave without a degree because they fail the licensing test; these candidates also lack a license to teach in that state’s public school system. States’ Alternative Preparation Programs for Teachers A majority of states also have supplemented college and university preparation programs for licensure with postbaccalaureate alternative routes for candidates to enter teaching from other fields (Feistritzer and Chester, 2000). These routes are called alternative because they provide options to the four-year undergraduate programs that were the only routes to licensure in many states until the 1990s. Although varying greatly, these routes generally include an entrance requirement for content expertise and experience in the field. The programs range from requiring a preservice program of teacher education (usually 9 to 15 months) to programs offering 3 to 12 weeks of instruction prior to granting a limited teaching license, such as an intern license or a temporary or emergency license, while other requirements are completed. Some states also provide individually tailored programs based on reviews of the academic and professional backgrounds of each candidate. Alternative programs are provided by states, local districts, or institutions of higher education. In most cases, teachers from these programs are required to pass the same tests as those who become teachers through traditional routes.
OCR for page 59
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Or Content Mastery Exam for Educators (CMEE) CMEE=850 (composite score) Californiag California Basic Educational Skills Test (CBEST) Composite score of 123, with a minimum score of 37 in each section: reading, math, and writing Entry into teacher education program (most traditional teacher education programs) • Completion of an approved teacher preparation program • Completion of a course or an exam on the U.S. Constitution • Completion of a teaching of reading course • Completion of a health education course, including nutrition and cardiopulmonary resuscitation Reading Instruction Competence Assessment (RICA) Elementary Teachers: Multiple Subjects Assessment for Teachers (MSAT) (a) Content Knowledge (b) Content Areas Exercises RICA Composite score =311 Content knowledge=156 Content area exercises=155 Marylandh Praxis I Paper/Pencil or Computer-Based Tests (CBT) accepted reading=177; writing=173; math=177 [CBT: reading=325; writing=319; math=322] Licensure • Graduation from an approved teacher education program • Completion of a special education course • Good moral character
OCR for page 60
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality States Required Tests Passing Scores Decisions About Students Supported by Test Scores Other Requirements for Initial Licensure Praxis II for elementary teachers: Elementary Education Content Knowledge Elementary School: Content Knowledge=136 Licensure Elementary Education Content Area Exercises Content exercises =150 Connecticuti Praxis I (CBT only) reading=324; writing=318; math=319 Entry into teacher education programs • Graduation from an approved teacher education program • Completion of a special education course • Completion of a U.S. history course Praxis II (for elementary teachers): Elementary Education: Curriculum, Instruction and Assessment 163 Licensure Elementary Education Content Area Exercises 148
OCR for page 61
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality aTable includes tests and other requirements for initial licensure only. Connecticut and other states have induction and beginning teacher support programs that have additional assessment requirements. bHigher education institutions in the state require students to pass a basic skills test before entering teacher education, although this is not a state requirement. Idaho State Department of Education, Certification Department <www.sde.state.id.us/certification/>. cHigher education institutions in the state require students to pass a basic skills test before entering teacher education, although this is not a state requirement. dWyoming State Department of Education, Certification Department <www.k12.wy.us/ptsb/index>. eAlaska State Department of Education, Certification Department <www.eed.state.ak.us/TeacherCertification>. fNebraska State Department of Education, Certification Department <www.nde.state.ne.us/TCERT/TCERT.html>. gCalifornia Commission on Teacher Credentialing <www.ctc.ca.gov>. hMaryland State Department of Education, Certification Department <www.msde.state.md.us/certification/index.htm>. iConnecticut State Department of Education, Certification Department <www.state.ct.us/sde/dtl/cert/index.htm>.
OCR for page 62
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality The admission criteria for teacher education programs also can differ at institutions within a state and in some cases even exceed the state’s requirement for initial licensure. For example, Maryland does not require basic skills testing for entry to teacher education, yet some institutions in the state require students to pass Praxis I (at the state’s established passing score for licensure) to enter their programs. In Idaho and Wyoming, where there are no testing requirements, some higher-education institutions require a basic skills test for admission to their teacher education programs; different tests are used by different institutions. A review of the case studies in Appendix D also reveals some variation in the curriculum of teacher education programs across states. Several states (Idaho, Maryland) have adopted the National Association of State Directors of Teacher Education and Certification standards as the basis for the curriculum in their teacher education programs, while other states (California, Nebraska) require that the curriculum of teacher education programs be aligned to standards adopted by the state. Additionally, states vary in the amount of education required to earn an initial teaching license. For example, in California most traditional teacher education programs require completion of a baccalaureate program before admission to a one-year preparatory program. In most states a teaching license can be obtained after completion of an undergraduate program. In most of the states reviewed by the committee, initial licensure is contingent on institutional recommendations for candidates; this is especially true in states without testing requirements. Yet in some states (California and Maryland) alternative routes to initial licensure have been established that allow individuals to begin teaching (with supervision) without having completed a teacher education program. Candidates are typically granted an emergency or intern teaching certificate for the academic year. The alternative routes can include university internship programs, district internship programs, preinternship programs, and resident teacher programs. In some programs, candidates need to meet state testing requirements before beginning the alternate program and entering the classroom, while in other programs candidates can teach while fulfilling state testing requirements for the initial license. SETTING PASSING SCORES Passing scores on licensure tests are important because they help determine access to the profession. States set their own passing scores The passing scores currently required by different states are given in Appendix C (National Association of State Directors of Teacher Education and Certification, 2000b; U.S. Department of Education, 2000a). The standard-setting models used to estimate minimally competent performance are discussed below. Information used by policy makers to set final passing scores also is described.
OCR for page 63
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Standard-Setting Methods Most states set passing scores on teacher licensure tests by asking panels of educators to identify the level of content and teaching knowledge that they judge to be minimally necessary for beginning practice. Several methods for setting passing scores are currently used (Jaeger, 1989; Mills, 1995; Plake, 1998; Horn et al., 2000). The modified Angoff (1971) model is the most widely used method for setting standards on multiple-choice tests. It is used by ETS states. With this method, panelists estimate the proportion of minimally qualified candidates who would be able to answer each test question correctly. The standard is established by summing these proportions across questions. Though this approach is widely used in licensure testing, its methods and the meaningfulness of the standards it produces have been questioned (National Research Council, 1999b). Analysts argue that the estimation task given to panelists is too difficult and confusing, that the results vary significantly by question type, and that the method sometimes yields results that are not believable. Other methods are used to set standards on open-ended test questions. Estimating the performance of minimally competent candidates on questions for which responses are graded along a continuum (i.e., not just right or wrong), such as on essay questions, is more complex (Educational Testing Service, 1999b). The methods used on these items are not as well researched as those used with multiple-choice items, and there is less agreement in the field about which methods are appropriate. States using ETS exams either use item-level pass/fail procedures or a benchmark paper selection approach (Educational Testing Service, 1997). For the former, panelists look at the scoring tables for open-ended items and make estimates of the numbers of points minimally competent candidates can be expected to earn on each question. Then they are given input on the actual performance of the candidate population. Experts are given an opportunity to revise their initial estimates. The final averages across panelists are used as the recommended passing scores for open-ended questions. The benchmark paper selection approach calls for panels of educators to examine a preselected range of candidate responses and judge which work is indicative of the minimally competent candidate; often, panelists select the two papers that best represent this work. Sometimes panelists are given data on the actual performance of candidates after their first selection of papers and are allowed to make revised selections if they wish to after seeing the data. The scores of benchmark papers are averaged across panelists to generate the recommended passing score for the open-ended questions. In some states the results of these standard-setting studies are then provided to state officials for the final determination of passing scores (Educational Testing Service, 1999b). The panelists recommend passing scores to a policy body, such as the state board of education or the commissioner’s office. That body
OCR for page 64
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality usually makes the final decision based on the panelists’ recommendations and other information, including the standards of neighboring states, historical passing rates in their own state, eventual passing rates (as distinct from first-time passing rates), possible decision errors for candidates scoring at or near the proposed passing score, the impact of different standards on the passing rates of minority groups, and a desire to raise or lower standards in response to concerns about teacher quality and supply (Horn et al., 2000). These factors are similar to those influencing standards in other fields (Jasanoff, 1998). The extent to which states follow the standard-setting models described here is not known. It is known that research on standard-setting methods indicates that passing scores may vary as a function of several factors, such as the methods used to set the standard, the particular panelists who participate in the process, and the training and instructions the panelists receive (Kane, 1994; Horn et al., 2000). Indeed, on different occasions even the same panelists may arrive at different standards for the same test (Norcini and Shea, 1992). Various criteria, such as high agreement among different panels, have been suggested for evaluating the quality of standard-setting processes (Kane, 1994; Cizek, 1996). The professional Standards for Educational and Psychological Testing say that technically-sound standard-setting studies are systematic, both in the procedures used and in the way the panelists are chosen (American Educational Research Association et al., 1999). Standard-setting studies all should be well-documented, providing information on the panelists, the methods, and the results. No data were found in the published literature to document states’ particular practices. In order to better characterize and discuss states’ various standard-setting procedures, the committee fielded a survey on standard-setting methods in teacher licensure as part of its study. State licensing officials were asked about the methods used in their standard-setting studies, about the composition of their standard-setting panels, and about the decision models and data used by policy makers in setting final passing scores. The questions appear below in Box 3–2. The committee did not get enough responses to its survey to support useful descriptions of states’ practices. Responses to questions about the standard-setting methods that states use and about their panels were low. Responses to questions about the methods and data used by policy bodies in setting final passing scores were particularly low. Little is known about how states arrive at final decisions about passing scores. It is known, however, that several state policy bodies have recently raised passing scores on their licensure tests (Educational Testing Service, 1998b, 2000). Others have reported setting passing scores on newly adopted tests that exceed those set by other states using the same tests (Archer, 1998; The Ohio Wire, 2/18/98; Education World, 2000). These policies have accompanied teacher quality initiatives and make the standards for passing more stringent.
OCR for page 65
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality BOX 3–2 Survey of Standard-Setting Methods for Initial Teacher Licensure Examinations 1. There are several methods used to set performance standards, or cut scores, on tests. Which method most closely describes the method used by your state to set a passing standard for your initial teacher licensure examination(s)? If your state uses different methods for different tests, please mark all methods used: ___ Modified Angoff. For each item or exercise, panelists indicate the proportion of minimally competent examinees who would be able to answer the item satisfactorily. The standard is established by summing these proportions across all items or exercises. ___ Bookmarking. This method is used for both multiple-choice and constructed-response tests. Test items are arranged in descending order of difficulty and panelists place “bookmarks” immediately after the last item that all minimally competent examinees should be expected to know. ___ Judgmental policy capturing. This method is used for tests such as performance assessments and test batteries that contain items or exercises that have a score scale of three or more points (rather than simply right/wrong). Panelists review profiles of scores and provide judgments about the performance level of each examinee. Statistical analysis is used to yield a recommended standard for each panelist. ___ Nedelsky’s method. This method is used with multiple-choice tests. For each item, panelists eliminate all response options that the minimally competent examinee should be able to eliminate and records the reciprocal of the remaining number of responses. (If a panelist eliminates two out of five response options, three options would remain and the panelist would record “1/3”). ___ Ebel’s method. This method is used with multiple-choice tests. Test items are classified in a two-dimensional grid of item difficulty and item relevance. Each cell of the grid indicates how many items are at a given level of difficulty and relevance. Panelists indicate the proportion of items in each cell that a minimally competent examinee should answer correctly. ___ Contrasting groups. Panelists engage in discussion and agree on what constitutes minimally acceptable performance. They then identify candidates who, in their judgment, are clearly above the standard and those who are clearly below the standard. The examination is administered to both groups of examinees, score distributions are plotted for the two groups, and the performance standard is set at the point of intersection between the two score distributions. ___ Other (please describe) Composition of the Standard-Setting Panel 2. How many panelists typically serve on your state’s standard-setting panel? ___ 3. What is the composition of your state’s standard-setting panel? Please indicate the approximate percentage of your state’s standard-setting panel represented by each type of person listed below.
OCR for page 66
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality ___ Teachers ___ Principals ___ District-level curriculum specialists ___ University-level content experts ___ Teacher educators ___ Professional association representatives (e.g., National Council of Teachers of English, National Council of Teachers of Mathematics, International Reading Association, etc.) ___ Business representatives ___ Parents ___ Other (please describe) Review of the Recommended Passing Standard 4. After the standard-setting panel recommends a passing standard, what happens next? ___ Nothing. The panel’s recommended standard is adopted as the final standard. ___ An automatic adjustment based on the standard error is applied to the panel’s recommended standard. ___ The panel’s recommended standard is reviewed by a technical advisory committee. ___ The panel’s recommended standard is reviewed by the state department of education. ___ The panel’s recommended standard is reviewed by a professional standards board. ___ Other (please describe) Changes to the Recommended Standard 5. If you indicated that the technical advisory committee, state department, professional standards board, or other body has the authority to modify the standard-setting panel’s recommended standard, which of the following best describes the experience in your state? ___ The standard has not been changed. ___ The standard has been adjusted upward based on the standard error. ___ The standard has been adjusted upward based on other considerations. ___ The standard has been adjusted downward based on the standard error. ___ The standard has been adjusted downward based on other considerations. Current Standards Current passing scores on several widely used Praxis tests appear in Table 3–6 . For each test, the lowest passing score currently in use by a state is shown in the first data column. The percentiles associated with these passing scores in the national candidate population are given in the second data column. Column
OCR for page 67
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality TABLE 3–6 Range of Passing Scores for States Using Selected Praxis Tests, 1998–1999 Praxis Testa Lowest Passing Score National Percentile Equivalent for Lowest Score Highest Passing Score National Percentile Equivalent for Highest Score PPST: Reading 169 10 178 43 PLT: K-6 152 6 169 34 Middle School English/Language Arts 145 6 164 29 Mathematics: Proofs, Models & Problems, Part 1 139 22 154 46 Biology Content Knowledge, Part 1 139 4 161 32 Biology Content Knowledge, Part 2 135 25 156 68 aThese tests were selected because currently they are widely used; they represent a range of test types, content, item types, and levels of schooling; and they are not currently scheduled for retirement. Scores range from 100 to 200 on these tests.
OCR for page 68
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality three shows the most stringent passing scores set by states for each of the tests, and data column four shows the associated national percentiles. Table 3–6 shows substantial variation among states in the passing scores set for teacher examinations. For example, state passing scores on the Principles of Learning and Teaching (PLT): K-6 test range from 152 to 169 on a scale of 100 to 200. A score of 152 on the test places candidates at about the 6th percentile on the national distribution; a score of 169 places examinees at about the 34th percentile. That is, in one state, teacher candidates can pass the PLT testing requirement for licensure by scoring slightly above the 6th percentile of candidates in the national distribution. In another, applicants for licensure must score above the bottom third of the national candidate population on the PLT to satisfy the licensing requirement. Variations in passing scores for states using the same tests show the states’ differing minimum requirements for entry-level teaching. It is not known whether differences in passing scores on current tests reflect methodological differences in the standard-setting process, differences in the judgments of state panels about the minimum requirements for beginning teachers, or policy makers’ adjustments to panelists’ recommendations. Also not known is the extent to which differences in states’ teaching and learning standards or differing concerns about decision errors, teacher quality, or teacher supply influence variability across states. The large variation in passing standards that occurs among states that use the same test is not a phenomenon unique to teacher tests. For example, the differences in passing standards among states are larger for the law bar exam than they are on Praxis I, and these differences occur even though virtually every state claims that its bar exam is testing for minimum or basic competency to practice law (Wightman, 1998). States differ in the specific bar exam test score level of proficiency that they believe corresponds to this standard. The same is true in teaching. CONCLUSION The primary goal of licensing beginning teachers is to ensure that all students have competent teachers. Teacher licensing is under the authority of individual states. There are 51 unique licensure systems in the United States, with some commonalties however. As in other professions, teacher licensing relies on more than tests to judge whether candidates have the knowledge, skills, abilities, and dispositions to practice responsibly. Teacher candidates generally must fulfill education requirements, successfully complete practice teaching, and receive the recommendations of their preparing institutions. These requirements help ensure that a broad range of competencies are considered in licensing new teachers.
OCR for page 69
Testing Teacher Candidates: The Role of Licensure Tests in Improving Teacher Quality Initial teacher licensure tests are designed to identify candidates with some of the knowledge and skills needed for minimally competent beginning practice. The tests currently used measure basic skills, general knowledge, content knowledge, and knowledge of teaching strategies. The tests are designed to separate teacher candidates who are minimally competent in the areas assessed from those who are not. Initial teacher licensure tests do not provide information to distinguish moderately qualified from highly qualified teacher candidates nor are they designed to test all of the competencies relevant to beginning practice. States decide whether to use tests and what tests to use to license beginning teachers. Each of the 42 states that requires tests uses a different combination of them, uses them at different points in the candidate’s education, and sets its own passing scores on them. Several hundred different initial licensure tests are in current use. Two test developers, Educational Testing Service (ETS) and National Evaluation Systems (NES), develop the vast majority of these tests. States set passing scores on licensure tests based on judgments about the levels of knowledge and skill needed for minimally competent beginning teaching in their state. Although many states rely on commonly used standard-setting procedures, there is little documentation about these procedures and how states actually use this information in arriving at a final decision about passing scores. In attempts to raise teacher standards, some states have recently raised their passing scores on particular tests. Some report having set passing scores that are higher than those of other states. The committee draws the following conclusions from these findings: Because a teacher’s work is complex, even a set of well-designed tests cannot measure all of the prerequisites of competent beginning teaching. Current paper-and-pencil tests provide only some of the information needed to evaluate the competencies of teacher candidates. States have gradually adopted tests for teacher licensure, and test developers have made various tests available over time. Therefore, it is not surprising that states have adopted a variety of tests to license beginning teachers. States differ in how high they set passing scores. The committee does not know the extent to which this variation in passing scores reflects differences among states in standard-setting methods; state teaching and learning standards; the characteristics of applicant pools; or different concerns about measurement error, teacher quality, or teacher supply. Appropriate, technically sound tests are difficult and costly to develop. Collaborations among states participating in the Interstate New Teacher Assessment and Support Consortium and other states, professional associations, and test developers bring the intellectual and financial resources of several organizations to this difficult work.
Representative terms from entire chapter: