9
The Impact of Certification on Teachers’ Career Paths

A fundamental reason for establishing the National Board for Professional Teaching Standards (NBPTS) and offering advanced-level certification was to make the teaching profession more appealing for high-performing teachers, thus encouraging them to remain in it. Hiring difficulties and the loss of good teachers are significant problems in many jurisdictions. Well-prepared, experienced teachers—particularly math and science specialists who are likely to have other higher paying career options—are in short supply. Experienced teachers often find that the only way to advance in their careers is to move out of the classroom and become administrators or to leave teaching entirely. Teaching has also been less likely to attract the most successful students from top undergraduate programs, in part because of comparatively low pay scales, lower prestige, and flat career trajectories (National Commission on Teaching and America’s Future, 2003).

The committee addressed the possibility that a program that offers advanced-level certification for teachers could make the field more attractive and thus mitigate these problems. In Chapter 2, we posited that such a program might be expected to improve the conditions that affect teachers’ career decisions in several ways. Such a program could help to professionalize the field, lead to higher pay for teachers who obtain board certification, lead to expanded opportunities for leadership in the field, and increase their satisfaction with their careers. Such improvements could, in turn, impact the career paths of teachers. Once again, we turn to Figure 2-1 for a visual display of how these factors could interact to improve the teaching profession.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 198
9 The Impact of Certification on Teachers’ Career Paths A fundamental reason for establishing the National Board for Profes- sional Teaching Standards (NBPTS) and offering advanced-level certification was to make the teaching profession more appealing for high-performing teachers, thus encouraging them to remain in it. Hiring difficulties and the loss of good teachers are significant problems in many jurisdictions. Well- prepared, experienced teachers—particularly math and science specialists who are likely to have other higher paying career options—are in short supply. Experienced teachers often find that the only way to advance in their careers is to move out of the classroom and become administrators or to leave teaching entirely. Teaching has also been less likely to attract the most successful students from top undergraduate programs, in part because of comparatively low pay scales, lower prestige, and flat career trajectories (National Commission on Teaching and America’s Future, 2003). The committee addressed the possibility that a program that offers advanced-level certification for teachers could make the field more at- tractive and thus mitigate these problems. In Chapter 2, we posited that such a program might be expected to improve the conditions that affect teachers’ career decisions in several ways. Such a program could help to professionalize the field, lead to higher pay for teachers who obtain board certification, lead to expanded opportunities for leadership in the field, and increase their satisfaction with their careers. Such improvements could, in turn, impact the career paths of teachers. Once again, we turn to Figure 2-1 for a visual display of how these factors could interact to improve the teaching profession. 

OCR for page 198
 THE IMPACT OF CERTIFICATION ON TEACHERS’ CAREER PATHS Our evaluation framework focused on career paths for teachers with this question: Question : To what extent and in what ways are the career paths of both successful and unsuccessful candidates affected by their participation in the program? To respond to this aspect of the evaluation, we identified three specific issues to investigate with regard to teachers’ career paths: a. What are the typical career paths for teachers? Does the career path change for those who obtain advanced certification? What are the effects on the career paths of teachers who attempt to become certified but who are unsuccessful? b. Do departure rates differ for board certified and nonboard-certified teachers with regard to leaving teaching (attrition), including those who leave classroom teaching for other jobs in schools (transition)? c. Does the program have any effects on teacher mobility within the teaching field? Does it encourage teacher mobility in ways that are beneficial for lower performing students or in ways that contribute to inequities—for example, do board-certified teachers move out of urban areas to wealthy suburban districts? Our literature review quickly revealed there are very few studies that have examined the job transitions and career changes teachers make after becoming board certified. One study (Goldhaber and Hansen, 2007) exam- ined teachers’ mobility patterns in North Carolina, and one question on the survey administered by Sykes et al. (2006) asked teachers about their future plans. In addition, follow-up surveys to the 1993 Baccalaureate and Be- yond Longitudinal Survey (B&B) collected information that allowed some basic comparison of career paths for board-certified and nonboard-certified teachers, and we conducted analyses of these data. We expected to be able to draw from the results from another national data collection, the Schools and Staffing Survey, which includes an item on board certification, but a flaw in the question rendered the data unusable. We describe the problems in more detail later in this chapter. Our analyses combined with the results from prior research provide some basic information about teachers’ longevity in the field and about teacher mobility. We were unable to locate any information that deals with teachers’ transitions out of the classroom to other positions in K-12 education. We begin this chapter with an explanation of the challenges of conduct-

OCR for page 198
200 ASSESSING ACCOMPLISHED TEACHING ing studies on teachers’ career paths and the kinds of data that are avail- able. We then turn to the issue of the kinds of career decisions that teachers make and discuss what is known about turnover rates (Subquestion b) and about the effects of teacher mobility (Subquestion c). For the most part, our analysis left us with many questions that need to be answered by additional studies, and we conclude the chapter with suggestions for the kinds of data that need to be collected. CHALLENGES IN STUDYING TEACHERS’ CAREER PATHS One way to examine teachers’ career path decisions is to simply ask them about the decisions they have made and the reasons for them. Surveys and interviews can be used for this, although obtaining an adequate sample and following teachers over time can be an arduous undertaking. Another possibility is to use existing data from large-scale administrative data sys- tems. Currently, there are three potential sources for such information— data systems maintained by states and two surveys of teachers conducted by the U.S. Department of Education. We discuss the benefits and limitations of each below. State-Level Data Systems Some states maintain administrative data systems on teachers, their characteristics, and their teaching assignment. When linked longitudinally, these systems allow the tracking of teachers from the time they are first hired in the state throughout their education careers in the public school system in that state. Such data systems can also provide information on the types of schools in which teachers are employed. These data are useful for investigating patterns of teacher mobility within the state and, depending on the type of information maintained, possibly on transitions to nonteach- ing positions in the state’s public school system as well. When a teacher leaves the state’s public school system, however, the tracking ceases. Thus these systems usually provide little or no information on these departures. Of those who left the state’s public school system, state data systems generally cannot distinguish whether the teacher moved to a teaching job in a private school (in or out of the state), moved to a teaching job in a public school in another state, or left K-12 teaching entirely. As a result, examining attrition from the teaching field usually requires national data.

OCR for page 198
20 THE IMPACT OF CERTIFICATION ON TEACHERS’ CAREER PATHS Schools and Staffing Survey One national source for studying teacher attrition and retention is the Schools and Staffing Survey (SASS), administered by the National Center for Education Statistics (NCES) of the U.S. Department of Education. To date, five independent cycles of SASS have been completed: 1987-1988, 1990-1991, 1993-1994, 1999-2000, and 2003-2004. SASS is an unusu- ally large survey. Each cycle of SASS administers survey questionnaires to a random sample of about 53,000 teachers, 12,000 principals, and 4,500 districts, representing all types of teachers, schools, districts, and all 50 states. The most recent (2003-2004) survey asked teachers about board cer- tification, but the question was ambiguous. It asked teachers if they “have taken any of the following tests?” and the list included “an exam for NBPTS.” Initial analyses of the responses to this question revealed that approximately 540,000 teachers (roughly 14.5 percent of all teachers in the sample) indicated they had earned this credential. This figure differs sharply from the numbers of board-certified teachers in the country reported by the NBPTS, which as of 2004, was approximately 40,217. This suggests that the vast majority of survey respondents misunderstood this question. Thus, as of this writing, the SASS database cannot be used to compare attrition or retention rates for board-certified and nonboard-certified teachers. Baccalaureate and Beyond Longitudinal Study A second national data collection effort by NCES, the Baccalaure- ate and Beyond Longitudinal Study of 1993 (B&B 93:03), focuses on the postbaccalaureate experiences of college students who graduated in 1992- 1993. Potential sample members were identified through the cross-sectional National Postsecondary Aid Study of 1993. The subsample eligible for the B&B 93:03 study consisted of approximately 11,200 college graduates who received their bachelor’s degrees between July 1992 and June 1993. Students eligible for the B&B study were asked questions about their plans for the future, particularly expectations for employment and graduate edu- cation. Follow-up studies were conducted in 1994, 1997, and 2003, and information was obtained about postbaccalaureate experiences, including information about employment experiences, such as occupation, salary, and job satisfaction. A unique focus of the B&B 93:03 follow-up was on graduates who had considered or entered teaching. Approximately 2,000 of the college gradu- ates in the sample had taught at some point between 1993 and 2003. These teachers were asked about preparation, initial licensure, grades and subjects taught, job satisfaction, and reasons for staying in or leaving teaching.

OCR for page 198
202 ASSESSING ACCOMPLISHED TEACHING The 2003 follow-up included, for the first time, a question about na- tional board-certification status. Respondents who had previously indicated that they were licensed to teach at the K-12 level were asked “if they were working toward or had already earned a national board certificate (issued by the National Board for Professional Teaching Standards (NBPTS)).” Of those who were asked, 45 indicated that they were working toward board certification and 104 said they had obtained board certification (roughly 7.5 percent). While this is a very small sample, it does provide some informa- tion that can be used to compare retention in teaching for board-certified and nonboard-certified teachers. The data set does not contain any infor- mation about the schools in which teachers were employed or the types of teaching assignments they held. Thus, this data set cannot be used to study teacher mobility at the national level. TEACHER TURNOVER Retention and Attrition A large body of research documents the extent of teacher attrition from the profession. A recent estimate based on the SASS data indicated that approximately 6 percent of teachers leave the profession each year (Ingersoll, 2001, p. 521). Approximately 33 percent of teachers leave within the first three years of teaching and 46 percent leave within the first five years (Ingersoll, 2002b, cited in National Commission on Teaching and America’s Future, 2003). When asked about their reasons for leaving the profession, nearly half (49 percent) reported dissatisfaction or pursuit of another job (Ingersoll, 2001). For those who reported that they were dis- satisfied with the profession, the top reasons they cited were poor salary, lack of student motivation, inadequate administrative support, and student discipline problems. These data address attrition for all teachers, and few data sources con- tain the kind of information needed to compare attrition rates for board- certified and nonboard-certified teachers. Sykes et al. (2006) reports survey results that addressed this issue. One survey question, which was intention- ally worded to be similar to a question posed on the SASS questionnaire, asked respondents: “How long do you plan to continue teaching?” The researchers compared the responses of their board-certified respondents from Ohio and South Carolina with those of all teachers surveyed in SASS 1999-2000. Their results are presented in Table 9-1. These results, while hardly definitive, suggest that board-certified teach- ers in these two states are more likely than other teachers to indicate that they plan to remain in teaching and less likely to indicate that they plan

OCR for page 198
20 THE IMPACT OF CERTIFICATION ON TEACHERS’ CAREER PATHS TABLE 9-1 Comparison of the Reponses of Board-Certified and All Teachers to the Survey Question: How Long Do You Plan to Continue Teaching? I plan to leave teaching as As long as I am able to: soon as I can: Board-Certified Board-Certified Teachersa All Teachersb Teachersa All Teachersb State Ohio 52% 38% 0.2% 2% S. Carolina 49% 35% 0.2% 5% aBased on Sykes survey of teachers in Ohio (n = 587) and South Carolina (n = 566). bBased on SASS 1999-2000 results for 1,525 teachers. SOURCE: Data excerpted from Sykes et al. (2006, Table 4). Reprinted with permission from the National Board for Professional Teaching Standards, http://www.nbpts.org. All rights reserved. to leave teaching. To further explore this issue, we conducted a small-scale analysis of the B&B data set. The B&B 93:03 data set followed 1993 college graduates for 10 years. Of those 1993 college graduates who had gone into teaching, the 2003 follow-up obtained information on whether they were still doing so as of 2003 and on whether they had ever attained board certification. The B&B includes 204 teachers who indicated that they were board certified. Because this is such a small sample from a single cohort (the class of 1993), we would expect its composition to differ from those in the NBPTS database of all teachers who ever applied and obtained board certification.1 For example, we found that 6.7 percent of the B&B 93:03 sample obtained board certification, compared with less than 2 percent of all teachers in the nation. Moreover, the B&B 93:03 sample contains a greater propor- tion of men (25.7 versus 11.2 percent from the NBPTS database), fewer whites (84.6 versus 90.1 percent in the NBPTS database), more African Americans (7.8 versus 4.7 percent in NBPTS database), and fewer teach- ers with advanced degrees (42.6 versus 61.6 percent in NBPTS database). Not surprisingly considering the timing of the survey, the B&B sample is younger (on average 35.2 versus 40.3 years) and less experienced (8.9 ver- sus 12.6 years in the NBPTS database). The B&B 93:03 sample also has a greater proportion of teachers working in elementary schools (61.9 versus 51.8 percent in the NBPTS database). Despite these differences between the two data sets, we used the B&B 93:03 to examine the attrition rates between 1993 and 2003 for board- 1 Chapter 6 provides additional details about the NBPTS data set that we used for these analyses.

OCR for page 198
20 ASSESSING ACCOMPLISHED TEACHING certified and nonboard-certified teachers. While the majority of teachers in the B&B 93:03 began teaching in 1993 or 1994, some did not do so until subsequent years. We therefore estimated attrition rates separately for teachers depending on when they entered the teaching profession (those who began teaching between 1993 and 1995 and those who began teach- ing between 1996 and 1999). If teachers were no longer teaching in 2003, they were labeled as “leavers,” and those who remained in teaching were labeled as “stayers.” Table 9-2 presents the attrition rates for the two groups of teachers, separated by when they entered teaching and their board certification status. Overall, attrition rates were lower, at a statistically significant level (p < .05), for the B&B 93:03 sample of board-certified teachers than for the nonboard-certified teachers. For those who began teaching between 1993 and 1995, 31.8 percent of board-certified teachers had left teaching by 2003 compared with 35.6 percent of the nonboard-certified teachers. For teachers who entered the field between 1996 and 1999, 26 percent of the board-certified teachers had left teaching by 2003 compared with 34 percent of those without board certification. These analyses are described in additional detail in Perda (2007). It is important to point out that these analyses were descriptive in nature and do not allow us to conclude that obtaining board certification TABLE 9-2 Teacher Attrition Rates by 2003 National Board- Certification Status Total Attrition Stayers Leavers Teachers Rate Board-Certified Teachers: Started initial teaching job between 1993 and 1995 6,073 2,838 8,911 31.8% Started initial teaching job between 1996 and 1999 1,703 599 2,302 26.0% Nonboard-Certified Teachers: Started initial teaching job between 1993 and 1995 94,541 52,275 146,816 35.6% Started initial teaching job between 1996 and 1999 31,619 16,288 47,907 34.0% SOURCE: Perda (2007, Table 10), based on data from B&B 93:03.

OCR for page 198
20 THE IMPACT OF CERTIFICATION ON TEACHERS’ CAREER PATHS caused teachers to stay in the field longer. The fact that board-certified teachers were less likely to leave the profession does not necessarily imply that the process of going through certification increased their attachment to the profession. Indeed, standard human capital theory shows that workers who intend to remain in a profession longer are more likely to undertake costly activities to enhance their skills (Ben-Porath, 1967). Put differently, teachers who feel strongly tied to teaching as a career are the most likely to find board certification worthwhile. In this sense, board certification may provide administrators with a signal of teachers’ preexisting commitments to remain in the profession. Mobility and Transition Goldhaber and Hansen (2007) investigated the impact of board cer- tification on the job transitions and career paths of teachers employed in North Carolina. Their primary sample included most of those who taught in the state public schools between the 1996-1997 and 1999-2000 school years, and they restricted the analyses to teachers who had at least three years of experience (and were thus eligible for board certification) but less than 30 years of experience (to eliminate mobility due to retirement). This teacher sample was tracked over an eight-year period from 1997 to 2003, and data were obtained on several types of job transitions during this pe- riod: (1) moving to another teaching position at a different public school within the same district; (2) moving to another teaching position in another public school district within the state; (3) leaving the North Carolina public school system. As with other state databases, the data used in this study provided no information on whether those in the latter category—leaving the North Carolina public schools—had moved to a public school job out of state, had moved to a private school job in or out of the state, or had left teaching entirely. This is an important limitation because it means the study could not specifically isolate the influence of board certification on attrition from the teaching occupation. For the above three job transitions included in the database, the analy- ses made several comparisons: (1) those who obtained board certification versus those who had not; (2) those who had never applied for board certification versus those who had applied; and (3) among those who had applied, the analysis compared successful applicants with unsuccessful ap- plicants. Because those who apply for board certification may be different than those who never apply, the latter comparison among successful and unsuccessful applicants is especially useful to mitigate selection bias. In addition, the analyses were broken out by teachers’ experience level and by race, because the data showed differences among these groups in the likelihood of passing and in the impact of obtaining board certification. As

OCR for page 198
20 ASSESSING ACCOMPLISHED TEACHING a result of these many groupings—by type of transition, by board certifica- tion status, by experience, and by race—there were many permutations of possible comparisons in the results. The analyses used competing-risks models to estimate the hazard (i.e., probability) of an individual experiencing each of the types of job transi- tions, after controlling for a series of teacher and school characteristics. To further mitigate selection bias, the analyses also used a quasi-experimental method—regression-discontinuity analyses—in the comparisons of the suc- cessful applicants with unsuccessful applicants. This method was used to estimate the effects of successfully passing the NBPTS assessment on two outcomes: (1) the likelihood of experiencing one of the above three job transitions and (2) on the characteristics of the new schools to which they moved. The results of the analyses showed that, overall, those who obtained board certification had more mobility than those who were not board certified. However, the analyses also showed that these differences lay not so much with those who had never applied for board certification (the ma- jority of teachers in the state), but rather, the differences in mobility were primarily found among those who had applied: between the successful ap- plicants and unsuccessful applicants. A more nuanced picture emerged when comparing the latter two groups. Although the coefficients were not always statistically significant across the different teacher experience levels, the direction of the signs was consistent. Those who passed the assessment and obtained the certification were more likely to move between schools and districts and more likely to leave the North Carolina public school system than were those who applied but did not pass the assessment. A different picture also emerges depending on the race of the teacher. For successful African American applicants, the results indicate that board certification has little impact on career mobility. EFFECTS OF MOBILITY In Chapter 6, we presented results from a study by Humphrey, Koppich, and Hough (2005) that described the characteristics of the schools in which board-certified teachers work in six states. Their results documented that board-certified teachers are not equitably distributed across schools in these states (with the possible exception of the Los Angeles school district) and tend to work in schools with higher achieving, advantaged students. Inequality in the distribution of the most qualified teachers is not a new finding. Numerous studies have shown that higher poverty, more disadvantaged schools have less qualified, less experienced, lower scoring teachers (e.g., Boyd, Lankford, Loeb, and Wyckoff, 2005; Stinebrickner, Scafidi, and Sjoquist, 2003). In short, data have long documented that there

OCR for page 198
20 THE IMPACT OF CERTIFICATION ON TEACHERS’ CAREER PATHS is an unequal distribution of high-quality teachers, regardless of the criteria used to define teacher quality (e.g., years of experience, master’s degrees, undergraduate subject-matter major, or board certification). Moreover, in general when teachers change teaching jobs, they tend to move to more advantaged schools (Ingersoll and Perda, 2008). Hence, along with the issue of an unequal distribution of board-certified teachers, there is the related question of how teacher mobility impacts their distribution. Humphrey et al. (2005) did not establish where teachers were working before they become board certified and, thus were unable to investigate whether teachers move to (or away from) schools with high-needs students after earning board certification. However, Goldhaber and Hansen (2007) did examine this issue. Their analysis compared the nature of the school moves made by those who earned board certification compared to teachers who were unsuccessful applicants. The school characteristics the authors examined included the percentage of enrolled students in poverty, the per- centage of minority students, per-pupil expenditures, and median housing values in the district. The authors reported the results separately for white and African American teachers. For white teachers, the results were generally weak. There was some consistency in the direction of the sign of the coefficients; that is, white board-certified teachers tended to move to schools with fewer students in poverty and fewer minority students than did unsuccessful applicants. However, most of the coefficients were neither statistically nor substantively significant. For example, there was generally less than a 1 percent difference between successful and unsuccessful applicants in terms of the percentages of students in poverty at the schools to which they moved, and differences in the range of 1 to 2.5 percent in the percentages of minority students at their new schools. For African American teachers, the results were also generally weak, with the exception of the percentages of minority students at the schools to which board-certified teachers moved. That is, compared to unsuccess- ful applicants, African American board-certified teachers tended to move to schools with fewer minority students, and the differences were generally large and statistically significant. With regard to the other school charac- teristics studied (percent of students in poverty, per-pupil expenditures, and median housing values), differences were not significant, and the sign of the coefficients was not consistent. Thus, the Goldhaber and Hansen study indicates that for white teacher applicants, obtaining board certification is associated with a substantial increase in career mobility at all levels (interschool, interdistrict, and out of the state system), but it is not clear whether their new schools were defini- tively different from those to which otherwise equal teachers moved. For African American teacher applicants, the results indicate that board certi-

OCR for page 198
20 ASSESSING ACCOMPLISHED TEACHING fication has little impact on career mobility, but it has a fairly substantial impact on the racial composition of the schools to which they moved. CONCLUSIONS AND RECOMMENDATIONS The available research on career paths suggests that teachers who earn board certification may remain in the field longer than teachers who do not earn it. It also suggests that teachers who earn board certification become more mobile, and we speculate that they may possibly use the certification as a means for leaving the state to work elsewhere. We note that these are tentative conclusions based on the results from two studies and our own analyses. The findings need to be corroborated before any solid conclusions can be drawn. The available evidence is clearly insufficient to answer the questions we posed in our evaluation framework. While some sources document aspects of the career path, these sources do not allow comparisons between board- certified and nonboard-certified teachers. Given that a major objective of the NBPTS is to provide a means for encouraging teachers to remain in the profession, we think it is important to study the career paths of board- certified teachers as well as the impact the credential has had on teachers’ career decisions. We understand that the NBPTS has recently begun to investigate this issue and is in the process of collecting information from board-certified teachers about their current employment status. We did not have the op- portunity to review plans for this analysis, but we encourage the board to pursue this avenue of research using scientifically sound sampling proce- dures, instrument design, and analytical methodology. One way to conduct such research would be to identify a specific time frame and select a ran- dom sample of teachers who applied for board certification (both success- fully and unsuccessfully) during that time frame. It might be advisable to oversample teachers from specific groups, such as racial/ethnic minorities. A questionnaire could then be distributed to the sample to inquire about the career options they have pursued since applying for board certification. Comparisons of responses for successful and unsuccessful candidates would address questions about the impact of the credential on career paths. Specifically, we recommend: Recommendation 9-1: The NBPTS and other researchers should study the subsequent career choices of teachers who have applied for board certifica- tion. The information they collect should be analyzed for successful and unsuccessful candidates separately so the correlation between board certifi- cation and career choice can be evaluated. Studies that track teachers over long periods should also be used to test whether the process alters career

OCR for page 198
20 THE IMPACT OF CERTIFICATION ON TEACHERS’ CAREER PATHS choices. The data collected should include information about the extent to which state or district policies influenced the respondents’ career choices. We also encourage further investigation of the impact of board certi- fication on teachers’ career choices using national data sets, such as SASS. As a first step, we recommend that NCES further investigate the problems with the question on the SASS questionnaire, which asked respondents about their board-certification status. If the problems are indeed caused by the wording of the question, we suggest that alternative wordings of this question be pilot-tested before operational use to ensure that accurate information is collected when the survey is repeated. We also encourage education researchers to conduct analyses at the state level using procedures such as those employed by Goldhaber and Hansen (2007). Both national and state-level studies should consider a broad set of events in teaching ca- reers in order to distinguish the effects of board certification from those of other kinds of career enhancements (such as obtaining an advanced degree or a promotion). Together these kinds of research could vastly improve understanding of the impact of board certification on teachers’ careers. On this point, we specifically recommend: Recommendation 9-2: The National Center for Education Statistics should amend the Schools and Staffing Survey so that it collects information about respondents’ board certification status. In designing the survey questions on this topic, the National Center for Education Statistics should pilot-test alternate versions to ensure that respondents will accurately understand the question. Recommendation 9-3: Researchers should use the data available from state- level data systems to expand the evidence on the mobility of board-certified teachers. These studies should use methodologies that permit comparisons of teachers’ career choices before and after becoming board certified and should compare the choices of unsuccessful applicants for board certifica- tion, teachers who successfully obtained the credential, and teachers who did not apply for board certification.