Page 413

Appendix B—
Federal and State Interviews

This appendix provides information on the interviews conducted at the federal and state levels as input to the discussion of research infrastructure and related issues in Chapter 10 and Appendix A. It presents a list of those interviewed and the interview protocol used with people who know about federal and state research.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 413
Page 413 Appendix B— Federal and State Interviews This appendix provides information on the interviews conducted at the federal and state levels as input to the discussion of research infrastructure and related issues in Chapter 10 and Appendix A. It presents a list of those interviewed and the interview protocol used with people who know about federal and state research.

OCR for page 413
Page 414 List Of Interviews: Federal Judith Anderson National Institute on the Education of At-Risk Students Office of Educational Research and Improvement U.S. Department of Education Eve Bither Acting Director of Office of Reform Assistance and Dissemination Office of Educational Research and Improvement U.S. Department of Education John Chapman Senior Budget Analyst Budget Office Division of Elementary, Secondary, and Vocational Analysis U.S. Department of Education Daryl Chubin Division of Research Evaluation and Communication U.S. National Science Foundation Joseph Conaty Acting Director National Institute on Student Achievement, Curriculum and Assessment Office of Educational Research and Improvement U.S. Department of Education Lou Danielson Division of Educational Services U.S. Department of Education James English Office of Migrant Education U.S. Department of Education Eugene Garcia Director Office of Bilingual Education and Minority Language Affairs U.S. Department of Education Gilbert Garcia Office of Educational Research and Improvement U.S. Department of Education Kristin Gilbert Education Program Specialist Migrant Education U.S. Department of Education Alan L. Ginsburg Director, Planning and Evaluation Service Office of the Under Secretary U.S. Department of Education Norman Krasnegor Human Learning and Behavior Branch National Institute for Child Health and Development U.S. Department of Health and Human Services Preston Kronkosky President Southwest Educational Development Laboratory Austin, TX

OCR for page 413
Page 415 Michael Lopez Branch Chief, Research, Demonstration & Evaluation Branch Office of the Commissioner Administration on Children, Youth, and Families U.S. Department of Health and Human Services Edith McArthur Acting Program Director Data Development National Center for Education Statistics U.S. Department of Education Denise McKeon Director of Outreach American Educational Research Association Washington, DC Barry McLaughlin Former Co-Director Center for Research on Cultural Diversity and Second Language Learning University of California, Santa Cruz Martha Moorehouse Office of Planning and Evaluation U.S. Department of Health and Human Services Mollie Oliveri Branch Chief Behavioral, Cognitive and Social Sciences Research Branch National Institute of Mental Health U.S. Department of Health and Human Services Amado Padilla School of Education Stanford University Ray S. Perez U.S. Army Institute for the Behavioral and Social Sciences U.S. Department of Defense Valena Plisko Policy and Evaluation Services U.S. Department of Education Sharon Robinson Assistant Secretary Office of Educational Research and Improvement U.S. Department of Education Jeff Rodamar Planning and Evaluation Service U.S. Department of Education Edward Simermeyer Director, Office of Indian Education Indian Fellowship Program U.S. Department of Education Robert Slavin CSOS-CRESPAR The Johns Hopkins University Gerald Sroufe Director of Governmental & Professional Liaison American Educational Research Association Washington, DC G. Richard Tucker Department of Modern Languages Carnegie-Mellon University

OCR for page 413
Page 416 Interview Protocol For The Federal Government Organization and Administration 1. What are the mission and objectives of your office (unit, division, etc.)? 2. How is your office (division, unit, etc.) organized and administered? (Please provide an organizational chart.) 3. If your office (etc.) is responsible for activities other than research—provision of services, processing and monitoring of grants, statistics—how do the research activities relate to these other activities? 4. Have there been major organizational changes in the last five years that would affect research on bilingiual/LEP students? If so, what are they? What Is Funded For each of the following areas, for the past five years, please elaborate on what was funded, including amount and principal investigators/organizations for funded work. If you haven't done so already, can you provide us with a list of these projects and any other written information you might have describing these projects? 1. Does your office (department, agency, etc.) support research related to the education of limited-English-proficient and bilingual students including:   • basic research on the linguistic, cognitive, and social processes involved in the education of limited English proficient and bilingual students;   • applied research looking at effective instructional practices and schooling;   • demonstrations, evaluations; and   • survey research to find out about demographics, educational context, and student outcomes, including the supply of educational researchers and teachers?   Please elaborate on the above. 2. Does it support centers, labs, or other entities that conduct research in the above areas on bilingual/LEP students? Is so, please elaborate. 3. Does it support information services (clearinghouses) or resource centers? If so, please elaborate. 4. Does it support training grants for scholars who work on the education of LEP and bilingual students? If so, please elaborate. 5. Was any of this work field-initiated? If so, please elaborate (how much? what?). 6. Was any of this work intramural? If so, please elaborate (how much?

OCR for page 413
Page 417   what?). Are there staff with expertise in this area working in your office (unit, division, agency, etc.)? Development of Research Agenda 1. Do you have a research agenda? How is the research agenda in your agency, office determined? Who is involved in determining the agenda, and what is the process by which it is developed? How are priorities determined? 2. How much is determined within the agency? How much is determined by legislation or other outside entities? What legislation or entities? Procurement 1. What is the process for developing the RFPs? Is it a formal or informal process? Are there standards? If so, what are they? Who is involved? How are decisions made? 2. What is the process for soliciting proposals and making the awards? Do you have a peer review process? Are there explicit standards? If so, what are they? Monitoring 1. What is the process for monitoring the projects, both while they are implemented and before the final reports are released (i.e., are there report review and clearance procedures)? Accumulation of Results 1. Are there any mechanisms in place to provide for review and syntheses of the research that is funded? If so, please elaborate. 2. Are there mechanisms in place for consensus development of the research literature? If so, what are they? 3. Is there an archiving system in place? If archiving mandatory? If so, with whom? Collaboration 1. Do you collaborate (coordinate) with other offices, divisions, departments, institutions in the process of carrying out of research on bilingual/LEP students? If so, can you comment on benefits, drawbacks regarding this collaboration/coordination?

OCR for page 413
Page 418 Dissemination/Linkages 1. Is there a mechanism, process for dissemination? 2. If so, to whom and how are research findings disseminated? 3. Are there any efforts to connect the research efforts/findings to practice? If so, what are they? Strengths? Weaknesses? 4. Are there any efforts to connect the research findings to policy? If so what are they? Strengths? Weaknesses? Obstacles/Barriers to the Sponsorship of Research in This Area 1. Can you comment on obstacles/barriers to the sponsorship of research in this area? For example, are any of the following responsible to creating obstacles or barriers:   • amount of money available;   • infrastructure issues—control of money, coordination/collaboration across funders, leadership, process for development of RFPs, for making and monitoring awards;   • political climate; and   • quality and quantity of applicants? Promising Efforts 1. What are some promising efforts in the sponsorship of research (i.e., coordination both across offices and across types of research, larger and longer commitments for study and experimentation on critical issues, more field-initiated research)? 2. What do you see as the three most significant research endeavors in the last five years? Why? Supply of Educational Researchers 1. Does your unit support the training of educational researchers who work on language-minority education issues, through grants directly to colleges and universities or to students in the form of fellowships? Please elaborate. 2. Is there any information on the effectiveness of these efforts? Please elaborate. 3. Do you have any ideas or can you think of any examples of particularly effective efforts to train educational researchers, to increase the number of minorities working in this area?

OCR for page 413
Page 419 Supply of Teachers 1. Does your unit support the training (preservice and inservice) of teachers of language-minority students? Please elaborate. 2. Is there any information on the effectiveness of these efforts? 3. Do you have any ideas or can you think of any examples of particularly effective efforts to:   • train teachers to work with language-minority students or   • increase the number of teachers who work with language-minority students, especially language-minority teachers who are themselves bilingual? List Of Interviews: State Ms. Lupe Castillo Title VII Coordinator Department of Education New Mexico Mr. Bernardo Garcia Director, Office of Multicultural Student Language Education Florida State Department of Education Mr. James Greco Manager, Office of Bilingual/Dicultural Education California Department of Education Ms. Anne Kessler Program Manager Bilingual Education Program Alaska State Board of Education Ms. Josephine Pablo Director, Title VII, ESEA General Education Branch/Languages Section Hawaii State Department of Education Ms. Verma Pastor Bilingual/Migrant Program Director Arizona Department of Education Ms. Carmen Perez Director, Bilingual Education New York State Department of Education Ms. Maria Seidner Interim Director Division of Bilingual Education Texas Education Agency

OCR for page 413
Page 420 State Department Of Education (Sea) Responses To Questionnaire Question 1: Does the SEA fund research on LEP and bilingual students, including basic research, applied research, demonstrations, evaluations, or survey research? Please elaborate on research funded in the past five years and amounts of funding, if available, as well as identifying the administrative units within the SEA that fund this research. State A: The SEA does not conduct research per se, although it does collect information on student performance on statewide achievement tests. Achievement testing is conducted in grades 3, 5, and 8, and students are also required to pass a tenth grade competency examination in order to graduate. It is possible to identify ethnicity (broadly defined—e.g., Hispanic, Native American), as well as gender in these state performance data. At present, these data are not reported separately by ethnicity in a state report. State B: The main state-supported research efforts are (1) evaluations of state-funded programs, (2) a census that includes counts and descriptions of English-language learners, and (3) analysis of data from the state performance testing program. With respect to evaluation, any project with state funds must conduct an evaluation. Any district that accepts state (formula) aid for bilingual education must use state-prescribed tests (the state mandates a pre/post-testing program), as well as develop program plans and evaluate its programs (not all districts accept aid, but most with significant numbers of English-language learners do). Using these data, state officials do analysis and issue an annual report to the State Board. They also provide technical assistance to districts. There is some concern about the quality of the testing data from some districts, however. There is also a state categorical funding program for bilingual education. This program has a competitive awards process, and the evaluation requirements are similar to those of the formula aid program. In addition, the state conducts a census of language-minority children, which is used to distribute state formula aid for English-language learners. From this exercise, it is possible to identify such students, as well as the type of educational programs in which they are enrolled. The state maintains individual student records data, as well as extensive information on the districts and their programs. The state's performance testing program data make it possible to identify English-language learners, as well as the programs in which they are enrolled. Performance testing is conducted at grades 3 and 6. There are also competency tests administered at intermediate and high school levels that can yield similar data. A new policy approved by the state board of education requires that the state measure English-language learner growth separately in the state assessment

OCR for page 413
Page 421 program (for many years these students were exempted from statewide testing, but under the new policy, districts must use tests in native languages, if available). Although not currently a research activity, SEA staff anticipate that it will soon be necessary to develop new teacher certification examinations (this will probably be a Request for Proposals process). State C: The main state research activity is district-level data collections to develop a state report card. The state prescribes the data elements and definitions (including an LEP definition), and districts provide reports to the SEA. The SEA produces a state summary. Among the data districts are required to submit are data about English-language learners: performance on state assessment (see below) and ethnic/linguistic background. The state maintains a student-level data records system. At present, the data on English-language learners are not aggregated at the state level in a report. They are used, however, to allocate state aid funds to districts. The state assessment system requires tests at grades 4, 8, and 11. Many districts test students at all grades, however. The data collected by the state allow for the identification of English-language learners by school. It is also possible to isolate the performance of students in bilingual and bicultural programs. In addition, the state received Title VII funds to work with districts. State officials monitor projects and do site evaluations. There is an Academic Excellence site in the state that has conducted research on English-language learner performance. State D: This state conducts an annual (fall) district-level survey, yielding data by school and by individual student for its state management information system. These fall surveys provide data on enrollment, race, ethnicity, and language. It is possible to identify LEP, LEP/bilingual, LEP/special education, and LEP/parental denial status. An individual student-level record system is maintained by the state. (There is a weight in the state aid formula for English-language learners, and this data collection is used to assist in the distribution of formula aid.) From this data collection, the SEA publishes ''snapshots" of district-level data. Aggregated program data on English-language learners are currently published by grade and region of the state, but it would be possible to disaggregate these data further. The state's performance assessment system collects school-level data. Information on attendance and dropout rates, as well as achievement data, are analyzed by the state. There is also an annual criterion-referenced third grade testing program in reading and math, as well as other grades in some years. The state receives school-level data and can separately examine the performance of schools with concentrations of language-minority students.

OCR for page 413
Page 422 There is a state testing program in Spanish for grades 3-6 for English-language learners in math, reading, and writing. The program is currently undergoing new test development and beginning to implement new criterion-referenced tests. At present, some of the tests are being "benchmarked," and standards are being developed. Others are being field tested. Districts that receive state program funding for bilingual education are required to do pre- and post-testing of students using these tests. Under contract, the SEA is also developing a new language proficiency test for bilingual teacher certification. The development process involves identifying skills needed by teachers and assessing the strengths and weaknesses of the current test. The SEA research and evaluation division has conducted various evaluation studies, including a study of pre-K programs for English-language learners. State E: This state has undertaken two specific research studies in the recent past with a focus on bilingual education. First, it issued a contract to a university in the state to develop an English as a second language (ESL) curriculum manual. To develop the manual, the university researchers undertook a "best practices" study, as well as a survey of districts and professional organizations. The aim was to develop ways of identifying levels of English proficiency of English-language learners. The results will be published soon. Second, the assessment and bilingual offices of the SEA cooperated to conduct two symposia, bringing together experts on assessment, surveys, and legal issues on English-language learners and bilingual education programs. This state also compiles an annual report on the status of English-language learners and their programs. The report provides district-level data on the participation and performance of English-language learners by type of program, assessment status, former English-language learners (i.e., those who have exited the program within the past 2 years), and non-English-language learners. Background data shown include native language, country of origin, and race/ethnicity. This report is based on individual student records submitted by districts (under an automated state student information database), but only aggregated district-level data are currently published. State F: The SEA conducts needs assessments, evaluations, and on-site monitoring of Title VII projects and sends reports to OBEMLA. In addition, the state-funded LEP program requires districts to develop evaluation plans and collect achievement data on the English-language proficiency of participants. Several years ago, the state conducted a specific research activity to examine the relative benefits of pull-out and self-contained classrooms. The state assessment program's regular achievement testing applies to English-language-proficient students only. Some districts also give native-language proficiency tests for diagnostic purposes. Through school-level data submitted to

OCR for page 413
Page 423 the state from the regular testing program, it is possible to estimate, by school, the percentage of English-language learners (by ethnicity) achieving at grade level. State G: The state conducts an annual census of public school districts (and cooperating private schools) that collects school-level data on a wide variety of variables. The data are analyzed in the bilingual office, and results are published in a yearly report. They include (by school) the number of English-language learners (based on a state legal definition), the number of students reassessed, the number of languages spoken, the types of programs in which students are participating, the number of qualified bilingual teachers, and what tests are used. The SEA used to collect dropout data by ethnic group, but the respondent was unsure whether these data are still being collected. For the past 5 years, the state has operated a performance-based testing program. Tests were developed by the SEA in Spanish, but they are no longer in use. At present, testing is taking place in grades 4, 8, and 12 in reading, writing, and math, but the program is being reexamined. However, districts also conduct standardized testing in grades 4, 7, and 11 using nationally known standardized achievement tests, and it is possible to identify English-language learners by program from the test results (which are submitted to the state). Last year, the SEA analyzed district-level state-wide testing data and found that English-language learners in bilingual programs did better than those in ESL (i.e., overall, their gap with English-proficient students was smaller, and in some districts they performed better on average). State H: This state has conducted several research studies in the recent past. It published a volume that provided a theoretical framework on schooling and language-minority children to which key researchers in the field contributed. A Title VII grant was used to identify five schools in which to implement a theoretically sound program with extensive evaluation. Results of the pilot program were published, and a major city in the state revised its program based on the research. The state is currently supporting a research program examining staff development for teachers who work with English-language learners (see the entry for this state under question 2 below). In addition, the statewide assessment program (which is currently being revised) requires standardized achievement tests in grades 2 through 10. Spanish versions of tests are currently being requested of publishers who seek to have their tests approved for the statewide program. Providing such tests will enhance the publishers' chances of test adoption, but they are not required of all publishers, so some will probably not submit them. The state can analyze achievement data to determine, for example, whether English-language learners who are fluent in English do better. The state maintains a demographic database by school. This database includes

OCR for page 413
Page 424 the results of a home language survey and English-language assessments, as well as teacher counts. These data are used in compliance reviews, and a state report is issued each year. The state has a legal requirement to monitor districts on education for English-language learners every 3 years, following up where problems are identified. There is state funding for bilingual education, which is allocated by formula (weights for poverty, transiency, ethnicity). Summary: SEAs engage in a fairly limited amount of direct research on English-language learners/language-minority students or bilingual education, but many maintain student-level record databases that could be used to conduct sophisticated analysis (at least cross-sectionally). States also prescribe local evaluation requirements for state-funded programs. At present, the main direct research activities include the following: • Conducting annual (or other repeated) censuses of school districts that obtain descriptive information on counts of English-language learners by various background and programmatic variables. These data are sometimes published by district or in aggregated form. • Analyzing data from state assessment programs to determine the performance of English-language learners in relation to the performance of other students (or by LEP program or designation). A number of states maintain individual student record systems that provide background and performance data, although little is known about the quality of these data. The performance data are generally drawn from performance on English-language tests. Only a few states conduct testing in Spanish or another language. • Designing and monitoring local evaluations of funded bilingual or other programs for English-language learners/language-minority students. States play a role in monitoring and evaluating Title VII programs. They also prescribe evaluation requirements for state-funded programs, monitor implementation, and provide technical assistance. State requirements appear to be similar to those of Title VII. • Designing teacher certification exams. In the course of developing language-proficiency and other certification exams, SEAs identify and study the skills teachers need and the links between exams and performance. They also do various research tasks associated with test development. Most of this work is done on a contractual basis. Question 2: Does the SEA support centers, labs, or other entities that conduct research in the above areas? Please elaborate. State A: The state does not support any other entities engaged in research.

OCR for page 413
Page 425 Although the program is categorical, all funds for bilingual education are distributed by formula to districts, so there are no discretionary funds for use by other entities. State B: There are technical assistance centers in the state that help the SEA obtain state-requested data from districts as one function, but they do not do research. The state does work with universities to develop teacher training programs. The SEA is beginning a dialogue with several universities about research needed to design new programs for bilingual and ESL teacher certification. There is a considerable demand for additional qualified teachers. The state currently issues provisional certificates, and teachers have several years to obtain certification. In conjunction with universities, the state has spent $1 million to develop the courses necessary to enable provisional teachers to qualify and pass the test certification exam. The state has conducted a needs assessment to determine how many teachers are needed (based on the numbers of provisional certificates it has issued). State C: This state supports a language center at a branch of the state university system. The center conducts a wide variety of research on native languages and their survival (this center dates back to the Lau Centers). The results of this research are important in state policy supporting the teaching of native languages. The center publishes monographs and booklets with SEA funding. State D: There are a number of regional service centers in the state, some focusing on bilingual education. There are also university-based teacher training centers. It was not known how much research takes place at either type of facility. State E: There are no entities supported on an ongoing basis to conduct research in these areas, but sometimes there is support on an ad hoc basis for a specific research activity requested by the SEA. State F: The state helps support a federal regional educational laboratory that also receives Title VII funds. State G: There are no such entities supported by the SEA to conduct research in these areas. The SEA does keep in touch with university-based researchers who conduct research on English-language learners. State H: The state supports a language-minority program at a campus of the state university system; the program brings together scholars from several campuses.

OCR for page 413
Page 426 The SEA sometimes provides guidance on specific studies. For example, under the auspices of this program, researchers from several campuses are currently examining the question of what is the most appropriate staff development for teachers who work with English-language learners. Summary: Development of teacher training programs appears to be the main research and development activity supported by SEAs in centers, laboratories, or other entities. Interestingly, curriculum development was not often mentioned as an activity of centers. In addition, it should be noted that there are other sources of state support for research on language-minority students and English-language learners—primarily state agencies that support faculty and researchers at universities. SEAs also make contributions to the federal regional educational laboratories, although they may not always be aware of specific research on English-language learners being conducted by these laboratories. Question 3: Does the SEA support information services or training grants for scholars who work on the education of LEP and bilingual students? Please elaborate. State A: The SEA supports a project that is designing and revising the teacher examination for bilingual education teacher certification. An evaluation of the previous examination was conducted, and it concluded that a change was needed. The new exam is being developed by a committee of researchers and teachers and is currently undergoing prepiloting at one university. The project cost is $85,000. State B: The SEA is not now supporting such efforts, but there is a large need in the state. State C: The state staff development network receives funds from the SEA for training grants. The funds support research and teacher training in districts. Many of the districts that receive these funds use them in projects for native-language students. As noted previously, a state-supported university-based center conducts a variety of research functions, including training of researchers. State D: The SEA does not support such activities directly, but regional service centers in the state provide staff development. State E: The SEA supports a variety of teacher training and curriculum development projects that are related to language-minority students and involve research. It has recently developed a 300-hour inservice program on ESL and is conducting training of trainers for the program. The program has an evaluation component and supervised implementation. University researchers at a center for

OCR for page 413
Page 427 applied linguistics were heavily involved in the development of this program. The state also funded the development of a K-12 language arts program involving pilot testing and formative evaluation in a major school district in the state. State F: As noted under question 3 above, this state supports faculty at a university-based language center that conducts a variety of research activities. In addition, it makes specific grants to the university system to support research by Ph.D and M.A. candidates, as well as faculty, on issues affecting language-minority students. State G: The state pays tuition for teachers who are seeking certification in bilingual education. It is possible that some of these persons are doing research as part of their education. State H: The SEA supports teacher training, but not scholar training per se. Summary: With a few exceptions, SEAs do not directly support information services or scholar training on English-language learners. It should be noted, however, that basic state support of university and college systems also supports the research activities of graduate students and faculty. As already noted, there are, in addition, university-based centers in several of these states that specialize in research on language-minority students and English-language learners. Question 4: How does your agency decide what kinds of research to support, as well as which particular projects to fund? State A: In general, the State Board of Education decides what research to undertake, but occasionally legislative requirements drive research (e.g., a decision to revise teacher exams was made after complaints to legislators by persons taking the test). State B: The decision to require district-level planning and evaluation for state-funded bilingual programs follows the pattern established for most state-funded programs. Furthermore, the SEA has supported the decision strongly because data from these evaluations are used to decide which programs to continue to fund. The data are also used to identify low-performing schools, which then must develop plans to improve performance (schools that continue to perform poorly face takeover). In the past, English-language learner performance was not a factor in the identification of low-performing schools, but that is now changing, and LEP performance will be a separate criterion in state review. Another current consideration in research decision making is that senior state officials and the board of education want to streamline reporting. They want fewer reports and simple reports that are user-friendly. The state may adopt

OCR for page 413
Page 428 school report cards in which the performance of English-language learners would be identified. State C: SEA activity on standards and curriculum frameworks is driving the research currently being done. In that activity, standards for English-language learners/language-minority students are being identified. Also affecting current research is an initiative by the governor's office to reexamine the distribution of state aid. A task force is examining the foundation funding formula and has identified districts with funding gaps (e.g., large English-language learner population, but little tax base). The task force is exploring new ways to allocate state funds. State D: Most data collections result from legislative mandate. For example, the new education code calls for research on teacher training and learning. Availability of resources also plays a role. State E: Specific research is undertaken based on needs indicated by districts. For example, a symposium on assessment of English-language learners was offered because of district interest. District directors get together once a year to discuss priorities. Overall data collections are usually driven by legislative requirements. State F: State funding requirements play an important role in defining what research is undertaken. In addition, the state superintendent is "graded" on student performance, which also affects what research is conducted. For example, one current performance standard is that 25 percent of English-language learners will exit each year. As a result, the SEA is currently studying the efficacy of the 25 percent figure and cutoff scores for achievement tests. State G: The annual census of public school districts is the result of legislative mandate. In addition, Title VII grant rules require information on English-language learners. State H: There are three ways that research is likely to be undertaken: (1) a specific legislative mandate (e.g., the legislature wants to examine the efficacy of year-round schools); (2) a state board of education mandate (e.g., one board member wants the SEA to compare the results of English-only as opposed to transitional bilingual education); (3) staff initiative (e.g., staff bring together a consortium of university scholars to address an issue). Summary: Some respondents had a difficult time identifying the primary influences on decision making about research. In general, board of education and legislative mandates appear to play an important role. In addition, the data

OCR for page 413
Page 429 requirements of state aid formulas are instrumental in data collections in some states (counts are needed to distribute funds). Title VII information requirements and state traditions with respect to what kinds of information and evaluation accompany grant awards also appear important in establishing the planning and evaluation requirements for district-level programs that accept state aid. A few states noted that relatively new state standards for English-language learner performance are also having an effect on research priorities. The responses are notable for what they did not say as well as what they did. None of the respondents saw themselves as having a major role in decision making on research. Question 5: What is your perception of promising state efforts in the sponsorship and conduct of research on these students? State A: SEA staff have proposed a variety of research activities. For example, they have proposed a research project that would identify the achievement levels of all English-language learners, then identify the curriculum in which they are enrolled. The study would look at modalities and administration that make a difference. Studies that show the reasons for achievement (or its lack) would help justify the resources for bilingual education. The financial climate for such research is viewed as poor, however (see the entry for this state under question 6 below). State B: Officials in this state have recently developed a research agenda on English-language learners. They have posed a series of research questions. They have not yet identified sources of financial support, however. The questions suggested by the respondent during the interview included the following: How long do students (with different characteristics) need bilingual or ESL instruction? Is it possible to construct student profiles for different levels or kinds of service? Are instruments that test performance measuring what we want them to measure? Are test translations fair? Are tests appropriate for recent immigrants with little prior schooling? What programs are most appropriate for recent immigrants? How do we diagnose students' real needs—not just their LEP status? What are the relative advantages of phonics as opposed to other approaches? This is a period of massive change in the SEA. The current push is for greater accountability, and there is a strong need to answer the question, "What have we accomplished?" Research that supports accountability (e.g., examining LEP performance as part of school report cards) would fit with the change in direction. State C: The current superintendent and senior SEA staff are interested in carrying research to the student level (not just aggregated to schools or districts). Also, the SEA has been reorganized to merge data collection and assessment with teacher training in the same division. This is seen as beneficial to research.

OCR for page 413
Page 430 State D: The current effort to establish benchmarks and state standards for performance tests in Spanish is an important, positive development. Once the tests are fully implemented, it is likely that initial scores will be lower for English-language learners than for English-language students. It is hoped that analysis of these data will then be conducted and will lead to efforts to improve instruction. Better assessments should also be able to show instructional effects. Further, the performance assessment system is designed to identify whether students taking the test in English were formerly LEP (the previous year only), which should enable additional analysis of instructional effects. Districts can opt out of the assessment program for a year or so, but eventually they must participate. State E: Awareness of the need for research on English-language learners is high. There is a clear perception that new SEA policies will affect these students. The RFP to overhaul the state assessment program specifically mentions the needs of these students (testing accommodations, waivers, disaggregation of data for this population, alternative assessments). New curriculum framework development plans also refer to English-language learners' needs. In addition, there has been a legislative change; English-language learners are now encouraged to participate in compensatory education, so they will be included in research on compensatory programs. State F: The emphasis in the SEA is on collecting data that show how students are performing. The respondent believes that there is now a need to identify and study effective classroom strategies. In supporting professional education, the state encourages students in advanced-degree programs to conduct classroom research. It also encourages similar research through innovation grants to schools, some of which are being used to study programs for English-language learners (although much of the funding is used for increased services or inservice education). State G: The respondent did not identify new research efforts on the horizon. State H: There is a strong movement in the state toward greater accountability and research that supports it. One positive development is the push for state-level standards for programs serving English-language learners. Creation of such standards will require assessment to develop and then track students into various programs and approaches. Districts will have to present plans for how they will achieve the standards and then conduct research to show the results. All of these

OCR for page 413
Page 431 requirements promise a greater emphasis on research on English-language learners. Summary: The major development that seems to be setting the direction for research on English-language learners is the movement toward greater accountability for educational outcomes. Respondents in several states noted that these students are now being included in state assessment programs (sometimes with tests in native languages) and that statewide standards for their performance are being established. This development promises to increase attention to tests and other measures for assessing the performance of English-language learners, as well as to research that examines the best approaches and programs to improve performance. Question 6: What is your perception of obstacles at the state level in the sponsorship and conduct of research on these students? State A: The biggest obstacle to research is money. State officials (SEA, legislature, state board) are all supportive of research by the SEA, but there is simply not the money to do it. In general, the governor is inclined to give new resources to districts directly. In part, this is due to conflicts between the board and the governor, but it is also a function of resource scarcity. State B: The obstacles are both money and sufficient state staff to conduct research. State C: Money is quite limited at present because industries in the state are in a slump and because federal dollars are likely to decrease. Under those circumstances, it is difficult to justify additional dollars for research. In addition, shifting state leadership makes it difficult to sustain longer-term research efforts. State D: There is considerable fear among educators who deal with English-language learners about state performance assessment. They do not want to be told that the students they work with are not performing well. This fear of assessment translates into a fear of conducting research. State E: Lack of resources is the main obstacle to research. It would also help state efforts if there were a clear federal message to conduct research. The development of a centralized research function at the federal level would send an important message. Also, there should be more opportunities for states, especially large states with sizeable English-language learner populations, to conduct joint research. Such collaborations have proven beneficial in the past.

OCR for page 413
Page 432 State F: A major obstacle is the inability to obtain high-quality data from schools. In using state data for research purposes, there is an SEA clearance process. It is easier for the bilingual office to facilitate access to data for SEA or district staff, more difficult for graduate students. In general, the state should do more to encourage research, especially research on learning styles. At present, activities are undertaken only when there is sustained individual initiative. State G: Lack of money and staff are the chief obstacles to research. The SEA would like to conduct research on what is working (programmatically) in bilingual education. State H: A major obstacle to research is that the whole field of bilingual education is politically charged. There are some political officials seeking research simply to prove that bilingual education is or is not effective. It is difficult to find research that is not conducted by a ''hired gun" with a particular viewpoint. In this politically charged atmosphere, it is difficult to examine programs honestly and critically. Summary: Lack of resources is clearly a strong theme among the responses, but other important obstacles to research were noted by respondents. Several states mentioned lack of staff; this generally means numbers of staff, but it also means persons with the technical and research skills necessary for the task. Some SEAs have offices of research (or the assessment office serves that purpose), but issues of English-language learners and bilingual education need to be state priorities for these offices to focus their resources on those issues. Data quality is also a concern in some states. SEAs are often dependent on districts to supply both student-level and aggregated data. There are sometimes problems in the quality of those data—both evaluation data from projects and data from regular state-wide data collections. Resources are critically important to allow follow-up with districts when data quality is an issue. Finally, SEAs are administrative entities, but they are administered by or experience oversight from elected officials or their appointees. The responses to the question about the locus of decision making for research makes clear that these political influences are important in deciding what research gets done. As a result, research questions may be narrowly framed or may reflect a hidden (or not so hidden) political agenda.