International Education and Foreign Languages: Keys to Securing America's Future (2007)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

appendix B The Committee’s Approach to Its Review T his appendix describes the approach taken by the committee in ful- filling its charge to review the effectiveness of the foreign language, area, and international studies programs of the U.S. Department of Education (ED). It describes the activities we undertook and the conceptual model we developed to examine program effectiveness. The appendix in- cludes a number of attachments: (B-1) a summary of studies evaluating the Title VI and Fulbright-Hays (Title VI/FH) programs, (B-2) a list of written comments submitted by people and organizations knowledgeable about the programs, (B-3) agendas of the public meetings held, and (B-4) the guides we developed for the site visit interviews held at eight universities. To begin our study and ensure a complete understanding of the pro- grams’ statutory history and missions, including how their missions relate to those of other similar federal programs, the committee (1) reviewed descriptive information about the Title VI/FH programs and related federal foreign language and international education programs and (2) commis- sioned an analysis of the program’s legislative history. As a result of our review of congressional language and the statutes and the judgment of our individual members, the committee identified other questions in addition to the eight key areas specified by Congress, aimed at assessing the effectiveness of the programs at meeting their statutory missions. We asked: • How responsive are the programs to changing national priorities? What is the role of Title VI programs in relation to other programs in ad- dressing national needs? 309 

310 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES • To what extent are the programs producing best practices that are used broadly in the international education arena? How are best practices identified and shared? • To what extent does Title VI help channel other university re- sources toward national goals and fund programs or resources that they might not otherwise fund? • How effective are the programs at coordinating with one another within programs, within universities, and across universities? • How are the programs administered? What can and should be done to improve future monitoring of the programs? To guide the review and provide a model for how the programs sup- port the eight key questions and are expected to lead to particular results, we outlined a potential conceptual model. This model, which illustrates the interconnectedness of the programs, is discussed in more detail below. To assess the degree to which the programs meet the combined set of questions, the committee undertook a series of activities: • Reviewed all extant evaluations.  The committee conducted a lit- erature review, requested from ED copies of evaluations funded through its International Research and Studies (IRS) Program, and asked experts in the field about available evaluations. The committee reviewed all extant evaluations identified, although very few were available. (See Attachment B-1 for a summary of available evaluations.) • Reviewed program monitoring data, selected grant applications, his- torical financial data, and written comments from experts and officials.  The committee requested and received from ED all available program data, including historical funding information and program monitoring data collected via the Evaluation of Exchange, Language, International and Area Studies (EELIAS) database. We had hoped that the EELIAS database would be a rich source of information about program performance. Instead, the problems we encountered using EELIAS and the comments from grantees on our site visits led us to make the database itself a focus of our review (see Box B-1 for a summary of database issues). In addition to the EELIAS data, we also reviewed the FY 2006 performance plan for international edu- cation and foreign language studies programs (domestic, overseas, and the Institute for International Public Policy, IIPP) and the Program Assessment   Although committee staff worked with ED staff to clarify obvious errors or questions about the data provided, the data were accepted as accurate and not independently verified by the committee. 

APPENDIX B 311 BOX B-1 Technical Issues with EELIAS Database The committee encountered multiple technical issues with the EELIAS database that hampered our ability to use the data for analyses: • Lack of internal controls.  Although the text of data entry screens indicates that certain fields are required, the system does not always have internal con- trols that enforce that requirement before data are submitted. • Use of open-ended questions.  For questions that might require a large num- ber of response options, responses are left as open-ended questions. One example is a recently added question asking whether a standardized proficiency test was given to Foreign Language and Area Studies (FLAS) recipients and, if so, the type of test. This resulted in responses such as “ACTFL,” “ACTF Scale,” “ACTFL OPI,” “Oral Proficiency Interview,” “ACTFL/ILR,” and “ACCTF? I don’t remember exactly,” all of which were probably indicating the same test. More useful data might have been obtained by a series of predefined options for the most common measures, even if it necessitated an additional, “other” category. If the “other” option were supplemented by a question related to the language, it would have provided even more useful information. • Use of “other” category for key questions.  In several cases, EELIAS supple- ments a drop-down menu for key data with an “other” option; for example, languages offered and type of language instructor. In the case of languages, responses submitted in the other category included, among others, “Advanced Arabic,” “Intensive Elementary Portuguese,” and “Elementary Persian I,” even though Arabic, Portuguese, and Persian were included in the drop-down menu. • Limited data validation/review.  The committee was told that reports are re- viewed by individual project officers and that the information submitted is used to make continuation awards. It was clear, however, that project officers have limited time to review the consistency or reliability of data submitted from indi- vidual grantees. ED has begun efforts with its contractors to review the consis- tency of data across grantees and to address identified data issues. • Inconsistency across programs.  The data collected for individual programs vary. Although this might be expected to some extent given the variability in program missions, the differences do not always appear to be purposeful. For example, it is not clear why only the Technological Innovation and Cooperation for Foreign Information Access (TICFIA) Program is required to report on the use of technology despite its recognized importance. Similarly, not all programs are required to submit abstracts. • Contractor coding of some information.  The service area descriptor, a po- tentially useful analytic tool added by the department after it assumed manage- ment of the system, has been coded by the contractor based on the project abstract. Given the limited information available in the abstract, this approach has significant limitations. Project staff will be required to enter this information in the redesigned system. 

312 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES Rating Tool (PART) performance measurements of the Office of Manage- ment and Budget (OMB) for international education domestic programs. The review of extant evaluations and monitoring data showed that there were insufficient current data available to provide meaningful in- sights on five of the eight original questions. Given the time constraints on the review, the committee decided to use commissioned targeted analyses, meetings with stakeholders and experts, public testimony, and site visits to selected universities to address the research questions. • Commissioned papers and targeted analyses.  The committee re- viewed or commissioned targeted analyses on increasing representation of minorities, technology and instructional materials, oral proficiency as- sessment, and implementation issues. We conducted an analysis of Mod- ern Language Association enrollment data to explore the possible role of National Resource Centers (NRC) in the teaching of less commonly taught languages, and we examined various definitions of “internationalization” of higher education. The committee also obtained preliminary data from ED generated by a contracted study by InfoUse entitled “Study of Graduate Fellowship Programs Participants,” which includes a survey of all FLAS and Doctoral Dissertation Research Abroad (DDRA) Fellowships from 1997 to 1999. • Reviewed written comments and conducted public meetings to get input from experts, officials, and others and met with grantees in Wash- ington and ED officials with responsibilities for these programs.  Written comments were submitted in response to a set of questions developed by the committee (see Attachment B-2), and public meetings included presen-   PART reviews program evaluation and performance measurement information to assign program ratings. The international education domestic programs were reviewed by OMB in 2004 and assigned a rating of “results not demonstrated.”   Joyner, C.C. (2006). Increasing representation of minorities in international service through international education programs. Paper commissioned by the National Research Council Committee to Review the Title VI and Fulbright Hays International Education Programs, Washington, DC.   Joyner, C., and Suarez, T. (2006). Technology and instructional materials in Title VI and Fulbright-Hays international education programs. Paper commissioned by the National R ­ esearch Council Committee to Review the Title VI and Fulbright-Hays International Educa- tion Programs, Washington, DC.   Malone, M.E. (2006). The oral proficiency interview approach to foreign language assess­ ment. Paper commissioned by the National Research Council Committee to Review the Title VI and Fulbright-Hays International Education Programs, Washington, DC.   Ruther, N.L. (2006). Implementation issues and options for the HEA Title VI and ­Fulbright Hays programs. Paper written for the National Research Council Committee to Review the Title VI and Fulbright-Hays International Education Programs, Washington, DC. 

APPENDIX B 313 tations by grantees, federal agency staff, and outside experts (see Attach- ment B-3). We also met with NRC and Language Resource Centers (LRC) grantees when they were in Washington, DC, for an ED-sponsored grantee meeting and with ED officials with responsibilities for these programs. • Conducted site visits to eight universities.  To become acquainted with Title VI center programs and gather illustrative examples, committee members and staff conducted site visits to eight universities (Georgetown University, George Washington University, Indiana University, New York University, Ohio State University, San Diego State University, the University of California, Los Angeles, and the University of Wisconsin-Madison) and reviewed selected grant applications from these universities. The universities were chosen to represent a variety of public and private institutions, Title VI funding levels, and geographic locations. Although the site visits do not represent a random sample, they were selected to provide a general picture of the diversity of NRC, Centers for International Business Education and Research (CIBER), and LRC grantees. At each location, we met with uni- versity administrators and staff, faculty, students, and librarians associated with the Title VI programs on their campuses. An interview guide based on the eight key areas specified by Congress guided the interviews at each location (see Attachment B-4). Several factors limited our review. First, in the case of the large center programs (NRC, LRC, CIBER), Title VI typically represents a funding stream rather than a discrete program; that is, Title VI funds combine with other funding sources in order to achieve the desired outcomes, so that it is difficult to attribute specific outcomes to Title VI (U.S. General Accounting Office, June, 1998). Second, we did not have time to conduct a survey of all grantees, so some of our observations and findings are not generalizable; the illustrations based on the site visits, for example, may not be applicable to all grantees. In addition, extant data provided significant information for only three of the eight congressional questions, and the major data system (EELIAS), which theoretically could have provided answers to many ques- tions, has significant problems. CONCEPTUAL MODEL Understanding how a program is expected to lead to particular results can be extremely useful in designing and implementing it and in planning its evaluation. A basic logic model provides a systematic and visual way to present and share an understanding of the relationships among the resources/inputs, activities, outputs, outcomes, and impact expected of 

314 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES TABLE B-1  Categories of Performance Measures Category Definition Examples Impacts The degree to which broad Increased competitiveness, enhanced level of social objectives are achieved education, income of graduates, improved quality of educational institutions Outcomes Accomplishment of program Academic performance improvement, students objectives attributable to accepted at next level of education, graduates program outputs certified as teachers, employer satisfaction Outputs The direct result of program Number of students enrolled, targeted students activities completing training, students applying to next level of education Activities The work performed by the Amount of training given, counseling provided, grantee that directly produces conferences held, reports published the core products and services Inputs Resources consumed by the Generally limited to funds and grant years organization SOURCE: U.S. Department of Education. (1998). Demonstrating results: An introduction to the Government Performance and Results Act. Washington, DC: Author, Office of Postsec- ondary Education. the program. Logic models can range from an activities approach model, which describes activities in great detail and is especially useful as a man- agement tool, to a theory approach model, which emphasizes the theory of change that has influenced the design and plan for a program and is especially useful during program planning and design. A third kind of logic model is perhaps more relevant to the committee’s work: an outcome approach model, which shows the causal linkages thought to exist among program components and is especially useful in designing effective evalua- tion and reporting strategies. In considering the programs and the results expected from them, it will be useful to keep in mind the distinctions among several categories of measures established under the Government Performance and Results Act (GPRA) and used by ED in monitoring and reporting on its activities. Table B-1 shows one way of describing those categories.   W.K. Kellogg Foundation (2003). Logic model development guide. Battle Creek, MI: Author.   Others might define categories in slightly different ways. In addition, assigning a measure to one or another category (for example, outcome or impact) is often a judgment call on which opinions might differ from one situation to another. Distinctions are sometimes made, as well, between short-term outcomes and longer-term outcomes. 

APPENDIX B 315 In requesting this review, Congress asked the committee to address the performance of these programs in eight specific areas, which implies that these areas might be seen as program goals about which informa- tion is needed. Some of those eight areas reflect questions about activities, whereas others address outputs or outcomes. Table B-2 shows these eight areas with comments about the category of performance measure specified or implied. Figures B-1 through B-5 represent a modified “outcome approach” logic model. That is, individual programs are grouped in ways that show how their activities in combination are expected to lead to certain short- term and longer term outcomes. While grouping programs provides a better appreciation for the ways in which they interact, it necessitates some loss of detail in describing the activities and their associated output measures. Figure B-1, which presents an overview of all 14 programs, has the least amount of detail. The schematic in Figure B-1 shows all 14 programs along with their links to the goals identified in the congressional request, other short-term and longer term outcomes that can be inferred from that request, and the legislation and regulations addressing allowable activities in the programs. For the purpose of this report, the goal of “supporting research, educa- tion, and training” is represented at the level of three inferred subgoals: (1) increased research capacity, (2) enhanced body of knowledge about foreign languages and area studies, and (3) increased institutional capac- ity of education and training in K-12 and institutions of higher education (IHE). Reading from left to right, the relationship between short-term and longer term outcomes can be seen. For example, the schematic posits that improving instructional materials is important because of its contribution to improving the institutional capacity of education and training, which in turn contributes to a decrease in the shortage of experts as well as an increase in the global understanding of academics and U.S. educators. The figure also shows that nearly all of the programs are related to multiple outcomes. In Figures B-2 through B-5, programs are grouped by similarity in the way they function or in their purpose. By reducing the number of programs in a single schematic, it is possible to include more detail about the activities that are expected to lead to the outcomes. Figure B-2 shows the six programs that provide funds to individuals: the four Fulbright-Hays programs (DDRA, Faculty Research Abroad or FRA, Group Projects Abroad or GPA, and Seminars Abroad or SA) as well as FLAS and IIPP. Some of these programs could be expected to contribute to a reduction in the shortage of experts, including those who are minori- ties, while others should lead to an enhanced body of knowledge about foreign languages and area studies or improved global understanding. As 

316 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES TABLE B-2  Program Areas to Be Addressed in Responding to Congress Area Comment 1.  upporting research, S As stated, output measure would assess the extent to which education and training research, education, and training occurred. One could in foreign language and plausibility link the question to short-term and longer term international studies, outcomes of (a) improved institutional research capacity, (b) including opportunities for improved institutional capacity to provide quality education such research, education, and training in K-12 and institutions of higher education, and training overseas and (c) improved body of knowledge about foreign language and area studies. 2.  educing shortages of R Outcome measure would be decreased shortage of foreign foreign language and area language and area studies experts in public service and experts academia. 3.  nfusing a foreign language I Short-term outcome measure would be increased infusion and area studies dimension of this dimension into elementary, secondary, and higher throughout the education education, including professional education. One could system and across relevant make an expected causal link between this outcome and a disciplines, including longer term outcome of improved institutional capacity in professional education K-12 and IHE. 4.  roducing relevant P Short-term outcome measure would be improved instructional materials that instructional materials. That is, one would go beyond the meet accepted scholarly output measure of quantity of materials produced to assess standards that they are “improved” over what previously existed by virtue of meeting accepted scholarly standards. This measure, in turn, would lead to the longer term outcome of improved institutional capacity for education and training. 5.  dvancing uses of new A Output measure would describe the ways in which projects technology in foreign use technology. Short-term outcome measure would be language and international improved technology to enhance foreign language and studies international studies and its use. This, in turn, would contribute to the longer term outcome of an improved institutional capacity for research, education, and training. 6.  ddressing business A A short-term outcome measure would be enhanced needs for international materials and activities to meet business needs. knowledge and foreign language skills 7.  ncreasing the numbers I Outcome measure would be increased representation of of underrepresented minorities in international service. Lacking that information, minorities in international output measure would describe program activities and service participants. 8.  onducting public C As stated, output measure would assess the extent to which outreach/dissemination these outreach and dissemination activities were conducted. to K-12 and higher One could plausibly link the question to the short-term education, media, outcome measure of improved outreach and dissemination government, business, and of information about other cultures. the public 

CIBE ↑ Materials & activities to BIE meet business ↑ Global needs understanding: media & general public AORC ↑ Research capacity ↑ Global NRC understanding: government LRC ↑ Global understanding: ↑ Outreach & business IRS dissemination TICFIA ↑ Global ↑ Technology understanding: academia UISFL ↑ Infusion of foreign ↑ Institutional languages & area capacity/quality studies in elem., sec., of education DDRA & higher ed, including and training: ↑ Global professional education K-12 and IHE understanding: U.S. educators FRA ↑ Instructional materials ↓ Shortage of foreign languages GPA & area studies experts in public ↑ Opportunities service & for individuals academia SA and groups to study & do research ↑ Educational ↑ Minorities in activities international FLAS IIPP targeted to service minorities FIGURE B-1  Schematic of all programs, showing short-term and longer term outcomes. NOTE: Boxes with solid shading show programs. Bold boxes show goals and inferred subgoals identified in the congressional B-1 317 request for the study. Landscape view Beneath image is a half inch left for caption, which can be seven inches long 

318 Activities/Outputs Short-Term Outcomes Longer Term Outcomes Graduate DDRA students do Quality dissertation research is research produced on abroad for 6 foreign to 12 months languages & Enhanced body area studies of knowledge topics about foreign Faculty do languages & FRA research area studies abroad Participants’ language competency Teachers, students, &/or & area Decreased faculty go to short-term studies shortage of GPA study or intensive knowledge foreign overseas programs improve Program alumni languages & are employed area studies as experts in experts in public service public service Graduate students use &/or academia & academia academic year & summer FLAS programs to study foreign language &/or area studies Increased representation Increased infusion of of minorities in foreign languages & international area studies in service Minorities are targeted as elementary & undergraduates and secondary education IIPP followed through graduate school with special activities ↑ Institutional capacity/quality of education and training: Alumni gain an improved K-12 and IHE K-12 educators do short- understanding & Improved global knowledge of peoples & Improved outreach & understanding: SA term study & travel seminars cultures of other dissemination of U.S. educators abroad countries, which they information about share with colleagues other cultures upon return FIGURE B-2  Programs that provide funds to individuals. NOTE: Boxes with solid shading show programs. Bold boxes show goals and inferred subgoals identified in the congressional request for the study. B-2 Landscape view We have unified some of the boxes without having to recreate the entire figure 

APPENDIX B 319 interim outcomes, one would expect to see improvement in individuals’ language competency, production of quality research, and an improved understanding of peoples and cultures of other countries on the part of nonspecialists. In Figure B-3, the two programs specifically intended to support inter- national business education (CIBER and Business and International Educa- tion or BIE) are shown. Meeting business needs is intended to lead to an improved global understanding in the business community, which in turn is expected to enhance U.S. leadership in the global economy. The three programs that function through institutional centers (Ameri- can Overseas Research Centers, AORC, NRCs, and LRCs) are shown in Figure B-4. In a variety of ways, these program activities are expected to improve institutional capacities for research and for education and train- ing. In turn, these institutional changes should lead to an improved body of knowledge about foreign languages and area studies and to both an increase in experts and to enhanced general global understanding. Their activities are expected to focus especially on the less commonly taught languages (LCTLs). The remaining three programs (IRS, Undergraduate International Studies and Foreign Language or UISFL, and Technological Innovation and Cooperation for Foreign Information Access or TICFIA) address the specialist and generalist outcomes through the intermediary outcome of improving the institutional capacity for education and training on foreign languages and area studies in K-12 and IHE. As Figure B-5 shows, their activities increase the infusion of these areas into higher education, improve instructional materials, improve outreach and dissemination, and improve technology, which in turn improves institutional capacities for research and education and training. Not shown in the individual schematics is the overall impact expected of the set of programs and some of the theoretical assumptions that underlie their creation. For example, in its FY 2007 Performance Plan, ED describes the program goal being addressed by all these programs as “to meet the nation’s security and economic needs through the development of a national capacity in foreign languages, and area and international studies.” Given this goal, an impact assessment would attempt to determine whether the nation is more secure and economically sound as a result of the programs. An assessment of this sort is beyond the scope of the committee’s charge. Indeed, although we endeavored to assess outcomes, the available data of- ten restricted our analysis to assessment of inputs, activities, and outputs. In the report, the committee makes recommendations aimed at moving future program monitoring and evaluation efforts toward assessment of outcomes. 

Program Activities/outputs Centers serve as regional and Short-Term Outcomes Longer Term Outcomes Outcomes national resources for business 320 people, students, and teachers at all levels. Activities include (1) applied research on Enhanced body of trade and U.S. knowledge about competitiveness international (2) instruction in critical trade and foreign languages & competitiveness CIBE international fields issues (3) outreach to local businesses (4) faculty and academic program development on improved business Increased infusion of techniques, strategies, & foreign languages & Improved Improved Enhanced methodologies that area studies into institutional global U.S. emphasize an higher education, capacity/quality of understanding leadership in international context including education and in the business the global professional training in IHE community economy education Enhanced IHEs partner with a business materials & enterprise, trade organization, or activities to meet association in applying for grants to business needs (1) improve the academic teaching of the business curriculum (2) conduct outreach activities that BIE expand the capacity of the business community to engage Improved in international economic outreach & activities dissemination of (3) promote education and training information about that will help businesses other cultures prosper FIGURE B-3  Programs that support international business education. NOTE: Boxes with solid shading show programs. Bold boxes show goals and inferred subgoals identified in the congressional request for the study. B-3 Landscape view 

Program Activities/Outputs Short-Term Outcomes Longer Term Outcomes Enhanced AORC IHE consortia operate overseas research centers that promote Improved global postgraduate research, exchanges, & area studies institutional understanding: research media & capacity general public Centers maintain linkages with overseas organizations Improved body Enhanced of knowledge global Centers conduct research/training in international studies and about foreign understanding: issues in world affairs and collect materials to support research languages & government area studies Improved NRC Centers serve as a resource on language aspects of fields of outreach & Enhanced study through outreach & consultative services dissemination global about foreign understanding: languages & area business studies Centers teach modern foreign languages, especially LCTLs, and provide instruction about areas, regions, or countries in Enhanced which the languages are commonly used global Increased infusion understanding: of foreign Improved academia languages & area institutional Centers conduct & disseminate research on language studies in elem., capacity/quality teaching methods sec., & higher of education Enhanced education, and training: global including K-12 and IHE Centers develop & disseminate materials for language understanding: professional U.S. educators learning, including in K-12 education Centers develop/disseminate tools for language assessment Improved Instructional LRC materials Centers train foreign language teachers Decreased shortage of foreign Centers focus on LCTLs, including dissemination of languages & area instructional materials Improved studies experts technology in public service & Centers operate intensive summer language institutes for academia students and teachers FIGURE B-4  Programs that support centers with a focus on foreign languages and area and international studies. NOTE: Boxes with solid shading show programs. Bold boxes show goals and inferred subgoals identified in the congressional 321 request for the study. B-4 Landscape 

Program Activities/Outputs Short-Term Outcomes Longer Term Outcomes 322 Increased infusion Activities support surveys, studies, and instructional materials of foreign IRS development to improve & strengthen instruction in modern languages & area foreign languages, area studies, & other international fields studies in elem., sec., & higher education including Programs to strengthen & improve undergraduate instruction professional in international studies & foreign languages may include but education are not limited to • Development of a new language program Enhanced • Development of an interdisciplinary global global studies/international studies program understanding: • Development of a program that focuses on specific Improved academia issues or topics, such as international health Instructional • Development of an area studies program & its materials UISFL languages • Creation of innovative curricula that combine teaching Enhanced international studies with professional and pre- global professional studies, such as engineering understanding: • Research for & development of specialized teaching U.S. educators materials Improved Improved • Establishment of internship opportunities for outreach & institutional faculty/students in domestic & overseas settings dissemination capacity/quality • Development of study-abroad programs about foreign of education Decreased languages & area and training: shortage of studies K-12 and IHE foreign languages & area studies experts in public service Grants fund development of innovative & academia techniques/programs using new electronic technologies to collect information from foreign sources. Activities related to information on world Improved Improved regions & countries other than the U.S. include technology institutional Enhanced body TICFIA • Accessing research of knowledge • Collecting capacity about foreign • Organizing languages & • Preserving area studies • Widely disseminating FIGURE B-5  Programs intended to improve instruction in foreign languages and area studies. NOTE: Boxes with solid shading show programs. Bold boxes show goals and inferred subgoals identified in the congressional request for the study. B-5 Landscape 

APPENDIX B 323 SUMMARY The model outlined above illustrates the interconnectedness of the pro- grams and the varied but complementary ways in which they contribute to the program goals and to the key areas specified in the committee’s charge. We aimed to understand the synergies between the programs and the ways in which they complement or duplicate one another. Some programs (CIBER, BIE, TICFIA, IIPP) are by design focused on one or more of the key specific areas. Others (NRC, LRC) are broader in focus and will ad- dress a larger set of outcomes. The available data do not support an analy- sis of the individual program components in the Title VI/FH programs. Instead, the committee has approached the review as an assessment of the degree to which the 14 programs—as a synergistic set of complementary activities—respond to the eight key areas. 

324 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES ATTACHMENT B-1 Summary of Title VI and Fulbright-Hays Evaluation Studies Name of Study Description Main Findings Increasing The study was Criticisms: Representation commissioned by IIPP reaching only a small number of of Minorities in the committee. The students and data is lacking on what International evaluator reviewed students do after they participate in the Service Through grant applications and program. International program descriptions More potential is in grants to Education to find Title VI-funded institutions that serve a larger number of Programs (Joyner, projects that involved students. IRS-funded studies will identify 2006a) efforts to reach out to barriers to participation in international minority students and program for minority students, and how encourage participation to overcome those barriers. in programs which emphasize international study and employment in international fields. The evaluator also examined and built upon data on post-program employment for participants in the IIPP. Shortcomings: Lack of data, many programs and research studies not yet completed, evaluations of programs not yet completed. 

APPENDIX B 325 Name of Study Description Main Findings Technology and The study was Two-thirds of the reviewed projects Instructional commissioned by the (82 projects) used computer-based Materials in Title committee. The purpose technology in the generation of VI and Fulbright- was to gauge the extent instructional materials and assessments. Hays International to which Title VI/FH are Technology is frequently used as Education using new technology an important delivery tool to make Programs (Joyner to advance foreign information more broadly available and Suarez, 2006) language and international to users, such as distance learning. studies, and the extent Technology is also used to enhance to which the programs the content being produced. This use are producing quality was especially obvious in the TICFIA instructional materials. and IRS projects, where the use of The author reviewed technology included activities such as 124 Title VI-funded gathering information from diverse projects, including all of locations and modifying it to make it those under TICFIA and accessible to English-speaking users and IRS, and those in other using the interactive and multimedia programs that had a focus capabilities of the Internet for instruction on use of new technology or assessment. and/or development of new instructional materials The study describes efforts to provide and assessments. materials for instruction and assessment for the study of various languages and Shortcomings: cultures. Materials were produced No available criteria for instruction in a wide range of as to how much use of languages, including many of the technology or production LCTLs. The materials included some of instructional materials resources for instructors, but most were would represent a “high” products that would be used directly or “admirable” extent in instruction or by students. Projects and how much would produced assessment tools less often represent a “low” or than other instructional materials. All “unacceptable” extent. the assessments reported were in projects Lack of criteria made it funded by IRS or LRC. impossible to judge the quality of the instructional Criticisms: materials produced. ED’s current EELIAS database does not provide convenient access to Missing data in ED’s reliable program information for use EELIAS database, and by Congress, agency managers, and the limited information public. on grantee’s plans and progress. continued 

326 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES Name of Study Description Main Findings Institute for This study is an internal The program has positive effects in International review of the IIIPP terms of its impact on participants, Public Policy program. It is based who report that the program has Impact Assessment on surveys of program “increased intellectual capabilities” (United Negro participants and two focus and “strengthened interpersonal College Fund group sessions. communication skills” as well as other Special Programs positive personal outcomes. Other Shortcomings: Corporation, relevant findings: Low response rate, August 2006) particularly for earlier • 40% of former participants found cohorts. The survey also employment congruent with had inadequate response program goals. choices for program • Over half of participants speak alumni career paths; another language for work-related therefore the percentage reasons. of alumni who go onto • Much better data collection and careers in international tracking of participants is required. service cannot be • Graduate school and career service established with any advising programs should be precision. improved. 

APPENDIX B 327 Name of Study Description Main Findings Language and This is an update of the Title VI-funded NRCs and LRCs Critical Area Brecht and Rivers study. produce a disproportionate amount of Studies After It examines outputs of expertise in language and area studies, September 11: An career paths, dissertations, compared to non-Title VI funded Evaluation of the articles and academic institutions, even the most prestigious Contributions of awards in Middle Eastern ones. They are especially critical in Title VI/FH to the and Slavic studies, research, teaching, and maintaining the National Interest comparing Title VI and capacity to teach LCTLs. (Brecht et al., non-Title VI institutions. Criticisms: 2007) It also attempts to isolate Funding for the programs should be the programs’ “criticality,” increased, and they should be well- i.e., the extent to which integrated with the coming National the programs would Security Language Initiative. For function as they do this reason, the authors argue for currently without federal the creation of a National Language support. Advisor. Shortcomings: Lots of data on student placement in the EELIAS system is missing. The study only examines Middle East and Slavic studies, not all subject areas. In addition, measuring “criticality” is difficult. The programs have an impact, but value to society is a “societal question, best answered by the political process.” E-LCTL Initiative This was a collaborative In centers funded by Title VI, more than (2005) effort among several 30,000 students were enrolled in 128 Title VI-funded centers LCTLs in 55 universities in 27 states in to assemble data on the 2001-2002. teaching of LCTLs; the These centers enroll more than 60% of study itself was partially the students in the 10 languages deemed funded by ED. The project “critical” by the Department of Defense. collected and analyzed data from applications During 2001-04, 226 LCTLs were for NRC and FLAS available through Title VI centers while awards and compared federal agencies offered 75 LCTLs. with published data about For the past decade, an estimated more language instruction in the than 80% of all instruction in LCTLs Foreign Service Institute, was in Title VI-supported centers. Defense Language Institute (DLI), and all U.S. universities. continued 

328 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES Name of Study Description Main Findings Securing Our This study performed CIBER leveraged additional resources Nation’s Future by staff at one CIBER from universities, foundations and Through sought to summarize businesses. International the program’s CIBER programs have had a “wide Business accomplishments over spillover effect” on other universities & Education: 15 15 years. It collected colleges in “deepening the international Years of CIBER, data from CIBERs expertise of US higher education.” 1989-2004 nationwide on the number (University of of doctorates awarded, CIBERs are “building capacity and then Connecticut School courses taught, courses collaborating to develop networks across of Business CIBER, created or upgraded as the the nation which are greatly enhancing 2005) result of CIBER funding, the overall impact of programs.” number of students, and the number of new degree programs initiated or revised. Title VI BIE This study was based on BIE has been “highly successful” Funding: A Survey a questionnaire sent to in meeting the intended program of Success (Gerber, institutions receiving BIE goals. Funded projects have “not 2002) grants. It surveyed the only demonstrated high impact on institutions on funded targeted populations in education and activities: faculty training business, but have proved to be highly and research; course sustainable.” development; curriculum The receipt of BIE grants opened the development; business door to funding from other sources. outreach; and academic outreach. BIE programs helped to establish relationships with the business Shortcomings: community and university One drawback of the administrators. study was that the respondents (there was a 58% response rate) might have given a more positive outlook on the program than non-respondents. Producing The study compared 31 The study found CIBER-funded International MBA programs at CIBER programs were more likely to have Expertise in MBA institutions with 35 MBA international business (IB) courses in Programs (Folks, programs at non-CIBER core program of business schools, and 2003) institutions. offer more international business courses in general; integrate language instruction into the MBA curriculum and offer a greater variety of languages, especially LCTLs. 

APPENDIX B 329 Name of Study Description Main Findings Language and This is a wide-ranging Most of Brecht and Rivers conclusions National Security study that evaluated the center around the value of Title VI/FH in the 21st Title VI/FH against the in supporting the study and teaching of Century: The overall strategic goals of LCTLs. The authors clearly indicate that Role of Title the programs and a set of much of the teaching of LCTLs would VI/Fulbright-Hays language-specific objectives not exist were it not for Title VI/FH in Supporting the authors believe follow funding. Title VI/FH is less successful National Language from those goals. The in other areas, and many aspects of its Capacity (Brecht analysis was conducted, to effectiveness are difficult to measure due and Rivers, 2000) the extent possible, within to lack of data. Their conclusions: the GPRA framework. It Title IV/FH has had a significant did not cover area studies. impact on basic research in LTCLs, the The authors attempted to production of scholars and teachers address the “criticality” in LCTLs, and in applied research of the set of programs, in the form of learning and teaching through the question materials. It has played a critical role “what evidence exists that in the development and maintenance of a language function critical language capacity in its strengthening of to the nation would not be flagship programs in LCTLs. It has also accomplished were it not lead to greater levels of proficiency in for the programs funded these languages. Much of the teaching of under Title VI/FH?” LCTLs would not exist were it not for Shortcomings: Title VI/FH funding. Focuses on language NRCs and LRCs contributed to about expertise, not area studies. half of all published research on LCTLs. Evaluates program as a Its role in supporting the more whole, not component commonly taught modern languages is parts. less critical. The contribution of Title IV/FH to the national pool of practitioners is unclear, because of a lack of data, particularly on the career paths of students it supports. Criticisms: Should be refocused on languages rather than area studies. Goals of the program are very broad, and a debate over the programs is whether they should fund programs as a matter of general educational importance or to train specialists. Legislation suggests both but current funding levels make that impossible. continued 

330 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES Name of Study Description Main Findings Federal Funding The study was intended to 80% reported a high overall institutional for International inform ED about grantee impact of the project. Studies: Does It performance. It took the Over 90% reported that the institution Help? Does It form of a questionnaire is currently supporting a program Matter? Long- sent to a sample of 107 that received funding 5 or more years Term Impacts of institutions receiving previously. Federal Funding UISFL funding in alternate on International years between 1982 and The “program had a positive effect Studies and 1990 regarding campus on many elements deemed critical to Foreign Language environment, project the development and strengthening Programs focus, outcomes, and of international education,” such as (Schneider and factors contributing to the requiring an “international” course Burn, 1999) project. There was a 75% as a general education graduation response rate. There were requirement and having a formally also site visits to half of designated advisor for students doing the institutions surveyed. international or area studies. Shortcomings: Grantee institutions made considerable Some supplemental data “matches” to UISFL grants. There could not be obtained were many instances of leveraging for from ED. additional funds and use of seed money for innovation that would not have taken place otherwise. Grants bestowed a great deal of prestige, which facilitated interdisciplinary activity. Grantees reported adding new courses and languages, and nearly all were still being offered with solid enrollments after the grant ended. Programs offered an average of 2.8 new and 2.9 revised courses. Enrollment in languages increased. Criticisms: There were no criticisms, but recommendations are meant to address some shortcomings, including: ED should keep better records of program outcomes and other basic information. ED should encourage applications from a wider array of institutions. Programs deserve more funding. 

APPENDIX B 331 Name of Study Description Main Findings Three Decades The study tried to Analysis of changes across time showed of Excellence determine the impact of an increase in fellowships awarded to 1965-1994: receiving a DDRA Grant women to 51% in 1990-1994. The Fulbright- on a person’s career. It By 1995, fellows were teaching in nearly Hays Doctoral collected information 400 academic institutions across the Dissertation on recipients, such as country. Research Abroad the regions where their Fellowship research was conducted, Almost all those responding (more Program and its languages used, academic than 99%) said that the program was Impact on the disciplines, the time it took a critical element in their success as American Academy to complete a Ph.D., future professional area-studies and foreign (Council of employment, etc. The language specialists. American Overseas survey was sent to 783 Respondents also described intangible Research Centers, of 3,200 total recipients, and unquantifiable benefits such as a 1998) and the response rate was deeper understanding of other cultures, 53%. the experience of living in a foreign Shortcomings: country, and the greater awareness of Length of questionnaire one’s global identity. may have affected response rate among highest- achieving participants, thus understating effects of the program. Continuing and The purpose of the Respondents had a high degree of Emerging National study was to gauge the satisfaction with their projects’ impacts. Needs for the impact of UISFL-funded Most felt they had been successful in Internationalization programs. Administrators maintaining gains made during the of Undergraduate of all of the 195 UISFL course of the projects. Education projects funded 1990-1995 (McCarthy, 1998) were sent surveys. The Vast majority of the projects focused respondents were asked to on improving existing structure rather rate UISFL project’s impact than making major changes. Specifically, on their home institution, developing skills and knowledge of unanticipated multiplier existing faculty, acquiring library and effects, and questions. teaching materials, revising existing courses, and revising or expanding Shortcomings: foreign language curriculum. A methodological issue was the 44% response Respondents cited several institutional rate and self-reporting. A changes that went beyond specific majority of administrators project activities. rated their own programs Criticisms: a “5” or “6” on a scale of None, but some recommendations for 1 to 6. increased funding and sharing of best practices. 

332 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES ATTACHMENT B-2 Written Comments Submitted to the Committee The committee invited individuals and organizations knowledgeable about the Title VI and Fulbright-Hays programs to submit comments based on the following guiding questions: • What do you view as the purpose of the Department of Educa- tion’s Title VI and Fulbright-Hays program(s)? What are the program(s)’ strengths, weaknesses, and criticisms? • How would you define “success” for the program(s)? How would you determine if the program(s) are “successful” or “effective”? Would you measure effectiveness differently for the various Department of Education Title VI and Fulbright-Hays programs? • What data/information do you use to measure the effectiveness of this program(s)? • What data/information do you think is necessary or should be used to measure effectiveness? • How would you use the results of an evaluation? • How do you view the role of the Department of Education’s Title VI and Fulbright-Hays programs as similar or different from the role of other related federal programs? The following individuals or organizations submitted written comments: Melissa H. Birch, Director, Center for International Business Education and Research, University of Kansas, Title VI Centers for International Business Education and Research Program William Brustein, Director, University Center for International Studies, University of Pittsburgh, General Statement/Title VI and Fulbright-Hays Mark Chichester, Director, Institute for International Public Policy, The College Fund/UNCF, Title VI Institute for International Public Policy *Donna Christian, President, Center for Applied Linguistics (CAL), CAL’s Perspective on the International Research and Studies Program Dan E. Davidson, President, American Council of Teachers of Russian, Fulbright Group Projects and Seminars Abroad Programs * These individuals did not present their comments to the committee in person. 

APPENDIX B 333 *J. David Edwards, Executive Director, JNCL-NCLIS, The Joint National Committee for Languages and the National Council for Languages and International Studies Perspectives on Title VI Uliana Gabara, Dean of International Education, University of Richmond, Title VI Undergraduate International Studies and Foreign Language Programs *Interagency Language Roundtable (ILR), written comments submitted on May 9, 2006, by: Frederick H. Jackson, Coordinator, ILR Scott G. McGinnis, Steering Committee, ILR Glenn Nordin, Office of the Secretary, U.S. Department of Defense Catharine Keatley, Associate Director, The National Capital Language Resource Center, Title VI Language Resource Center Program *Stanley Kurtz, Fellow, Hudson Institute, six articles on Title VI: The Language Gap by Lee Smith Learning the Language by Kenneth Whitehead Title VI: Turn on the Defogger by Martin Kramer Boycott Exposure by Stanley Kurtz Scholars Revive Boycott of U.S. Grants to Promote Language Training by Anne Marie Borrego Who Will Defend the Defenders by Stanley Kurtz Mary Ellen Lane, Executive Director, Council of American Overseas Research Centers, Title VI American Overseas Research Centers Gilbert Merkx, Vice Provost for International Affairs, Center for International Studies, Duke University, Title VI NRC and FLAS; Fulbright DDRA and FRA Programs Kelly Jett Murphrey, Director, Center for Study of Western Hemispheric Trade, Center for International Business Studies, May Business School, Texas A&M University, Title VI Business and International Education Program Amy W. Newhall, Executive Director, Middle East Studies Association, University of Arizona, The Higher Education Act, Title VI, and the Mutual Educational and Cultural Exchange Act, Section 102(b)(6), International Education and Foreign Language Tony Stewart, Professor of South Asian Religions, North Carolina State University, Title VI Programs 

334 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES Vivien Stewart, Vice President, Education, Asia Society, K-12 International Education David Wiley, Director, African Studies Center, Michigan State University, Title VI Research and Studies and TICFIA Programs 

APPENDIX B 335 ATTACHMENT B-3 Open Committee Sessions First Committee Meeting February 14-15, 2006 OPEN SESSION AGENDA 500 Fifth Street, NW Room 204 Washington, DC Tuesday, February 14 10:45 a.m. Welcome and Introductions Janet Norwood, Committee Chair, Committee to Review the Title VI and Fulbright-Hays International Education Programs 10:55 a.m. Sponsor Perspective Susan Beaudoin, Deputy Assistant Secretary, Higher Education Programs, Department of Education 11:15 a.m. Department of Education Study of Graduate Fellowship Programs Participants Lewis Kraus, Vice President, InfoUse 12:15 p.m. Lunch (Room 213) 12:45 p.m. Program Administration and Monitoring at the Department of Education Ralph Hines, Director, International Education Programs Service (IEPS), Department of Education Karla Ver Bryck Block, Senior Program Specialist, IEPS 1:45 p.m. Public Forum: Part I (8-9 Minutes per Speaker) William Brustein, Director, University Center for International Studies, University of Pittsburgh, General Statement/Title VI and Fulbright-Hays 

336 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES Dan E. Davidson, President, American Councils for International Education and Professor of Russian, Bryn Mawr, Fulbright-Hays Group Projects and Seminars Abroad Programs Gilbert Merkx, Vice Provost for International Affairs, Center for International Studies, Duke University, Title VI NRC and FLAS, and Fulbright DDRA and FRA Programs David Wiley, Director, African Studies Center, Michigan State University, Title VI Research and Studies, and TICFIA Programs Catharine Keatley, Associate Director, The National Capital Language Resource Center, Title VI Language Resource Center Program Uliana Gabara, Dean of International Education, University of Richmond, Title VI Undergraduate International Studies and Foreign Language Programs Mary Ellen Lane, Executive Director, Council of American Overseas Research Centers, Title VI American Overseas Research Centers 3:15 p.m. Break 3:30 p.m. Public Forum: Part II (8-9 Minutes per Speaker) Melissa H. Birch, Director, Center for International Business Education and Research, University of Kansas, Title VI Centers for International Business Education and Research Program Kelly Jett Murphrey, Director, Center for Study of Western Hemispheric Trade, Center for International Business Studies, May Business School, Texas A&M University, Title VI Business and International Education Program Mark Chichester, Director, Institute for International Public Policy, The College Fund/UNCF, Title VI Institute for International Public Policy Vivien Stewart, Vice President, Education, Asia Society Amy W. Newhall, Executive Director, Middle East Studies Association, University of Arizona Tony Stewart, Professor of South Asian Religions, North Carolina State University Ann Imlah Schneider, International Education Consultant 5:00 p.m. Adjourn 

APPENDIX B 337 Second Committee Meeting May 18, 2006 Workshop OPEN SESSION AGENDA The Lecture Room—The National Academy of Sciences 2100 C Street, NW, Washington, DC Thursday, May 18 8:00 a.m. Continental Breakfast 8:30 a.m. Welcome and Introductions Martin Orland, Director, Center for Education Janet Norwood, Chair, Committee to Review the Title VI and Fulbright-Hays International Education Programs 8:45 a.m. Profiles of Two University International Education Programs Moderator: Fernando Reimers, Committee Member and Professor of International Education, Harvard Graduate School of Education University of Michigan: • Mark Tessler, Vice Provost for International Affairs, Director, University of Michigan International Institute • Bradley Farnsworth, Director, Center for International Business Education • Michael Kennedy, Director, Center for Russian and East European Studies, Center for European Studies and European Union Center of Excellence • Linda Lim, Professor of Strategy, Stephen M. Ross School of Business, Director, Center for Southeast Asian Studies • Barbara Metcalf, Director, Center for South Asian Studies Yale University: • Nancy Ruther, Associate Director, Yale Center for International and Area Studies 

338 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES • William Foltz, Heinz Professor of African Studies and Political Science • Sandra Sanneh, Director, Program in African Languages 11:00 a.m. Break 11:15 a.m. Discussion of Profiles Moderator: Sheila Biddle, Committee Member and Independent Consultant 12:30 p.m. Lunch 1:30 p.m. The Role of Universities in Addressing the Demand for Area Studies, International, and Foreign Language Expertise: A Moderated Panel Discussion of Demand Perspectives Moderator: Ambassador Michael Lemmon, Committee Member and Faculty Advisor, National Defense University Panelists: • Christine Brown, Assistant Superintendent, Glastonbury Public Schools • Diane Castiglione, Director of Recruitment, U.S. Department of State • Susan Kelly, Special Assistant to Deputy Undersecretary of Defense Policy Planning and Evaluation, Department of Defense • Charles Kolb, President, Committee for Economic Development • James Nye, Director, South Asia Language and Area Center, University of Chicago 3:00 p.m. Break 3:15 p.m. The Role of Universities in Addressing the Demand for Area Studies, International, and Foreign Language Expertise: A Moderated Panel Discussion of University/Title VI Perspectives Moderator: Chris Cross, Committee Member and Chairman, Cross & Joftus, LLC Panelists: • Ben L. Kedia, Wang Professor of International Business, Director, Wang CIBER, The University of Memphis 

APPENDIX B 339 • Linda Lim, Professor of Strategy, Stephen M. Ross School of Business, Director, Center for Southeast Asian Studies • Patrick O’Meara, Professor and Dean, International Programs, Indiana University • Steven M. Poulos, Director, South Asia Language Resource Center, The University of Chicago • Karla Ver Bryck Block, Senior Program Specialist, IEPS, Department of Education 5:00 p.m. Adjourn Fourth Committee Meeting October 5-6, 2006 OPEN SESSION AGENDA The Beckman Center of the National Academies The Board Room Irvine, CA (and via videoteleconference at The National Academy of Sciences 2100 C Street, NW, Room NAS 250, Washington, DC) Thursday, October 5 8:00 a.m. PST Breakfast (for those in Irvine) (11:00 a.m. EST) 8:30 a.m. PST Welcome (11:30 a.m. EST) Janet Norwood, Chair, Committee to Review the Title VI and Fulbright-Hays International Education Programs 8:45 a.m. PST Panel Discussion of Title VI in the 21st Century (11:45 a.m. EST) Al Fishlow, Columbia University (via videoteleconference from Columbia University) 

340 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES Martin Kramer, The Washington Institute for Near East Policy Robert Blake, University of California Language Consortium 10:15 a.m. PST Break (1:15 p.m. EST) 10:30 a.m. PST Recent Developments in IEPS Data Collection and (1:30 p.m. EST) Reporting Karla VerBryck Block, International Education Programs Service 11:15 a.m. PST (2:15 p.m. EST) The Honorable David S.C. Chu, Undersecretary of Defense for Personnel and Readiness (via videoteleconference from the Department of Defense) 12:00 p.m. PST Break (for lunch in Irvine) (3:00 p.m. EST) 1:00 p.m. PST End of Open Session (4:00 p.m. EST)   NOTE FOR PUBLIC MEETINGS: This meeting is being held to gather information to help the committee conduct its study. This committee will examine the information and material obtained during this, and other public meetings, in an effort to inform its work. Although opinions may be stated and lively discussion may ensue, no conclusions are being drawn at this time; no recommendations will be made. In fact, the committee will deliberate thoroughly before writing its draft report. Moreover, once the draft report is written, it must go through a rigorous review by experts who are anonymous to the committee, and the committee then must respond to this review with appropriate revisions that adequately satisfy the Academy’s Report Review Committee and the chair of the National Research Council before it is consid- ered a National Research Council report. Therefore, observers who draw conclusions about the committee’s work based on today’s discussions will be doing so prematurely.   Furthermore, individual committee members often engage in discussion and questioning for the specific purpose of probing an issue and sharpening an argument. The comments of any given committee member may not necessarily reflect the position he or she may actually hold on the subject under discussion, to say nothing of that person’s future position as it may evolve in the course of the project. Any inferences about an individual’s position regarding findings or recommendations in the final report are therefore also premature. 

APPENDIX B 341 ATTACHMENT B-4 Site Visit Interview Guide 8-03-06 Topics to be addressed in every interview: • The 8 areas of review, including how they evaluate their own prog- ress in these areas. • Unique strengths/challenges of their programs. • Impact of Title VI/FH funding (except student interviews). • How can Title VI/FH be improved (except student interviews)? Key to eight areas of review: Q1.  Supporting research, education, and training in foreign languages and international studies, including opportunities for such research, education, and training overseas; Q2.  Reducing shortages of foreign languages and area experts; Q3.  Infusing a foreign language and area studies dimension throughout the educational system and across relevant disciplines including professional education; Q4.  Producing relevant instructional materials that meet accepted scholarly standards; Q5.  Advancing uses of new technology in foreign language and international studies; Q6.  Addressing business needs for international knowledge and foreign language skills; Q7.  Increasing the numbers of underrepresented minorities in international service; and Q8.  Conducting public outreach/dissemination to K-12 and higher education, media, government, business, and the general public. PROVOST/SENIOR INTERNATIONAL OFFICER Questions related to the eight study areas: 1.  Could you give us an overview of your international programs across the university, including among faculty, undergraduates, graduate students, and the professional schools? (Q1, Q3) 

342 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES 2.  How do you monitor and evaluate the effectiveness of these pro- grams? What sources of information do you use to assess outcomes and/or impacts? Question about the unique aspects of their program: 3.  What do you see as your university’s greatest strength(s) in the field of international education? Its greatest challenge(s)? Questions about the impact of Title VI/FH funding: 4.  Please tell me about the impact of Federal Title VI/FH funding on the development and evolution of your university’s international programs. Does the availability of Title VI/FH funding allow you to support programs or activities that you might not be able to oth- erwise? Can you provide some specific activities? (Probes):  Languages that would not otherwise be offered? Interdisciplinary work that might not otherwise be car- ried out? O utreach or support to K-12 education? 5.  What would happen at your university if Title VI funds were in- creased by 50%? If they were eliminated? Question about how to improve Title VI/FH: 6.  Are there ways in which Title VI/FH programs might be changed so that they were more effective? (Probes):  More flexibility in the use of funds. Different allocation of funds among the program’s several components and objectives. M ore funding for professional schools. NATIONAL RESOURCE CENTERS (NRC) DIRECTORS AND DEPUTIES Questions related to the eight study areas: 1.  Give us an overview of your center and how it fits in the broader university context. What has been the role of Title VI/FH in the development and evolution of your center? (Q1, Q3) 2.  How do you publicize your center and its offerings? How do you recruit students? 

APPENDIX B 343 (Probes):  you think the center is widely known on campus? Do Are your language classes fully subscribed, or could you accept more students into your classes or into the program as a whole? 3.  How and how well are the center’s instructional needs in language being met? (Q1) (Probe):  Through language and literature departments and/or other programs? Through collaboration with the LRC? 4.  What are the language requirements for the certificate and degree programs, for entering students and for completing students? What standard or standards do you use to measure language proficiency? (Q1) 5.  What career paths do students who receive certificates and/or de- grees follow? (Q2) 6.  Tell us about your outreach activities. Who is your primary audi- ence? Can you tell us about some of your most effective outreach activities and why you consider them effective? (Q6, Q8) Question about the unique aspects of the program: 7.  What do you see as your center’s greatest strength(s)? Your greatest challenge(s)? Questions about the impact of Title VI/FH funding: 8a.  (For Centers with continuous funding) Please tell me about the impact of federal Title VI/FH funding on your center’s programs. What activities are supported? (Probes):  it used to support core instruction? Is Is it used to support specialized activities like teaching of LCTLs? Does it allow you to support programs that you might otherwise not be able to offer, such as particular lan- guages, interdisciplinary research, or support to the K-12 system? 

344 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES (For centers that lost funding and then regained it)  Please tell me 8b.  about how your center’s programs were affected by the temporary loss of federal funding. (Probes):  What activities are you conducting or planning now that you were not able to carry out when you lost fed- eral funding? Can you provide examples? What sources of funding did you turn to when federal funds were gone? What types of activities were you able to maintain? What types of activities were you forced to eliminate? 9.  (For centers with continuous funding—do not use for centers that temporarily lost funding) What would happen at your center if Title VI funds were increased by 50%? If they were eliminated? Question about how to improve Title VI/FH: 10.  Are there ways in which Title VI/FH programs might be changed so that they were more effective? (Probes):  More flexibility in the use of funds. Different allocation of funds among the various pro- grams and objectives. More involvement of professional schools. LANGUAGE RESOURCE CENTERS (LRC) DIRECTORS AND DEPUTIES Questions related to the eight study areas: 1.  what specific ways is your center helping to support research, In education, and training in foreign languages across the university (including in the professional schools)? (Q1,Q3) 2.  Tell us about the instructional materials you produce. Who is the primary audience for these materials, and how do you ensure that they are of high quality and relevant to the audience? (Q4) (Probe):  Are the materials reviewed by other scholars who have   San Diego State University: none; George Washington University: none; Georgetown Uni- versity: none; Indiana University: East Asia and West Europe (lost funding 2000-2006); New York University: Latin America (lost funding 2003-2005) and Center for European Studies (lost funding 2000-2002); Ohio State University: East Asia (lost funding 2003-2005); Univer- sity of California, Los Angeles: Latin American Center (lost funding 2000-2003) and Europe/ Russian Center (lost funding 2003-2005). 

APPENDIX B 345 language and area expertise? If so, what is the process by which the materials are reviewed? 3.  How do you evaluate the use and impact of these instructional materials? (Probe):  What types of data do you maintain on the use of your materials? Among various audiences, such as K-12 education, higher education, professional schools, and business? (Q4, Q6, Q8) 4.  Do you coordinate or collaborate with other LRCs or NRCs on campus? With other international-focused programs on campus? Do you collaborate with Title VI-funded programs at other universities? Please give examples. Question about the unique aspects of the program: 5.  What do you see as your center’s greatest strength(s)? Your greatest challenge(s)? Questions about the impact of Title VI/FH funding: 6.  Please tell me about the impact of federal Title VI/FH funding on your LRC. What activities are supported that might not otherwise? (Probes):  it used to support core language instruction? Is Is it used to support specialized activities related to LCTLs? Does it allow you to support programs that you might otherwise not be able to offer, such as particular lan- guages, interdisciplinary research, or support to the K-12 system? 7.  What would happen to your center if Title VI funds were increased by 50%? If they were eliminated? Question about how to improve Title VI/FH: 8.  Are there ways in which Title VI/FH programs might be changed so that they were more effective? (Probes):  More flexibility in the use of funds. Different allocation of funds among the various pro- grams and objectives. 

346 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES CIBER DIRECTORS Questions related to the eight study areas: 1.  Can you provide some examples of the ways in which your pro- grams address business needs for international knowledge and for- eign language skills? (Q6) 2.  Tell us about your outreach activities. Do you target outreach pri- marily to business or to several different audiences? Which of these efforts do you view as most effective and why? (Q3, Q6, Q8) (Probe):  addition to business, do you reach out to other audi- In ences, such as K-12 education, other parts of your univer- sity, the general public, the media? 3.  you coordinate or collaborate with other CIBERs at other uni- Do versities? With other Title VI-funded programs on campus? Other international-focused programs on campus? Please give examples. Questions about unique aspects of the program: 4.  What do you see as your center’s greatest strength(s)? Your greatest challenge? Questions about the impact of Title VI/FH funding: 5.  Please tell me about the impact of federal Title VI/FH funding on your CIBER. What activities are supported that might not otherwise? 6.  What would happen to your center if Title VI funds were increased by 50%? If they were eliminated? Question about how to improve Title VI/FH: 7.  Are there ways in which Title VI/FH programs might be changed to be more effective? (Probes):  More flexibility in the use of funds? Different allocation of funds among the various pro- grams and objectives? FACULTY NOTE: Try to meet with faculty alone, without the presence of the NRC director, LRC director, or other administrator. 

APPENDIX B 347 Questions related to the eight study areas: General orientation Ask faculty to identify their discipline, home department, particular field of expertise, and center they are associated with. 1.  Would you describe the intellectual benefits you gain from partici- pating in the center? (Probe):  you gain insights from other disciplines? Do Does the center provide opportunities for interdisciplin- ary collaboration in research, teaching, and/or service? Do you learn about new scholarship? 2.  How do you publicize your center and its offerings? How do you recruit students? (Probes):  you think the center is widely known on campus? Do Are your language classes fully subscribed, or could you accept more students into your classes or into the program as a whole? 3.  (For language instructors) How many levels of instruction are of- fered in the language(s) you teach? (Q1) (Probes):  What levels of proficiency are required for students in certificate programs, M.A., Ph.D. programs? How is proficiency measured? 4.  Are you or have you been an undergraduate or graduate student advisor? What are the motivations and goals of students pursuing certificate and degree programs in your field? In international/area studies in general? (Q2) (Probe):  What career fields do graduates enter (government, busi- ness, and academia)? Question about unique aspects of the program: 5.  What do you see as your university’s greatest strength(s) in the area of international education and foreign language instruction? Its greatest challenge? Questions about the impact of Title VI/FH funding: 6.  Please tell me about the impact of federal Title VI/FH funding on your teaching, research, and other activities. For what types of ac- 

348 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES tivities have you and/or your students received direct support from Title VI and/or Fulbright-Hays? (Probe):  Can you provide any examples of ways in which Title VI/FH funds helped to leverage additional university re- sources for your research and/or teaching? 7.  What would happen to your research and teaching if Title VI funds were increased by 50%? If they were eliminated? Question about how to improve Title VI/FH: 8.  Are there ways in which Title VI/FH programs could be changed to be more effective? (Probes):  More flexibility in the use of funds? Different allocation of funds among the program’s sev- eral components and objectives? More funding for professional schools? STUDENTS General orientation Ask students to identify themselves based on their field of study, whether they are studying a language, and if so, what language, the NRC with which they are associated, and whether they have benefited from direct support from Title VI (i.e., FLAS or DDRA) Questions related to the eight study areas: 1.  What motivated you to enroll in language/area studies? (Probes):  What did you expect to get from the program (or courses) when you enrolled in it/them? Did you have any ideas about a career in/related to this program? (Q2) 2.  Where and how did you first hear about the area or language pro- gram you are enrolled in? (Probes):  you think other students are aware of these pro- Do grams? Do you feel your center is isolated from or connected to other departments, divisions, and schools? (Q3) 3.  what subjects or disciplines has your learning experience been In the strongest? (Q1) 

APPENDIX B 349 (Probe):  Have you found that language instruction is integrated within courses in such fields as economics, political sci- ence, religion and culture? 4.  (For those who have gone abroad) Was the preparation appropri- ate and adequate for your to take advantage of the cultural and linguistic immersion? What aspects of the learning facilities and/or living conditions enhanced or detracted from your study or research abroad? (Q1) 5.  what ways (if any) do you expect to use your training in area In studies, international studies, or foreign languages after graduation or after completing your graduate degree? (Q2) (Probe):  Are you considering a job in government, business, aca- demia or a nonprofit? 6.  (For graduate students): What aspects of the facilities, that is the library and the technological resources, support or detract from your research? (Q5) 7.  what ways does the faculty integrate technology into instruction In in foreign language and area studies? (Q5) 8.  Have your studies met or failed to meet your expectations? (Probe):  what ways? Which aspects of your studies? In Question about the unique aspects of their program: 9.  What do you see as your university’s greatest strength(s) in the field of international education? Its greatest challenge? INTERNATIONAL STUDIES LIBRARIANS From your perspective: 1.  what ways have Title VI funds enabled you to provide greater In international studies and language resources to the university community? 2.  Are these resources accessed by people outside the university? Who? 3.  How do you measure the people who have accessed these resources?