|
Items in cart [1] |
Questions? Call 888-624-8373 |
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 309
appendix
B
The Committee’s Approach
to Its Review
T
his appendix describes the approach taken by the committee in ful-
filling its charge to review the effectiveness of the foreign language,
area, and international studies programs of the U.S. Department of
Education (ED). It describes the activities we undertook and the conceptual
model we developed to examine program effectiveness. The appendix in-
cludes a number of attachments: (B-1) a summary of studies evaluating the
Title VI and Fulbright-Hays (Title VI/FH) programs, (B-2) a list of written
comments submitted by people and organizations knowledgeable about the
programs, (B-3) agendas of the public meetings held, and (B-4) the guides
we developed for the site visit interviews held at eight universities.
To begin our study and ensure a complete understanding of the pro-
grams’ statutory history and missions, including how their missions relate
to those of other similar federal programs, the committee (1) reviewed
descriptive information about the Title VI/FH programs and related federal
foreign language and international education programs and (2) commis-
sioned an analysis of the program’s legislative history.
As a result of our review of congressional language and the statutes
and the judgment of our individual members, the committee identified other
questions in addition to the eight key areas specified by Congress, aimed
at assessing the effectiveness of the programs at meeting their statutory
missions. We asked:
• How responsive are the programs to changing national priorities?
What is the role of Title VI programs in relation to other programs in ad-
dressing national needs?
0
OCR for page 310
0 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES
• To what extent are the programs producing best practices that are
used broadly in the international education arena? How are best practices
identified and shared?
• To what extent does Title VI help channel other university re-
sources toward national goals and fund programs or resources that they
might not otherwise fund?
• How effective are the programs at coordinating with one another
within programs, within universities, and across universities?
• How are the programs administered? What can and should be done
to improve future monitoring of the programs?
To guide the review and provide a model for how the programs sup-
port the eight key questions and are expected to lead to particular results,
we outlined a potential conceptual model. This model, which illustrates the
interconnectedness of the programs, is discussed in more detail below.
To assess the degree to which the programs meet the combined set of
questions, the committee undertook a series of activities:
• Reviewed all extant evaluations. The committee conducted a lit-
erature review, requested from ED copies of evaluations funded through
its International Research and Studies (IRS) Program, and asked experts
in the field about available evaluations. The committee reviewed all extant
evaluations identified, although very few were available. (See Attachment
B-1 for a summary of available evaluations.)
• Reviewed program monitoring data, selected grant applications, his-
torical financial data, and written comments from experts and officials. The
committee requested and received from ED all available program data,
including historical funding1 information and program monitoring data
collected via the Evaluation of Exchange, Language, International and Area
Studies (EELIAS) database. We had hoped that the EELIAS database would
be a rich source of information about program performance. Instead, the
problems we encountered using EELIAS and the comments from grantees
on our site visits led us to make the database itself a focus of our review
(see Box B-1 for a summary of database issues). In addition to the EELIAS
data, we also reviewed the FY 2006 performance plan for international edu-
cation and foreign language studies programs (domestic, overseas, and the
Institute for International Public Policy, IIPP) and the Program Assessment
1 Although committee staff worked with ED staff to clarify obvious errors or questions
about the data provided, the data were accepted as accurate and not independently verified
by the committee.
OCR for page 311
APPENDIX B
BOX B-1
Technical Issues with EELIAS Database
The committee encountered multiple technical issues with the EELIAS
database that hampered our ability to use the data for analyses:
• Lack of internal controls. Although the text of data entry screens indicates
that certain fields are required, the system does not always have internal con-
trols that enforce that requirement before data are submitted.
• Use of open-ended questions. For questions that might require a large num-
ber of response options, responses are left as open-ended questions. One
example is a recently added question asking whether a standardized proficiency
test was given to Foreign Language and Area Studies (FLAS) recipients and, if
so, the type of test. This resulted in responses such as “ACTFL,” “ACTF Scale,”
“ACTFL OPI,” “Oral Proficiency Interview,” “ACTFL/ILR,” and “ACCTF? I don’t
remember exactly,” all of which were probably indicating the same test. More
useful data might have been obtained by a series of predefined options for the
most common measures, even if it necessitated an additional, “other” category.
If the “other” option were supplemented by a question related to the language,
it would have provided even more useful information.
• Use of “other” category for key questions. In several cases, EELIAS supple-
ments a drop-down menu for key data with an “other” option; for example,
languages offered and type of language instructor. In the case of languages,
responses submitted in the other category included, among others, “Advanced
Arabic,” “Intensive Elementary Portuguese,” and “Elementary Persian I,” even
though Arabic, Portuguese, and Persian were included in the drop-down
menu.
• Limited data validation/review. The committee was told that reports are re-
viewed by individual project officers and that the information submitted is used
to make continuation awards. It was clear, however, that project officers have
limited time to review the consistency or reliability of data submitted from indi-
vidual grantees. ED has begun efforts with its contractors to review the consis-
tency of data across grantees and to address identified data issues.
• Inconsistency across programs. The data collected for individual programs
vary. Although this might be expected to some extent given the variability in
program missions, the differences do not always appear to be purposeful. For
example, it is not clear why only the Technological Innovation and Cooperation
for Foreign Information Access (TICFIA) Program is required to report on the
use of technology despite its recognized importance. Similarly, not all programs
are required to submit abstracts.
• Contractor coding of some information. The service area descriptor, a po-
tentially useful analytic tool added by the department after it assumed manage-
ment of the system, has been coded by the contractor based on the project
abstract. Given the limited information available in the abstract, this approach
has significant limitations. Project staff will be required to enter this information
in the redesigned system.
OCR for page 312
INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES
Rating Tool (PART) performance measurements of the Office of Manage-
ment and Budget (OMB) for international education domestic programs.2
The review of extant evaluations and monitoring data showed that
there were insufficient current data available to provide meaningful in-
sights on five of the eight original questions. Given the time constraints on
the review, the committee decided to use commissioned targeted analyses,
meetings with stakeholders and experts, public testimony, and site visits to
selected universities to address the research questions.
• Commissioned papers and targeted analyses. The committee re-
viewed or commissioned targeted analyses on increasing representation of
minorities,3 technology and instructional materials,4 oral proficiency as-
sessment,5 and implementation issues.6 We conducted an analysis of Mod-
ern Language Association enrollment data to explore the possible role of
National Resource Centers (NRC) in the teaching of less commonly taught
languages, and we examined various definitions of “internationalization”
of higher education. The committee also obtained preliminary data from
ED generated by a contracted study by InfoUse entitled “Study of Graduate
Fellowship Programs Participants,” which includes a survey of all FLAS and
Doctoral Dissertation Research Abroad (DDRA) Fellowships from 1997 to
1999.
• Reviewed written comments and conducted public meetings to get
input from experts, officials, and others and met with grantees in Wash-
ington and ED officials with responsibilities for these programs. Written
comments were submitted in response to a set of questions developed by
the committee (see Attachment B-2), and public meetings included presen-
2 PART reviews program evaluation and performance measurement information to assign
program ratings. The international education domestic programs were reviewed by OMB in
2004 and assigned a rating of “results not demonstrated.”
3 Joyner, C.C. (2006). Increasing representation of minorities in international service through
international education programs. Paper commissioned by the National Research Council
Committee to Review the Title VI and Fulbright Hays International Education Programs,
Washington, DC.
4 Joyner, C., and Suarez, T. (2006). Technology and instructional materials in Title VI
and Fulbright-Hays international education programs. Paper commissioned by the National
Research Council Committee to Review the Title VI and Fulbright-Hays International Educa-
tion Programs, Washington, DC.
5 Malone, M.E. (2006). The oral proficiency interview approach to foreign language assess-
ment. Paper commissioned by the National Research Council Committee to Review the Title
VI and Fulbright-Hays International Education Programs, Washington, DC.
6 Ruther, N.L. (2006). Implementation issues and options for the HEA Title VI and Fulbright
Hays programs. Paper written for the National Research Council Committee to Review the
Title VI and Fulbright-Hays International Education Programs, Washington, DC.
OCR for page 313
APPENDIX B
tations by grantees, federal agency staff, and outside experts (see Attach-
ment B-3). We also met with NRC and Language Resource Centers (LRC)
grantees when they were in Washington, DC, for an ED-sponsored grantee
meeting and with ED officials with responsibilities for these programs.
• Conducted site visits to eight universities. To become acquainted
with Title VI center programs and gather illustrative examples, committee
members and staff conducted site visits to eight universities (Georgetown
University, George Washington University, Indiana University, New York
University, Ohio State University, San Diego State University, the University
of California, Los Angeles, and the University of Wisconsin-Madison) and
reviewed selected grant applications from these universities. The universities
were chosen to represent a variety of public and private institutions, Title
VI funding levels, and geographic locations. Although the site visits do not
represent a random sample, they were selected to provide a general picture
of the diversity of NRC, Centers for International Business Education and
Research (CIBER), and LRC grantees. At each location, we met with uni-
versity administrators and staff, faculty, students, and librarians associated
with the Title VI programs on their campuses. An interview guide based
on the eight key areas specified by Congress guided the interviews at each
location (see Attachment B-4).
Several factors limited our review. First, in the case of the large center
programs (NRC, LRC, CIBER), Title VI typically represents a funding
stream rather than a discrete program; that is, Title VI funds combine with
other funding sources in order to achieve the desired outcomes, so that it is
difficult to attribute specific outcomes to Title VI (U.S. General Accounting
Office, June, 1998). Second, we did not have time to conduct a survey of
all grantees, so some of our observations and findings are not generalizable;
the illustrations based on the site visits, for example, may not be applicable
to all grantees. In addition, extant data provided significant information for
only three of the eight congressional questions, and the major data system
(EELIAS), which theoretically could have provided answers to many ques-
tions, has significant problems.
CONCEPTUAL MODEL
Understanding how a program is expected to lead to particular results
can be extremely useful in designing and implementing it and in planning
its evaluation. A basic logic model provides a systematic and visual way
to present and share an understanding of the relationships among the
resources/inputs, activities, outputs, outcomes, and impact expected of
OCR for page 314
INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES
TABLE B-1 Categories of Performance Measures
Category Definition Examples
Impacts The degree to which broad Increased competitiveness, enhanced level of
social objectives are achieved education, income of graduates, improved
quality of educational institutions
Outcomes Accomplishment of program Academic performance improvement, students
objectives attributable to accepted at next level of education, graduates
program outputs certified as teachers, employer satisfaction
Outputs The direct result of program Number of students enrolled, targeted students
activities completing training, students applying to next
level of education
Activities The work performed by the Amount of training given, counseling provided,
grantee that directly produces conferences held, reports published
the core products and services
Inputs Resources consumed by the Generally limited to funds and grant years
organization
SOURCE: U.S. Department of Education. (1998). Demonstrating results: An introduction to
the Government Performance and Results Act. Washington, DC: Author, Office of Postsec-
ondary Education.
the program.7 Logic models can range from an activities approach model,
which describes activities in great detail and is especially useful as a man-
agement tool, to a theory approach model, which emphasizes the theory
of change that has influenced the design and plan for a program and is
especially useful during program planning and design. A third kind of
logic model is perhaps more relevant to the committee’s work: an outcome
approach model, which shows the causal linkages thought to exist among
program components and is especially useful in designing effective evalua-
tion and reporting strategies.
In considering the programs and the results expected from them, it
will be useful to keep in mind the distinctions among several categories
of measures established under the Government Performance and Results
Act (GPRA) and used by ED in monitoring and reporting on its activities.
Table B-1 shows one way of describing those categories.8
7 W.K. Kellogg Foundation (2003). Logic model development guide. Battle Creek, MI:
Author.
8 Others might define categories in slightly different ways. In addition, assigning a measure to
one or another category (for example, outcome or impact) is often a judgment call on which
opinions might differ from one situation to another. Distinctions are sometimes made, as well,
between short-term outcomes and longer-term outcomes.
OCR for page 315
APPENDIX B
In requesting this review, Congress asked the committee to address
the performance of these programs in eight specific areas, which implies
that these areas might be seen as program goals about which informa-
tion is needed. Some of those eight areas reflect questions about activities,
whereas others address outputs or outcomes. Table B-2 shows these eight
areas with comments about the category of performance measure specified
or implied.
Figures B-1 through B-5 represent a modified “outcome approach”
logic model. That is, individual programs are grouped in ways that show
how their activities in combination are expected to lead to certain short-
term and longer term outcomes. While grouping programs provides a better
appreciation for the ways in which they interact, it necessitates some loss
of detail in describing the activities and their associated output measures.
Figure B-1, which presents an overview of all 14 programs, has the least
amount of detail.
The schematic in Figure B-1 shows all 14 programs along with their
links to the goals identified in the congressional request, other short-term
and longer term outcomes that can be inferred from that request, and the
legislation and regulations addressing allowable activities in the programs.
For the purpose of this report, the goal of “supporting research, educa-
tion, and training” is represented at the level of three inferred subgoals:
(1) increased research capacity, (2) enhanced body of knowledge about
foreign languages and area studies, and (3) increased institutional capac-
ity of education and training in K-12 and institutions of higher education
(IHE). Reading from left to right, the relationship between short-term and
longer term outcomes can be seen. For example, the schematic posits that
improving instructional materials is important because of its contribution
to improving the institutional capacity of education and training, which
in turn contributes to a decrease in the shortage of experts as well as an
increase in the global understanding of academics and U.S. educators. The
figure also shows that nearly all of the programs are related to multiple
outcomes.
In Figures B-2 through B-5, programs are grouped by similarity in the
way they function or in their purpose. By reducing the number of programs
in a single schematic, it is possible to include more detail about the activities
that are expected to lead to the outcomes.
Figure B-2 shows the six programs that provide funds to individuals:
the four Fulbright-Hays programs (DDRA, Faculty Research Abroad or
FRA, Group Projects Abroad or GPA, and Seminars Abroad or SA) as well
as FLAS and IIPP. Some of these programs could be expected to contribute
to a reduction in the shortage of experts, including those who are minori-
ties, while others should lead to an enhanced body of knowledge about
foreign languages and area studies or improved global understanding. As
OCR for page 316
INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES
TABLE B-2 Program Areas to Be Addressed in Responding to Congress
Area Comment
As stated, output measure would assess the extent to which
1. Supporting research,
education and training research, education, and training occurred. One could
plausibility link the question to short-term and longer term
in foreign language and
outcomes of (a) improved institutional research capacity, (b)
international studies,
including opportunities for improved institutional capacity to provide quality education
such research, education, and training in K-12 and institutions of higher education,
and training overseas and (c) improved body of knowledge about foreign
language and area studies.
Outcome measure would be decreased shortage of foreign
2. Reducing shortages of
foreign language and area language and area studies experts in public service and
experts academia.
Short-term outcome measure would be increased infusion
3. Infusing a foreign language
and area studies dimension of this dimension into elementary, secondary, and higher
throughout the education education, including professional education. One could
system and across relevant make an expected causal link between this outcome and a
longer term outcome of improved institutional capacity in
disciplines, including
professional education K-12 and IHE.
Short-term outcome measure would be improved
4. Producing relevant
instructional materials that instructional materials. That is, one would go beyond the
output measure of quantity of materials produced to assess
meet accepted scholarly
standards that they are “improved” over what previously existed
by virtue of meeting accepted scholarly standards. This
measure, in turn, would lead to the longer term outcome of
improved institutional capacity for education and training.
Output measure would describe the ways in which projects
5. Advancing uses of new
use technology. Short-term outcome measure would be
technology in foreign
language and international improved technology to enhance foreign language and
studies international studies and its use. This, in turn, would
contribute to the longer term outcome of an improved
institutional capacity for research, education, and training.
A short-term outcome measure would be enhanced
6. Addressing business
needs for international materials and activities to meet business needs.
knowledge and foreign
language skills
Outcome measure would be increased representation of
7. Increasing the numbers
of underrepresented minorities in international service. Lacking that information,
output measure would describe program activities and
minorities in international
service participants.
As stated, output measure would assess the extent to which
8. Conducting public
outreach/dissemination these outreach and dissemination activities were conducted.
One could plausibly link the question to the short-term
to K-12 and higher
outcome measure of improved outreach and dissemination
education, media,
government, business, and of information about other cultures.
the public
OCR for page 317
CIBE
↑ Materials &
activities to
↑ Global
meet business
BIE
understanding:
needs
media &
general public
AORC
↑ Research
capacity
↑ Global
understanding:
NRC
government
LRC
↑ Global
understanding:
business
↑ Outreach &
dissemination
IRS
↑ Global
TICFIA
↑ Technology understanding:
academia
UISFL
↑ Infusion of foreign ↑ Institutional
languages & area capacity/quality
studies in elem., sec., of education
↑ Global
& higher ed, including and training:
DDRA
understanding:
professional education K-12 and IHE
U.S. educators
↑ Instructional
FRA
materials
↓ Shortage of
foreign languages
GPA & area studies
experts in public
↑ Opportunities service &
for individuals academia
and groups to
SA
study & do
↑ Educational ↑ Minorities in
research
activities international
targeted to service
FLAS IIPP
minorities
FIGURE B-1 Schematic of all programs, showing short-term and longer term outcomes.
NOTE: Boxes with solid shading show programs. Bold boxes show goals and inferred subgoals identified in the congressional
B-1
request for the study.
Landscape view
Beneath image is a half inch left for caption, which can be seven inches long
OCR for page 318
Activities/Outputs Short-Term Outcomes Longer Term Outcomes
Graduate
DDRA Quality
students do
research is
dissertation
research produced on
foreign
abroad for 6
Enhanced body
languages &
to 12 months
of knowledge
area studies
about foreign
topics
Faculty do languages &
FRA research area studies
abroad
Participants’
language
competency
Teachers, students, &/or Decreased
& area
faculty go to short-term shortage of
studies
GPA study or intensive foreign
knowledge
Program alumni
overseas programs languages &
improve
are employed area studies
as experts in experts in
public service public service
Graduate students use &/or academia & academia
academic year & summer
programs to study foreign
FLAS
language &/or area studies
Increased
representation
Increased infusion of of minorities in
foreign languages & international
area studies in service
Minorities are targeted as elementary &
undergraduates and secondary education
IIPP followed through graduate
school with special activities ↑ Institutional
capacity/quality
of education
and training:
Alumni gain an improved K-12 and IHE
understanding & Improved global
K-12 educators do short-
Improved outreach & understanding:
knowledge of peoples &
term study & travel seminars
SA U.S. educators
dissemination of
cultures of other
abroad
information about
countries, which they
other cultures
share with colleagues
upon return
FIGURE B-2 Programs that provide funds to individuals.
NOTE: Boxes with solid shading show programs. Bold boxes show goals and inferred subgoals identified in the congressional
request for the study.
B-2
Landscape view
We have unified some of the boxes without having to recreate the entire figure
OCR for page 319
APPENDIX B
interim outcomes, one would expect to see improvement in individuals’
language competency, production of quality research, and an improved
understanding of peoples and cultures of other countries on the part of
nonspecialists.
In Figure B-3, the two programs specifically intended to support inter-
national business education (CIBER and Business and International Educa-
tion or BIE) are shown. Meeting business needs is intended to lead to an
improved global understanding in the business community, which in turn
is expected to enhance U.S. leadership in the global economy.
The three programs that function through institutional centers (Ameri-
can Overseas Research Centers, AORC, NRCs, and LRCs) are shown in
Figure B-4. In a variety of ways, these program activities are expected to
improve institutional capacities for research and for education and train-
ing. In turn, these institutional changes should lead to an improved body
of knowledge about foreign languages and area studies and to both an
increase in experts and to enhanced general global understanding. Their
activities are expected to focus especially on the less commonly taught
languages (LCTLs).
The remaining three programs (IRS, Undergraduate International
Studies and Foreign Language or UISFL, and Technological Innovation
and Cooperation for Foreign Information Access or TICFIA) address the
specialist and generalist outcomes through the intermediary outcome of
improving the institutional capacity for education and training on foreign
languages and area studies in K-12 and IHE. As Figure B-5 shows, their
activities increase the infusion of these areas into higher education, improve
instructional materials, improve outreach and dissemination, and improve
technology, which in turn improves institutional capacities for research and
education and training.
Not shown in the individual schematics is the overall impact expected
of the set of programs and some of the theoretical assumptions that underlie
their creation. For example, in its FY 2007 Performance Plan, ED describes
the program goal being addressed by all these programs as “to meet the
nation’s security and economic needs through the development of a national
capacity in foreign languages, and area and international studies.” Given
this goal, an impact assessment would attempt to determine whether the
nation is more secure and economically sound as a result of the programs.
An assessment of this sort is beyond the scope of the committee’s charge.
Indeed, although we endeavored to assess outcomes, the available data of-
ten restricted our analysis to assessment of inputs, activities, and outputs.
In the report, the committee makes recommendations aimed at moving
future program monitoring and evaluation efforts toward assessment of
outcomes.
OCR for page 339
APPENDIX B
• Linda Lim, Professor of Strategy, Stephen M. Ross
School of Business, Director, Center for Southeast Asian
Studies
• Patrick O’Meara, Professor and Dean, International
Programs, Indiana University
• Steven M. Poulos, Director, South Asia Language
Resource Center, The University of Chicago
• Karla Ver Bryck Block, Senior Program Specialist, IEPS,
Department of Education
5:00 p.m. Adjourn
Fourth Committee Meeting
October 5-6, 2006
OPEN SESSION AGENDA
The Beckman Center of the National Academies
The Board Room
Irvine, CA
(and via videoteleconference at The National Academy of Sciences
2100 C Street, NW, Room NAS 250, Washington, DC)
Thursday, October 5
8:00 a.m. PST Breakfast (for those in Irvine)
(11:00 a.m. EST)
8:30 a.m. PST Welcome
(11:30 a.m. EST) Janet Norwood, Chair, Committee to Review the Title
VI and Fulbright-Hays International Education
Programs
8:45 a.m. PST Panel Discussion of Title VI in the 21st Century
(11:45 a.m. EST) Al Fishlow, Columbia University (via
videoteleconference from Columbia University)
OCR for page 340
0 INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES
Martin Kramer, The Washington Institute for Near
East Policy
Robert Blake, University of California Language
Consortium
10:15 a.m. PST Break
(1:15 p.m. EST)
10:30 a.m. PST Recent Developments in IEPS Data Collection and
(1:30 p.m. EST) Reporting
Karla VerBryck Block, International Education
Programs Service
11:15 a.m. PST
(2:15 p.m. EST) The Honorable David S.C. Chu, Undersecretary
of Defense for Personnel and Readiness (via
videoteleconference from the Department of
Defense)
12:00 p.m. PST Break (for lunch in Irvine)
(3:00 p.m. EST)
1:00 p.m. PST End of Open Session
(4:00 p.m. EST)
NOTE FOR PUBLIC MEETINGS: This meeting is being held to gather information to help
the committee conduct its study. This committee will examine the information and material
obtained during this, and other public meetings, in an effort to inform its work. Although
opinions may be stated and lively discussion may ensue, no conclusions are being drawn at
this time; no recommendations will be made. In fact, the committee will deliberate thoroughly
before writing its draft report. Moreover, once the draft report is written, it must go through
a rigorous review by experts who are anonymous to the committee, and the committee then
must respond to this review with appropriate revisions that adequately satisfy the Academy’s
Report Review Committee and the chair of the National Research Council before it is consid-
ered a National Research Council report. Therefore, observers who draw conclusions about
the committee’s work based on today’s discussions will be doing so prematurely.
Furthermore, individual committee members often engage in discussion and questioning for
the specific purpose of probing an issue and sharpening an argument. The comments of any
given committee member may not necessarily reflect the position he or she may actually hold
on the subject under discussion, to say nothing of that person’s future position as it may evolve
in the course of the project. Any inferences about an individual’s position regarding findings
or recommendations in the final report are therefore also premature.
OCR for page 341
APPENDIX B
ATTACHMENT B-4
Site Visit Interview Guide
8-03-06
Topics to be addressed in every interview:
• The 8 areas of review, including how they evaluate their own prog-
ress in these areas.
• Unique strengths/challenges of their programs.
• Impact of Title VI/FH funding (except student interviews).
• How can Title VI/FH be improved (except student interviews)?
Key to eight areas of review:
Q1. Supporting research, education, and training in foreign languages
and international studies, including opportunities for such research,
education, and training overseas;
Q2. Reducing shortages of foreign languages and area experts;
Q3. Infusing a foreign language and area studies dimension throughout
the educational system and across relevant disciplines including
professional education;
Q4. Producing relevant instructional materials that meet accepted
scholarly standards;
Q5. Advancing uses of new technology in foreign language and
international studies;
Q6. Addressing business needs for international knowledge and foreign
language skills;
Q7. Increasing the numbers of underrepresented minorities in
international service; and
Q8. Conducting public outreach/dissemination to K-12 and higher
education, media, government, business, and the general public.
PROVOST/SENIOR INTERNATIONAL OFFICER
Questions related to the eight study areas:
1. Could you give us an overview of your international programs
across the university, including among faculty, undergraduates,
graduate students, and the professional schools? (Q1, Q3)
OCR for page 342
INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES
2. How do you monitor and evaluate the effectiveness of these pro-
grams? What sources of information do you use to assess outcomes
and/or impacts?
Question about the unique aspects of their program:
3. What do you see as your university’s greatest strength(s) in the field
of international education? Its greatest challenge(s)?
Questions about the impact of Title VI/FH funding:
4. Please tell me about the impact of Federal Title VI/FH funding on
the development and evolution of your university’s international
programs. Does the availability of Title VI/FH funding allow you
to support programs or activities that you might not be able to oth-
erwise? Can you provide some specific activities?
(Probes): Languages that would not otherwise be offered?
Interdisciplinary work that might not otherwise be car-
ried out?
Outreach or support to K-12 education?
5. What would happen at your university if Title VI funds were in-
creased by 50%? If they were eliminated?
Question about how to improve Title VI/FH:
6. Are there ways in which Title VI/FH programs might be changed so
that they were more effective?
(Probes): More flexibility in the use of funds.
Different allocation of funds among the program’s
several components and objectives.
More funding for professional schools.
NATIONAL RESOURCE CENTERS (NRC)
DIRECTORS AND DEPUTIES
Questions related to the eight study areas:
1. Give us an overview of your center and how it fits in the broader
university context. What has been the role of Title VI/FH in the
development and evolution of your center? (Q1, Q3)
2. How do you publicize your center and its offerings? How do you
recruit students?
OCR for page 343
APPENDIX B
(Probes): Do you think the center is widely known on campus?
Are your language classes fully subscribed, or could
you accept more students into your classes or into the
program as a whole?
3. How and how well are the center’s instructional needs in language
being met? (Q1)
(Probe): Through language and literature departments and/or
other programs?
Through collaboration with the LRC?
4. What are the language requirements for the certificate and degree
programs, for entering students and for completing students? What
standard or standards do you use to measure language proficiency?
(Q1)
5. What career paths do students who receive certificates and/or de-
grees follow? (Q2)
6. Tell us about your outreach activities. Who is your primary audi-
ence? Can you tell us about some of your most effective outreach
activities and why you consider them effective? (Q6, Q8)
Question about the unique aspects of the program:
7. What do you see as your center’s greatest strength(s)? Your greatest
challenge(s)?
Questions about the impact of Title VI/FH funding:
8a. (For Centers with continuous funding) Please tell me about the
impact of federal Title VI/FH funding on your center’s programs.
What activities are supported?
(Probes): Is it used to support core instruction?
Is it used to support specialized activities like teaching
of LCTLs?
Does it allow you to support programs that you might
otherwise not be able to offer, such as particular lan-
guages, interdisciplinary research, or support to the
K-12 system?
OCR for page 344
INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES
8b. (For centers that lost funding and then regained it)1 Please tell me
about how your center’s programs were affected by the temporary
loss of federal funding.
(Probes): What activities are you conducting or planning now
that you were not able to carry out when you lost fed-
eral funding? Can you provide examples?
What sources of funding did you turn to when federal
funds were gone?
What types of activities were you able to maintain?
What types of activities were you forced to eliminate?
9. (For centers with continuous funding—do not use for centers that
temporarily lost funding) What would happen at your center if Title
VI funds were increased by 50%? If they were eliminated?
Question about how to improve Title VI/FH:
10. Are there ways in which Title VI/FH programs might be changed
so that they were more effective?
(Probes): More flexibility in the use of funds.
Different allocation of funds among the various pro-
grams and objectives.
More involvement of professional schools.
LANGUAGE RESOURCE CENTERS (LRC)
DIRECTORS AND DEPUTIES
Questions related to the eight study areas:
1. In what specific ways is your center helping to support research,
education, and training in foreign languages across the university
(including in the professional schools)? (Q1,Q3)
2. Tell us about the instructional materials you produce. Who is the
primary audience for these materials, and how do you ensure that
they are of high quality and relevant to the audience? (Q4)
(Probe): Are the materials reviewed by other scholars who have
1 San Diego State University: none; George Washington University: none; Georgetown Uni-
versity: none; Indiana University: East Asia and West Europe (lost funding 2000-2006); New
York University: Latin America (lost funding 2003-2005) and Center for European Studies
(lost funding 2000-2002); Ohio State University: East Asia (lost funding 2003-2005); Univer-
sity of California, Los Angeles: Latin American Center (lost funding 2000-2003) and Europe/
Russian Center (lost funding 2003-2005).
OCR for page 345
APPENDIX B
language and area expertise? If so, what is the process
by which the materials are reviewed?
3. How do you evaluate the use and impact of these instructional
materials?
(Probe): What types of data do you maintain on the use of your
materials?
Among various audiences, such as K-12 education,
higher education, professional schools, and business?
(Q4, Q6, Q8)
4. Do you coordinate or collaborate with other LRCs or NRCs on
campus? With other international-focused programs on campus? Do
you collaborate with Title VI-funded programs at other universities?
Please give examples.
Question about the unique aspects of the program:
5. What do you see as your center’s greatest strength(s)? Your greatest
challenge(s)?
Questions about the impact of Title VI/FH funding:
6. Please tell me about the impact of federal Title VI/FH funding on your
LRC. What activities are supported that might not otherwise?
(Probes): Is it used to support core language instruction?
Is it used to support specialized activities related to
LCTLs?
Does it allow you to support programs that you might
otherwise not be able to offer, such as particular lan-
guages, interdisciplinary research, or support to the
K-12 system?
7. What would happen to your center if Title VI funds were increased
by 50%? If they were eliminated?
Question about how to improve Title VI/FH:
8. Are there ways in which Title VI/FH programs might be changed so
that they were more effective?
(Probes): More flexibility in the use of funds.
Different allocation of funds among the various pro-
grams and objectives.
OCR for page 346
INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES
CIBER DIRECTORS
Questions related to the eight study areas:
1. Can you provide some examples of the ways in which your pro-
grams address business needs for international knowledge and for-
eign language skills? (Q6)
2. Tell us about your outreach activities. Do you target outreach pri-
marily to business or to several different audiences? Which of these
efforts do you view as most effective and why? (Q3, Q6, Q8)
(Probe): In addition to business, do you reach out to other audi-
ences, such as K-12 education, other parts of your univer-
sity, the general public, the media?
3. Do you coordinate or collaborate with other CIBERs at other uni-
versities? With other Title VI-funded programs on campus? Other
international-focused programs on campus? Please give examples.
Questions about unique aspects of the program:
4. What do you see as your center’s greatest strength(s)? Your greatest
challenge?
Questions about the impact of Title VI/FH funding:
5. Please tell me about the impact of federal Title VI/FH funding on your
CIBER. What activities are supported that might not otherwise?
6. What would happen to your center if Title VI funds were increased
by 50%? If they were eliminated?
Question about how to improve Title VI/FH:
7. Are there ways in which Title VI/FH programs might be changed to
be more effective?
(Probes): More flexibility in the use of funds?
Different allocation of funds among the various pro-
grams and objectives?
FACULTY
NOTE: Try to meet with faculty alone, without the presence of the NRC
director, LRC director, or other administrator.
OCR for page 347
APPENDIX B
Questions related to the eight study areas:
General orientation
Ask faculty to identify their discipline, home department, particular field of
expertise, and center they are associated with.
1. Would you describe the intellectual benefits you gain from partici-
pating in the center?
(Probe): Do you gain insights from other disciplines?
Does the center provide opportunities for interdisciplin-
ary collaboration in research, teaching, and/or service?
Do you learn about new scholarship?
2. How do you publicize your center and its offerings? How do you
recruit students?
(Probes): Do you think the center is widely known on campus?
Are your language classes fully subscribed, or could
you accept more students into your classes or into the
program as a whole?
3. (For language instructors) How many levels of instruction are of-
fered in the language(s) you teach? (Q1)
(Probes): What levels of proficiency are required for students in
certificate programs, M.A., Ph.D. programs? How is
proficiency measured?
4. Are you or have you been an undergraduate or graduate student
advisor? What are the motivations and goals of students pursuing
certificate and degree programs in your field? In international/area
studies in general? (Q2)
(Probe): What career fields do graduates enter (government, busi-
ness, and academia)?
Question about unique aspects of the program:
5. What do you see as your university’s greatest strength(s) in the area
of international education and foreign language instruction? Its
greatest challenge?
Questions about the impact of Title VI/FH funding:
6. Please tell me about the impact of federal Title VI/FH funding on
your teaching, research, and other activities. For what types of ac-
OCR for page 348
INTERNATIONAL EDUCATION AND FOREIGN LANGUAGES
tivities have you and/or your students received direct support from
Title VI and/or Fulbright-Hays?
(Probe): Can you provide any examples of ways in which Title
VI/FH funds helped to leverage additional university re-
sources for your research and/or teaching?
7. What would happen to your research and teaching if Title VI funds
were increased by 50%? If they were eliminated?
Question about how to improve Title VI/FH:
8. Are there ways in which Title VI/FH programs could be changed to
be more effective?
(Probes): More flexibility in the use of funds?
Different allocation of funds among the program’s sev-
eral components and objectives?
More funding for professional schools?
STUDENTS
General orientation
Ask students to identify themselves based on their field of study, whether
they are studying a language, and if so, what language, the NRC with which
they are associated, and whether they have benefited from direct support
from Title VI (i.e., FLAS or DDRA)
Questions related to the eight study areas:
1. What motivated you to enroll in language/area studies?
(Probes): What did you expect to get from the program (or
courses) when you enrolled in it/them?
Did you have any ideas about a career in/related to this
program? (Q2)
2. Where and how did you first hear about the area or language pro-
gram you are enrolled in?
(Probes): Do you think other students are aware of these pro-
grams?
Do you feel your center is isolated from or connected
to other departments, divisions, and schools? (Q3)
3. In what subjects or disciplines has your learning experience been
the strongest? (Q1)
OCR for page 349
APPENDIX B
(Probe): Have you found that language instruction is integrated
within courses in such fields as economics, political sci-
ence, religion and culture?
4. (For those who have gone abroad) Was the preparation appropri-
ate and adequate for your to take advantage of the cultural and
linguistic immersion? What aspects of the learning facilities and/or
living conditions enhanced or detracted from your study or research
abroad? (Q1)
5. In what ways (if any) do you expect to use your training in area
studies, international studies, or foreign languages after graduation
or after completing your graduate degree? (Q2)
(Probe): Are you considering a job in government, business, aca-
demia or a nonprofit?
6. (For graduate students): What aspects of the facilities, that is the
library and the technological resources, support or detract from
your research? (Q5)
7. In what ways does the faculty integrate technology into instruction
in foreign language and area studies? (Q5)
8. Have your studies met or failed to meet your expectations?
(Probe): In what ways? Which aspects of your studies?
Question about the unique aspects of their program:
9. What do you see as your university’s greatest strength(s) in the field
of international education? Its greatest challenge?
INTERNATIONAL STUDIES LIBRARIANS
From your perspective:
1. In what ways have Title VI funds enabled you to provide greater
international studies and language resources to the university
community?
2. Are these resources accessed by people outside the university?
Who?
3. How do you measure the people who have accessed these
resources?
