4
Assessment of the Impact of MRSEC Education and Outreach

Education and outreach (EO) covers a broad range of activities that serve kindergarten through grade 12 (K-12) students and teachers; undergraduate, graduate, and postdoctoral researchers; policy makers; and the public. Consistent with the breadth of activities, EO projects serve many different purposes: educating future scientists and engineers; broadening the participation of underrepresented groups in science, technology, engineering, and mathematics (STEM) disciplines; increasing science literacy in the public; informing the public and policy makers about scientific and technical issues; improving K-12 science education; and developing a scientific and technical workforce.

INTRODUCTION

Although all National Science Foundation (NSF) proposals are required to address the “Broader Impacts” of the proposed research, an EO component is specifically required by the Materials Research Science and Engineering Centers program (MRSEC program) announcement (see Box 4.1). In contrast to the efforts of most individual-investigator and small-group grants, many (but not all) Materials Research Science and Engineering Centers (MRSECs) choose to have at least one part-time person (the EO coordinator) dedicated to managing the EO projects.

NSF does not require MRSECs to conduct specific EO activities, with the exception of participation in the Research Experiences for Undergraduates (REU) program and a requirement for plans to increase the number of people from underrepresented groups (defined by NSF as women, Hispanics, African-Americans,



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward 4 Assessment of the Impact of MRSEC Education and Outreach Education and outreach (EO) covers a broad range of activities that serve kindergarten through grade 12 (K-12) students and teachers; undergraduate, graduate, and postdoctoral researchers; policy makers; and the public. Consistent with the breadth of activities, EO projects serve many different purposes: educating future scientists and engineers; broadening the participation of underrepresented groups in science, technology, engineering, and mathematics (STEM) disciplines; increasing science literacy in the public; informing the public and policy makers about scientific and technical issues; improving K-12 science education; and developing a scientific and technical workforce. INTRODUCTION Although all National Science Foundation (NSF) proposals are required to address the “Broader Impacts” of the proposed research, an EO component is specifically required by the Materials Research Science and Engineering Centers program (MRSEC program) announcement (see Box 4.1). In contrast to the efforts of most individual-investigator and small-group grants, many (but not all) Materials Research Science and Engineering Centers (MRSECs) choose to have at least one part-time person (the EO coordinator) dedicated to managing the EO projects. NSF does not require MRSECs to conduct specific EO activities, with the exception of participation in the Research Experiences for Undergraduates (REU) program and a requirement for plans to increase the number of people from underrepresented groups (defined by NSF as women, Hispanics, African-Americans,

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward BOX 4.1 The MRSEC Request for Proposals on Education and Outreach The scope of activities of each Materials Research Science and Engineering Center (MRSEC) depends on the capabilities of the proposing organization. Examination of the 2004 program solicitation from NSF reveals that among the list of activities that most MRSECs incorporate “to an extent consistent with the size and vision of the Center” are the following:1 Programs to stimulate interdisciplinary education and the development of human resources (including support for underrepresented groups) through cooperation and collaboration with other organizations and sectors, as well as within the host organization. Cooperative programs with organizations serving predominantly underrepresented groups in science and engineering are strongly encouraged. The program solicitation description of what should be included about education and outreach in the proposal is as follows: Education, Human Resources Development. Describe the education and human resource goals, provide a rationale for those goals, and indicate desired outcomes for the 6-year period. Briefly describe how the education goals integrate strategically with the research and organizational/ partnership opportunities of the Center. Outline plans for increasing the participation of women and underrepresented minorities in Center research and education activities. Outline plans for seminar series, colloquial workshops, conferences, summer school and related activities, as appropriate. Describe any additional education programs not included in other sections of the proposal. Limit: 3 pages. The program solicitation also specifies that “innovative interdisciplinary educational ventures” are appropriate topics for seed funding.    1National Science Foundation (NSF), Program Solicitation for Materials Research Science and Engineering Centers Program, NSF 04-580, NSF, Arlington, Va., 2004. and Native Americans/Pacific Islanders) involved in STEM fields. Each MRSEC is encouraged to pursue activities consistent with the research and organizational/ partnership opportunities of the center as well as the size and local context of the center. The committee collected data from a range of sources. Written sources included MRSEC annual reports, program descriptions, MRSEC Web sites, grant proposals, journal papers, and program evaluations. Additional sources included telephone interviews, Research Experiences for Teachers (RET) conference reports, MRSEC EO workshop proceedings, and materials from the National Research Center Educator Network (NRCEN) Web site. A survey specific to EO issues was sent to EO coordinators and MRSEC directors in April 2006 asking for information in order to address issues raised from the preliminary analysis. Information from site visits

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward was combined with data obtained during discussions with many of the MRSEC EO coordinators at a MRSEC directors’ meeting in Chicago in April 2006 (see Appendix F for more information). OVERVIEW OF MRSEC EDUCATION AND OUTREACH ACTIVITIES The flexibility of the NSF EO guidelines has produced a broad range of MRSEC EO activities. As a group, MRSECs reach many difference audiences, including current and future researchers, K-12 and college teachers, students from K-12 through graduate school and (to a lesser extent) journalists, policy makers, and the public. Most MRSECs have a dedicated EO coordinator. EO coordinators may have backgrounds in K-12 education, a STEM discipline, and/or education research. Many EO coordinators divide their time between MRSEC and other programs with similar missions. Some EO coordinators’ salaries come entirely from the MRSEC grant; however, it is not uncommon for part of their salary to be paid by the university where the MRSEC is located and/or by other grants. EO coordinators may be involved in setting goals and priorities for the MRSEC, developing curricular and other materials, establishing and maintaining partnerships, facilitating researcher involvement, obtaining additional funding for EO activities, coordinating with other (internal and/or external) EO programs, and assessing, evaluating, and disseminating research or education-assessment results. Although the EO coordinator is responsible for organizing EO activities and building infrastructure, researchers play an active role in many EO programs. Some MRSECs require a specific number of hours per year from each MRSEC researcher (which includes undergraduate and graduate students, postdoctoral researchers, and faculty), leading to a wide variety of reported researcher involvement. MRSECs also may provide funding for activities initiated by researchers or EO participants (such as teachers) through mini-grant programs. MRSEC EO activities can be separated into three general modes of operation: MRSEC-funded activities, in which the MRSEC takes the primary leadership role and provides the majority of the funding from the MRSEC grant; MRSEC-leveraged activities, in which the MRSEC has obtained additional funding (beyond the MRSEC grant) for EO projects and provides the primary leadership; and MRSEC-associated activities, in which MRSEC researchers participate in programs run by other entities. The MRSEC may provide a small portion or none of the funding for the program, but it may contribute significant volunteer time.

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward Goals of MRSEC Education and Outreach MRSEC EO goals generally originate during proposal development. The goals reported by EO coordinators fall into four main categories: Preparing the future scientific and technical workforce, including researchers at all levels from high school to postdoctoral researchers; Improving the scientific content knowledge of nonscientists through activities for the public, policy makers, and/or K-12 schools; Improving public attitudes toward science, again targeting both the public and K-12 students; and Broadening participation by increasing the number of women and other underrepresented groups involved in MRSEC activities. Most of the non-research-oriented programs (i.e., those for K-12 and the public) are driven by local factors, including existing programs and specific MRSEC personnel interests. Although most EO programs have a materials science theme, programs for K-12 students and teachers often focus more broadly on general science and engineering or on the scientific process. The subsections below describe some of the current MRSEC activities designed to address these goals for different audiences. Goal: Preparing the Future Scientific and Technical Workforce One of the most important functions of a MRSEC is preparing the future scientific and technical workforce. The majority of MRSEC research, as in most academic environments, is carried out by graduate students and postdoctoral researchers (see Table 4.1, and Figure 2.7 in Chapter 2). Although all research grants train graduate students and postdoctoral researchers, MRSECs have a unique opportunity to help students develop skills that they may not learn working for an individual investigator. Most MRSEC students work in collaborative, interdisciplinary groups and learn to use equipment and techniques in laboratories, and sometimes disciplines, beyond their own. Most MRSEC student and postdoctoral researchers receive mentoring from multiple professors but do not typically participate in other MRSEC EO activities. The preparation of the future scientific and technical leadership in materials research often is not reported formally as an EO component; however, it is a very important function of MRSECs. Specific activities falling under the general goal of preparing the future scientific and technical workforce include the REU and RET programs, described below. Research Experiences for Undergraduates is an NSF-wide program that provides undergraduates with a paid summer research experience lasting from 8 to 10

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward TABLE 4.1 Percentages of Women and Underrepresented Minorities (URMs) Working in MRSECs in 2005 MRSEC Demographics   Total Percentage REU students Total 279     Women 148 53.1   URM 95 34.1 Undergraduates (App. B) Total 218     Women 86 39.5   URM 44 20.2 RET teachers Total 69     Women 33 47.8   URM 16 23.2 Other pre-college Total 1,545     Women 890 57.6   URM 251 16.3 K-12 students Total 8,651     Women 3,939 45.5   URM 2,239 25.9 Undergraduate faculty Total 31     Women 11 35.5   URM 3 9.7 Graduate students Total 554     Women 149 27.0   URM 30 5.4 Postdoctoral researchers Total 164     Women 33 20.1   URM 5 3.1 Faculty Total 419     Women 56 13.4   URM 14 3.3 Technical support staff Total 81     Women 15 18.5   URM 3.5 4.3 Nontechnical support staff Total 41     Women 37 90.2   URM 1 2.4 NOTE: For comparison, Table 4.2 shows the approximate percentages of women and underrepresented minorities in the fields most represented in MRSECs. Note that the data in Table 4.2 are from 2002 for the bachelor’s and master’s degree data, and 2003 for the Ph.D. statistics. There is some uncertainty in the values in Table 4.1 because MRSECs are not required to report how much of the support came from the MRSEC. For example, the number of graduate students claimed in some reports is much greater than the number that the budget shows could be supported. REU, Research Experiences for Undergraduates; RET, Research Experiences for Teachers. SOURCE: Data in this table are as reported in annual reports of the MRSECs, Appendixes B and C. The annual reports are uniformly prepared in accordance with guidelines from the National Science Foundation.

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward weeks. NSF funds REU supplements, which are granted to individual researchers, and REU sites, which bring together larger numbers of students (usually from other campuses) under a common research theme. REU sites are expected to provide additional activities such as seminars on ethics, science communications, job strategies, and other professional development. Most REU participants present posters and/or talks at the end of their experience. NSF requires MRSECs to have an REU site in which the majority of participants are from other campuses. REU programs often admit students from a range of degree programs, which provides a path to graduate materials science study for students with non-materials-science undergraduate degrees. Some MRSECs work with students from departmental-based REU sites. REU programs may be funded directly from the MRSEC budget or through a separate grant proposal to the REU program. Some MRSECs provide other research opportunities for undergraduates. In addition to employing local undergraduates year-round, some programs bring undergraduates (with or without accompanying faculty members) from minority-serving institutions or primarily undergraduate institutions for summer research. Some programs offer the opportunity to continue research collaborations during the academic year as well. Research Experiences for Teachers (RET) is an NSF-wide program offering K-12 teachers opportunities to work with a MRSEC during the summer (see Box 4.2). The RET program has as a goal involving teachers in research and transferring the knowledge gained from these experiences to the classroom. Teachers typically spend from 6 to 8 summer weeks with the MRSEC and receive a stipend of up to 2 academic months’ salary. Some programs continue interactions with the teacher and his or her students during the academic year, which may include MRSEC researchers visiting schools or students visiting MRSEC laboratories. Many MRSECs provide a small amount ($1,000) of funding for supplies or other materials necessary to implementing curriculum. Some programs allow teachers to participate for more than 1 year, while others limit participation to 1 year. The implementation of the RET from MRSEC to MRSEC varies much more than that of the REU program. Some RET programs essentially duplicate the REU structure (and may have common activities). At the other extreme are programs that have little or no formal research component, with teachers developing materials-science-related curricula to use in their classrooms. Some MRSECs have involved exceptional high-school students in research. These experiences range from a few weeks in the summer to year-round involvement. High-school students may participate in REU and/or RET activities, and some have made presentations at local and national meetings and have coauthored publications.

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward BOX 4.2 Research Experiences for Teachers The Research Experiences for Teachers (RET) program was formalized by the National Science Foundation (NSF) Directorate for Engineering in FY 2001 with these goals: “to involve middle and high school teachers in engineering research in order to bring knowledge of engineering and technological innovation to the pre-college classroom.”1 Guidelines sent to the Materials Research Science and Engineering Centers (MRSECs) in January 2004 were based on a preceding “Dear Colleague Letter” of January 26, 1999, circulated by the NSF Directorate of Mathematical and Physical Sciences (MPS) and including the following directives (in fact, the first RET-type program was probably the Research Experiences for Science Teachers [REST] program at the Northwestern University MRSEC in 1999): The RET activity is designed to allow the participation of K-12 teachers in established Research Experience for Undergraduate (REU) sites. Eligible for this supplement are regular REU sites supported by MPS and all Centers that support REU site-like programs (such as MRSECs). The request should describe: 1) The plan for teacher activities and the nature of involvement with the REU site program; 2) Plans for incorporation of new learning into the K-12 classroom; 3) The teacher recruitment plan and the selection process; 4) The PI’s [principal investigator’s] experience in involving teachers or any previous collaborative work with teachers; 5) Plans for assessment of the program; and 6) Progress for any previously funded RET activity. Funding for the supplement may include up to two months of the teacher’s annualized salary. As with all REU awards, indirect costs are not allowed, an administrative allowance limited to 25% of the teacher stipend is permitted. Requests may be for one year or for a 3-year period. The RET program is further described by program solicitations originated in the NSF Directorate for Engineering and Directorate for Biological Sciences. Through these partnerships, the RET program aims to build long-term collaborative relationships between both in-service and pre-service K-12 teachers, community college faculty, and the engineering research community; support the active participation of these teachers and future teachers in research and education projects funded by NSF/ENG; facilitate professional development of K-12 teachers and community college faculty through strengthened partnerships between institutions of higher education and local school districts; and encourage researchers to build mutually rewarding partnerships with teachers. (NSF Program Solicitation 03-554) For example, the teacher may participate in the design of new experiments, modeling or analysis of experimental data or other activities that will result in intellectual contributions to the project. Since it is expected that the RET supplement experience will also lead to transfer of new knowledge to classroom activities, the RET supplement description should also indicate what sustained follow-up would be provided to help in translating the teacher’s research experience into classroom practice. (NSF Program Solicitation 05-524)    1National Science Foundation, Program Solicitation for Research Experiences for Teachers: Supplements and Sites, NSF 03-554, Washington, D.C., 2003.

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward Perhaps the most direct education impact of MRSECs is on the graduate students who research and learn within the program. These students are exposed to multiple principal investigators and shared facilities, and they often participate in center-based journal clubs and discussion groups. From its site visits, the committee learned that in some cases MRSECs are the great enabler of this broadened educational experience, and that in other cases MRSECs are the result of a preexisting disposition on the campus. However, independent of whether MRSECs uniquely train graduate students, this is an area of significant value for the program. Goal: Improving Understanding and Appreciation of Science and Engineering In addition to encouraging young people to pursue science and engineering study, some MRSEC EO programs attempt to increase the scientific literacy of the current and future citizenry. These efforts include approaches that are both formal (K-12 schools and universities) and informal (talks for the public, educational sessions for legislators or reporters). Programs include improving content knowledge by means of involvement in local K-12 and college-level education, developing curricular materials, and informing policy makers. Other approaches focus on improving understanding of how research works, contributing to awareness of career options, and promoting general enthusiasm for science and engineering. At the college level, MRSECs have developed courses and curricula for graduate and undergraduate courses. Most of these classes are highly interdisciplinary, focus specifically on the MRSEC topic area, and are designed to involve students in different departments. Some are cotaught by faculty members from different disciplines. Some MRSECs report developing and/or implementing new pedagogical techniques that enhance student learning (i.e., active learning techniques). See Appendix D, “Further Information on Education and Outreach Activities.” A broad range of activities at the K-12 level includes curriculum development, classroom visits from MRSEC researchers, professional development activities for teachers, summer enrichment programs for teachers and/or students, and laboratory visits. Because of the standardized testing requirements imposed by the No Child Left Behind Act of 2001 (Public Law 107-110), many K-12 activities focus on general science and/or engineering rather than on the research theme of the MRSEC. Outreach to the public generally occurs in informal settings including lectures, demonstration shows, building or contributing to exhibits at science museums, and workshops for policy makers, journalists, and business people. MRSECs also hold open houses, sponsor science days for parents and children, and may develop audio and/or video materials.

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward Goal: Broadening Participation One of the few activities specifically mandated by NSF is that of increasing the participation of women and other underrepresented groups in MRSECs. Although broadening participation by people in these groups has always been an important part of the Broader Impacts requirement of NSF proposals, MRSECs have been required to develop formal “diversity plans” since 2001 and are expected to show results from those plans over the course of the MRSEC grant. Table 4.1, which displays demographics of MRSEC participation, shows that MRSECs are having the most success at broadening participation in undergraduate and pre-college audiences but that the involvement of underrepresented minorities in particular needs to be much improved at the graduate-student and higher levels. For reference, Table 4.2 shows the overall percentages of women and minorities involved in materials research and related fields. Some of the strategies that MRSECs use to broaden participation include partnerships with minority-serving institutions (some through the PREM program—see Box 4.3) and/or women’s colleges; interactions with K-12 schools serving underrepresented populations; alliances with professional associations for minority scientists and engineers; and participating in or holding special programs for underrepresented groups. MRSEC–MRSEC Interactions One of NSF’s goals for the MRSEC program is for it to be a network of centers focused on advancing research and education in materials science and engineering (MSE). Some aspects of the EO program are shared by many MRSECs, which offers opportunities to share information and resources. Some of these interactions have developed around common programs such as REU and RET, while other efforts (such as the participation of EO coordinators in education-themed MRSEC TABLE 4.2 Percentage of Women and Underrepresented Minorities (URMs) in the Fields Most Represented in Materials Research Science and Engineering Centers   Physics Chemistry MS&E   Women (% in field) URM (% in field) Women (% in field) URM (% in field) Women (% in field) URM (% in field) Bachelor’s degree 23 10 50 16 30 7.5 Master’s degree 21 7 46 8 27 5.8 Ph.D. 14 6 35 6 16 6.0 NOTE: Figures for bachelor’s and master’s degrees are for 2002, while figures for Ph.D. degrees are from 2003. Figures do not include “unknown” designations. MS&E, Materials Science and Engineering. SOURCE: National Science Board, Science and Engineering Indicators 2006, National Science Foundation, Arlington, Va., 2006. See http://www.nsf.gov/statistics/seind06/.

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward BOX 4.3 Partnership for Research and Education in Materials The Partnership for Research and Education in Materials (PREM) was established in the NSF’s Division of Materials Research (DMR) in 2004 to develop materials research and education partnerships between minority-serving institutions (MSIs) and MRSECs. The 10 currently active PREM awards are listed below, with their dates of initial award in parentheses: California Institute of Technology–California State University at Los Angeles (2004) Carnegie Mellon University–Florida Agriculture and Mechanical University (2004) University of Pennsylvania–University of Puerto Rico at Humacao (2004) University of Wisconsin–University of Puerto Rico at Mayaguez (2004) Princeton University–California State University at Northridge (2006) University of California at Santa Barbara–Jackson State University (2006) Cornell University–Norfolk State University (2006) Johns Hopkins University–Howard University (2006) Cornell University–Tuskegee University (2006) Harvard University–University of New Mexico (2006) PREM strives to create cooperative research teams and provide experimental facilities to the partner institutions, thus providing additional research and education opportunities for students and faculty. Since PREM is still a rather new program, it is too early to determine its impact on this problematic issue. directors’ meetings) have been initiated by the NSF. These interactions are summarized in Appendix D. Distribution of Education and Outreach Resources MRSECs spend approximately 10 percent of their budgets on EO; however, this figure may be misleading, because some activities are funded by supplemental grants. Some MRSECs fund their REU and/or RET activities entirely from the MRSEC budget, while others fund them from a separate grant or supplement. The origin of the funding and whether it is being accounted for in the annual reports is not always clear. Therefore, a representative group of MRSECs was asked to provide more detailed information about how their EO budgets are distributed. Funding for the RET program comes entirely from the Office of Multidisciplinary Activities in the NSF Directorate for Mathematics and Physical Sciences, regardless of whether the RET program is included in the original MRSEC budget, is a separate grant, or is a supplement to the MRSEC budget. The analysis of the detailed budget breakdowns shows: The majority (more than 75 percent for most MRSECs) of the EO budget goes to research-related EO (involving students from high school to gradu-

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward ate school and teachers in research, research-related conferences and workshops, and the formation of research-based partnerships with primarily undergraduate and/or minority-serving institutions) and to EO personnel costs. The majority of the K-12 and public outreach programs, with a very few exceptions, comprise a very small fraction of the MRSEC EO budget. The MRSEC contributes a few percent or less of the total funding for the majority of these activities. Many MRSEC EO activities receive funding from sources outside the MRSEC grant per se. In addition to REU and/or RET supplements, institutions may provide support, and a number of MRSECs lead or participate in separate education grants, such as the Integrative Graduate Education and Research Traineeship, PREM (see Box 4.3), Nanoscale Undergraduate Education, and Graduate Fellows in K-12 Education. IMPACT OF MRSEC EDUCATION AND OUTREACH PROGRAMS EO plays an important role in supporting U.S. excellence in science, technology, engineering, and mathematics (STEM), and the MRSEC program invests significant resources in EO. The study committee addressed two questions: (1) Are MRSECs meeting NSF’s and their own self-determined goals in education and outreach? (2) Are those goals the best use of MRSEC resources? MRSECs were asked to provide the committee with copies of any evaluation instruments and/or studies they had conducted on their EO programs. Issues Affecting the Evaluation of MRSEC Education and Outreach Programs The committee received evaluation information from 13 MRSECs. Some of these evaluations were of separately funded programs. Of the remainder, the majority of the evaluations was of REU and RET programs. These evaluations focused primarily on logistics and participant satisfaction with the program. From these evaluations and the data described previously, the committee observed that: EO programs span a broad range of programs that serve many different audiences and, with the possible exceptions of REU and RET, are specialized to fit local situations. While this range of activities is encouraged by the NSF, each MRSEC has to manage multiple, often very different activities. Many MRSEC EO activities are leveraged by other programs, making it difficult to identify what can be attributed to the MRSEC. The data available are not sufficient for thoroughly evaluating MRSEC EO impact. The evaluations received by the committee rely almost entirely

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward curriculum development, with research playing a very limited role, if any. Although the RET program succeeds in involving women and underrepresented minorities, the committee is very concerned that there are Research Experiences for Teachers programs that do not focus on research. K-12 and the Public Few of the programs for K-12 and the public are evaluated at the same level at which the REU and RET programs are evaluated, so it is difficult to evaluate whether goals for these programs are being met. Well-evaluated programs often conduct those evaluations and reviews under the auspices of supplemental funding from other sources. The committee saw many examples of innovative programs that were enthusiastically received and executed; however, impacts of these programs beyond generating enthusiasm cannot be determined. MRSECs EO programs for K-12 and the public are highly responsive to local needs and interests. Many programs are driven by individual researchers who donate their time and effort. In many cases, researcher participation is facilitated by the EO coordinator, who handles logistics and organization. The ability to address local needs is a positive outcome of the flexibility allowed by the MRSEC program. Broadening Participation Although the MRSEC program as a whole is making strides in increasing the involvement of women at all levels, there is considerable variation among MRSECs. Few MRSECs attract sizable numbers of underrepresented minorities, in part because of the overall small numbers and the competition among institutions. The PREM program (see Box 4.3) is too new to evaluate, but long-term programs such as this have much higher potential for impact than is possible for isolated activities such as “Women in Science” days. The shifting national demographics demand that the materials science and engineering community increase efforts to broaden participation. There has been no attempt at a MRSEC-wide effort in this area, but such a strategy may be worth pursuing. Evaluating the Appropriateness of the Goals The second part of the committee’s task was to evaluate whether the EO goals are appropriate. The impact, or potential impact, of the programs was the most important consideration, with a second consideration being whether there were alternative programs with similar goals that might be more efficacious. Finally, the

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward committee evaluated the programs to determine whether the MRSEC provided any unique aspects that would not be duplicated by the same program run outside the MRSEC. REU Involving undergraduates in research continues to be an integral part of the NSF portfolio.2 The widespread involvement of undergraduates in research has generated a significant research base that addresses the impact of undergraduate research experiences (including, but not limited to REU).3-5 This research concludes: Undergraduate research experiences help students clarify their career goals, including whether they want to continue STEM study, the specific type of subdiscipline in which they choose to continue, and what graduate school they will attend; Undergraduate research experiences provide an apprenticeship in which students learn to “think and work like scientists” alongside working scientists. In particular, students appreciate how science is done, gaining a perspective often ignored in textbooks, and they learn to work independently and to rely on their own judgment; Students learn specific technical skills; and Most students experience personal gains, including increased confidence in their ability to be successful in STEM fields. There is ample evidence that involving undergraduates in research is positive and has a great impact on the participants, including the mentors. Researchers are overwhelmingly positive about the program and their participation as mentors. Because this type of activity is so widespread, there are a number of assessment 2 REU evaluation instruments are available from the MRSEC Web site (http://www.mrsec.org/links/). 3 See A.-B. Hunter, S. Laursen, and E. Seymour, “Becoming a Scientist: The Role of Undergraduate Research in Students’ Cognitive Personal and Professional Development,” Science Education, 91 (1): 36-74, January 2007. 4 See Susan H. Russell, “Evaluation of NSF Support for Undergraduate Research Opportunities,” SRI International, May 2006, Menlo Park, Calif.: SRI International, 2006. See http://www.sri.com:8000/policy/csted/reports/university/documents/URO%20FollowupSurveyReport%20for%20WebApr%2028%2006.pdf. 5 See E. Seymour, A.-B. Hunter, S. Laursen, and T. DeAntoni, “Establishing the Benefits of Research Experiences for Undergraduates in the Sciences: First Findings from a Three-Year Study,” Science Education 88: 493-534, 2004.

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward tools, results, and best practices that are shared at disciplinary and REU-specific conferences. REUs are especially appropriate for MRSECs because they offer undergraduates unique experiences owing to the interdisciplinary environment. REU programs in materials science are especially valuable to students at institutions without undergraduate materials science programs, as they open the door to graduate MSE study. The REU program is one of the areas in which MRSECs are attracting a diverse group of students, making it an important component in building the scientific and technical workforce. RET The RET program is newer than the REU program, so there is commensurately less information about its impact.6 Some preliminary conclusions can be drawn from the published literature (which comes from MRSEC and non-MRSEC RET programs),7-9 with the caveat that the studies are limited in number and scope. The primary impact of research experience on teachers is in improving their understanding of how science is done, their knowledge of current science, their awareness of the types of people who are and who become scientists, their awareness of STEM career opportunities, and in increasing their willingness to take on leadership roles. Constraints on teachers (time, standardized testing, and student capability) make it difficult to bring content from their research into the classroom. The majority of teachers focus on translating their understanding of scientific process rather than specific content to their students. Teachers who have a research experience exhibit an increased use of inquiry-based and problem-solving techniques with students, heightened emphasis on scientific process (working in groups, using graphs and charts), expect more students to design their own experiments, have more 6 RET Network Web site, http://www.retnetwork.org/evaluation.htm. 7 Carol S.C. Johnston, Translating the RET Experience to the Classroom, Redwood City, Calif.: Conference on Assessing, Determining, and Measuring the Impacts of the Research, 2003. 8 J. Dubner, S.C. Silverstein, N. Carey, J. Frechtling, T. Busch-Johnsen, J. Han, G. Ordway, N. Hutchinson, J. Lanza, J. Winter, J. Miller, P. Ohme, J. Rayford, K. Sloane-Weisbaum, K. Storm, and E. Zounar, “Evaluating Science Research Experience for Teachers Programs and Their Effects on Student Interest and Academic Performance: A Preliminary Report of an Ongoing Collaborative Study by Eight Programs,” Journal of Materials Education 23: 57-69 (2001). 9 Kevin Dilley, “How Do You Measure RET Success?” Redwood City, Calif.: Conference on Assessing, Determining, and Measuring the Impacts of the Research, 2003.

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward students involved in science fair projects and science clubs, and talk more to their students about STEM careers. The most comprehensive of the published studies shows an increase in students’ content knowledge with such teachers as measured against comparison classes on standardized tests; however, few studies have addressed this important impact. Many programs report that gains (regardless of type) come only after sustained teacher participation, which may include multiple summer RET experiences or a program that continues throughout the school year. Changes in teaching practice and/or student content knowledge may also take a year or two after the RET experience to be evident. Although the preliminary results indicate that RET has potential for high impact on student learning and future professional interests, the committee has two concerns about its role in the MRSEC program. The literature indicates that the most-transferred elements of the teacher research experience are the process skills derived from actually doing research. A number of MRSEC programs focus entirely or primarily on curriculum development, without a significant research component. The RET is not intended to be a curriculum-development program. NSF supports curriculum development through separate programs that require peer-reviewed proposals with formal evaluation and dissemination plans. In view of the emphasis on standardized testing in K-12, the committee is concerned that RET-based curriculum-development programs may have very limited impact. A second concern is the lack of evidence as to how involving teachers in research ultimately affects their students. Although the preliminary data suggest that increased student learning or even improved attitudes toward mathematics and science should result, the majority of MRSEC evaluations focus on logistics and self-reported satisfaction level. It is important to establish the impact of the MRSEC RET and especially whether unique outcomes result from an RET in a MRSEC compared with outcomes in other research fields. It is impossible to judge the value of the RET program within the MRSEC portfolio without an accurate representation of the benefits. The resources currently invested in the RET program might have more impact if focused on other types of professional development activities for teachers. K-12 and the Public The range of education and outreach programs targeted to K-12 students and the public is extremely broad. With a few notable exceptions, the evaluation of these programs is minimal, making it impossible to judge the efficacy of each

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward program. Many of the programs for these audiences are leveraged heavily by other funding sources, making it difficult to determine whether the MRSEC involvement has any impact. There are many convincing arguments for why MRSECs should be involved in K-12 and public outreach. Getting children interested in science early and maintaining that interest is critical to producing future scientists and a scientifically literate citizenry. Many students never hear about “materials science and engineering” in K-12, which may decrease the likelihood of their pursuing MSE study in college or even of appreciating the contributions that materials science and engineering make to their quality of life. Most programs targeted to K-12 students and the public are highly responsive to local needs, which is important; however, some MRSEC participants felt that they were downgraded in reviews for not consistently producing new and innovative programs rather than continuing to execute a program that they know works and fulfills a recognized need. Most MRSEC participants say that they enjoy participating in EO, and researcher enthusiasm is a large driving force. The difficulty in endorsing these EO programs is the lack of evidence as to their impact relative to the time and effort required to run them. It is the committee’s impression that the broad range and large numbers of programs in this category reflect the pressure that MRSECs feel to address every possible audience. Regardless of the origin of this pressure, the result appears to be a type of EO “arms race”: each MRSEC feels compelled to outdo the others by being able to cite a broad range of programs that reach large numbers of people of all ages. The unfortunate result is an emphasis on quantity over quality. There are a few exemplary programs in this category; however, executing a large number of programs with limited impact is not as effective as implementing a smaller number of high-quality programs that have the budget and responsibility for meaningful evaluation. Preparing Future Researchers for Participation in Education and Outreach An important and potentially overlooked aspect of the MRSEC EO program is that the involvement of graduate students, undergraduate students, and postdoctoral researchers in EO programs helps prepare them for future roles as materials science researchers and educators. The broad range of activities gives them myriad opportunities for participating. EO programs help researchers learn effective ways to engage students and the public while reinforcing the importance of integrating research and education. This is especially important for graduate students and postdoctoral associates, from whom these activities will be expected in the future. While it would be interesting to investigate how the MRSEC research atmosphere influences students at these levels, data from NSF were somewhat limited. NSF

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward was able to provide data on decisions of MRSEC Ph.D.s to pursue careers in industry (see Figure 5.3 in the next chapter), which when compared with MSE overall showed little difference in outcome. The committee, unfortunately, was not able to analyze Ph.D. student choices of academia and other arenas, as well as postdoctoral choices. FINDINGS AND RECOMMENDATIONS Education and outreach have an important role in developing the scientific and technical workforce, in educating the public about scientific issues, and in broadening the participation of women and other underrepresented groups. MRSECs have a great opportunity to contribute to this mission through their EO programs. Conclusion: The MRSEC education and outreach program has impacts on the NSF mission to educate and prepare the nation’s future workforce. MRSECs provide unique opportunities for interdisciplinary research experiences that are different from those an individual student would experience in a single-investigator laboratory. MRSECs foster environments that support interactions with other programs to leverage funds and coordinate activities across campuses and disciplines. This culture leaves a vital imprint on students who work in MRSECs. MRSECs foster a mind-set of outreach and a sense of responsibility in current and future researchers. The centralized EO infrastructure that a MRSEC offers empowers researchers to engage in EO who would not have ordinarily done so. The MRSEC EO requirement facilitates the involvement of interested researchers at all levels. EO coordinators are valuable participants who develop programs, arrange logistics, and build the partnerships that make it possible for researchers to be effectively involved in EO. The MRSEC EO requirement allows faculty members to pursue their EO interests and can provide funding and infrastructure support for that pursuit. General Finding: The most significant and well-documented contribution of MRSEC EO programs is the preparation of future researchers at all levels. Research-related education and outreach activities leverage MRSEC strengths and expertise. MRSECs can provide unique opportunities for interdisciplinary research experiences that are different from those that an individual would expe-

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward rience in a single-investigator laboratory. Although broadening the participation by women and underrepresented groups remains a challenge, the greatest contributions to meeting this challenge often come from EO programs such as REU and RET. Conclusion: Although the committee’s impression is that most MRSECs are doing good to excellent jobs with their EO programs and that many of these programs have a significant impact on their audiences, the lack of data to support these assertions poses a serious problem for NSF as it seeks to make the most efficient use of its resources. NSF manages the MRSEC program from a scientific and engineering research perspective. It is nonprescriptive, with few defined limits or requirements. The lack of specificity regarding EO expectations has led to some innovative, potentially high-impact programs; however, this lack of specificity also has led many MRSECs to try to carry out some type of activity in every aspect of EO that they see their peer (competitor) centers doing. REU and RET programs are much more likely to be evaluated (in general and especially by the MRSECs), although the evaluations focus primarily on logistics and self-reported participant perceptions. The quality of evaluations on other EO components varies greatly. MRSECs are reviewed primarily on the breadth of activities and the number of participants and not on documented outcomes. General Finding: The future impact of MRSEC EO activities is threatened. The continued lack of specificity in EO expectations at the agency level has led to an emphasis on quantity over quality and innovation over impact. It is evident to the committee that there is a multiplicity of EO activities in the MRSEC program and that the lack of guidance from NSF to the MRSECs and reviewers has contributed to what appears to have become a less productive enterprise. This has produced an emphasis on quantity over quality and on doing something new for its own sake rather than choosing to implement proven strategies. General Finding: Most MRSECs feel compelled to participate in many disparate education and outreach activities. This approach often does not make optimal use of the MRSECs’ strengths, dilutes their potential impact, and in fact reduces the likelihood of determining what that impact is. There is a perception that the demands of the EO program have grown significantly since the original inception of the MRSEC program. While the solicitations for the program show most growth in demands, the broad portfolio of activities, even in the smallest MRSECs, suggests that MRSEC resources are being spread too thinly and that the impact of those resources is being diminished. This perception should not be taken to suggest that the community does not

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward value EO. Though the tight coupling of resources to support EO programs makes it difficult for the committee to draw explicit conclusions about the appropriateness of the level of researcher involvement, the overwhelming majority of MRSEC participants expressed a belief that EO is important and indicated that they enthusiastically participate in EO activities. Nevertheless, there is a strong belief among the MRSEC participants, present and prospective, that the selection process rewards quantity over quality and innovation over impact. Two examples were most often mentioned by these individuals: The belief that a MRSEC must reach all audiences, including K-12, undergraduate and graduate students, and the public; and The belief that continuing an existing, successful program is received less favorably than proposing something new. The emphasis on breadth has led to evaluations that consist primarily of counting numbers of attendees, because the programs are so diffuse that more meaningful evaluation is impossible without funding from other sources. Some programs focus on generic outreach that has little to do with the MRSEC focus, much less materials science and engineering. While this type of outreach is important, it does not leverage MRSEC resources. Existing MRSECs mentioned that renewal reviews value doing something new over continuing programs that have been shown to be effective. The larger question is whether MRSECs should be required to innovate in the EO component of their programs or whether the focus should be on using best practices to make an impact on their communities. Focusing MRSEC resources into a select number of programs that address the local strengths and needs makes much more sense than trying to reach all audiences. The MRSECs that are successful in reaching a variety of audiences often are those with significant external funding for EO. Recommendation: Education and outreach should continue to be part of the overall MRSEC portfolio; however, MRSECs should focus resources on programs with proven high impact that leverage each MRSEC’s unique research strengths and that can be meaningfully evaluated. The committee believes that EO is an important part of the MRSEC program but that steps can be taken to increase its effectiveness. In particular: MRSECs should focus on a limited number of activities that are aligned with MRSEC research goals, are consistent with the MRSEC size, leverage participant expertise and interest, and address local needs. Because of their documented impact, REU programs should continue to

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward be required; providing research opportunities for faculty and students at predominantly undergraduate and minority-serving institutions should be strongly encouraged. MRSECs that offer RETs should provide teachers with research experiences in materials science and engineering. The RET is not meant to be primarily a curriculum-development program. Other EO projects should be peer reviewed by materials research education experts during the MRSEC proposal/review process. The best of these projects should be funded as long as the overall MRSEC is funded. MRSECs, especially those with smaller budgets, are trying to do too much with the resources they have. This is not intended to discourage MRSECs from developing and executing EO activities; however, resources would be better directed by funding a smaller number of high-quality, research-oriented activities whose impact can be meaningfully determined. There is ample evidence that the REU program has highly desirable impacts, and MRSEC researchers generally are enthusiastic and committed about their participation in REUs. MRSECs offer unique opportunities for students to get involved in interdisciplinary research at early stages of their careers and are an important pathway to graduate study in materials science and engineering. The RET recommendation is tempered by the committee’s concern that the impact of the RET program is largely undocumented. The RET program is NSF-wide, so the lack of data is not solely a MRSEC issue. Cooperative efforts to document the impact of the program, as have been done with the REU program, are necessary. However, validating the program is beyond the scope of what should be expected as part of a MRSEC EO component. Further, MRSEC RETs that do not focus primarily on providing research experiences for teachers are not addressing the intention of the RET program. All RET programs should focus on research. MRSECs should be encouraged to form partnerships with predominantly undergraduate and minority-serving institutions, and to extend research opportunities to faculty and students from those institutions. Participation in the PREM program has been and should continue to be encouraged. These activities are especially important in increasing the diversity of materials science and engineering. One way to accomplish this is by having MRSECs’ EO projects beyond the research-related activities discussed above evaluated separately by materials research education experts, as available. The committee believes that education expertise is more valuable than materials research expertise when evaluating these activities. Program managers would then fund the highest-ranked projects from those proposed by successful MRSECs.

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward Recommendation: In the context of the above recommendation, NSF should develop and support the MRSEC education and outreach community in sharing and facilitating ideas and resources, including best practices, for all activities. This would be especially helpful in the area of increasing the participation of underrepresented minorities. The collective impact of MRSECs in education and outreach could be enhanced by increased cooperation and coordination among the centers. Progress is being made in this direction, but more is possible. Despite the broad range of research interests, all MRSECs have common EO goals and activities, and an overall shift in emphasis from innovation to impact would make it easier for MRSECs to share best practices. This would facilitate the distribution of EO materials already developed and would decrease local reinvention of existing EO materials. In this vein, MRSECs should adopt a standardized evaluation instrument used at all sites to ensure that programs are using the established best practices. MRSECs should be encouraged to add to that evaluation; however, the adoption of a standard evaluation establishes a baseline for acceptable performance. The National Research Center Educator’s Network could be a starting point for this community; however, the meetings of EO coordinators run by NSF have the advantage of being run simultaneously with MRSEC directors’ meetings, which keeps directors informed about EO issues. EO coordinator meetings should be held annually, and the NSF MRSECs should establish an EO coordinators’ executive committee (similar to that of the directors) to facilitate coordination, communication, and dissemination. This group should plan the workshops (with input from the members) to address long-range strategic issues and to provide continuity. The PREM program is an excellent example of how NSF can act as a catalyst for activities that involve women and underrepresented minorities in materials science and engineering research. The committee believes that centralized activities such as PREM have a much higher probability for success than does leaving each MRSEC to its own resources. NSF should leverage the experience of its MRSECs to identify and share successful strategies in this area, not just with other MRSECs, but with the materials science and engineering community as a whole. Recommendation: NSF should provide appropriate guidance to MRSEC applicants and reviewers in order to refocus education and oureach activities and ensure the program’s effectiveness. It is evident to the committee that there is a multiplicity of EO activities in the MRSEC program and that the lack of guidance from NSF to the MRSECs and reviewers has contributed to what appears to have become a suboptimal enterprise. This should not be so. Reviewers should receive clear instructions about the role

OCR for page 118
The National Science Foundation’s Materials Research Science and Engineering Centers Program: Looking Back, Moving Forward of EO in the MRSEC: the impact of a MRSEC’s EO program should be of cardinal importance. Further, MRSEC EO programs have different objectives and therefore should not be evaluated using the same standards as those for research. NSF funds educational research under other programs, and major initiatives should be supported through those programs, with a separate review system.