Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 49
Engineering in K–12 Education: Understanding the Status and Improving the Prospects 3 The Case for K–12 Engineering Education Proponents have put forth a number of reasons for adding K–12 engineering education to the school curriculum (Box 3-1). Their arguments are similar to arguments for improving STEM education. Both cases are based on changes in the world—increasing complexity, interconnectivity, competitiveness, and technology dependence—that pose new challenges for individuals and for nations that cannot be met by continuing education as usual. We will need a steady supply of well-trained engineers, scientists, and other technical workers, as well as a technologically and scientifically literate general public, to succeed and prosper in the twenty-first century (Augustine, 2007; BSCS, 2007). In this chapter, we present a detailed discussion of the case for K–12 engineering education, focusing on various aspects of the argument and on supporting data. THE BENEFITS OF K–12 ENGINEERING EDUCATION The potential benefits to students of including engineering education in K–12 schools can be grouped into five areas: improved learning and achievement in science and mathematics; increased awareness of engineering and the work of engineers; understanding of and the ability to engage in engineering design;
OCR for page 50
Engineering in K–12 Education: Understanding the Status and Improving the Prospects BOX 3-1 Statements from Selected K–12 Engineering Education Programs The “Engineering by Design”™ Program is a model used by schools developing themes in the STEM and IT Clusters that are seeking to increase all students’ achievement in technology, science, mathematics, and English through authentic learning. ITEA http://www.iteaconnect.org/EbD/ebd.htm “The Infinity Project” is helping close the gap between the number of engineering graduates we currently produce in the United States and the large need for high-quality engineering graduates in the near future. For our next generation of college graduates to be competitive in the global world of technology, we need to take steps now to encourage more young students to pursue engineering. Southern Methodist University http://www.infinity-project.org/infinity/infinity_hist.html The “Engineering is Elementary” project aims to foster engineering and technological literacy among children. Boston Museum of Science http://www.mos.org/eie/index.php interest in pursuing engineering as a career; and increased technological literacy. Although only a small percentage of students has had an opportunity to study engineering in elementary and secondary schools in the United States, a number of curricula for teaching engineering have been developed—many of which are described in Chapter 4. Curriculum developers, cognitive scientists, and others have studied the effects of these curricula and other K–12 engineering initiatives on student learning, interests, and attitudes. Based on their research, it is possible to assess the evidence for these benefits. The remainder of this chapter provides the highlights and key findings of a commissioned review of the relevant research literature, which includes articles published in peer-reviewed journals, conference papers, program
OCR for page 51
Engineering in K–12 Education: Understanding the Status and Improving the Prospects evaluations, and unpublished documents such as dissertations (Svihla et al., unpublished). Overall, the review turned up limited evidence for many of the benefits predicted or claimed for K–12 engineering education. This does not mean that the benefits do not exist, but it does confirm that relatively few well-designed, carefully executed studies have been conducted on this subject. This issue is discussed in greater detail at the end of this chapter and in Chapter 6. Improved Learning and Achievement in Science and Mathematics One of the claims most often made about K–12 engineering education is that it improves learning and achievement in science and mathematics. This is a particularly compelling claim because, for the past two decades, many concerted efforts have been made to improve K–12 science and mathematics education in the United States. By most accounts those efforts have had relatively unimpressive results (Box 3-2). How might engineering education improve learning in science and mathematics? In theory, if students are taught science and mathematics concepts and skills while solving engineering or engineering-like problems, they will be able to grasp these concepts and learn these skills more easily and retain them better, because the engineering design approach can provide real-world context to what are otherwise very abstract concepts. Preliminary evidence supports this theory. For example, students who took courses developed by “Project Lead the Way” (PLTW) scored significantly higher on science and mathematics in the NAEP than students in a random, stratified comparison group (Bottoms and Anthony, 2005; Bottoms and Uhn, 2007). Research using a state achievement test as the basis of comparison has found more mixed results. PLTW students from schools serving a high proportion of low-income families showed less improvement in mathematics scores from grade 8 to 10 and no statistical difference in science achievement scores over that period, compared with a control group (Tran and Nathan, In press). And PLTW students attending schools serving predominantly affluent families exhibited small gains in mathematics achievement but no improvement in science achievement, compared with a control sample (Tran and Nathan, In press). Students who had taken the “Engineering Our Future New Jersey” course, which is offered in 32 elementary, middle, and high schools in the state, demonstrated significant improvements in scores on both science and
OCR for page 52
Engineering in K–12 Education: Understanding the Status and Improving the Prospects BOX 3-2 The Push to Improve K–12 Science and Mathematics Education In 1990, the Department of Education National Education Goals Panel released a report detailing necessary improvements in U.S. education. In that report, science and mathematics were the only subjects addressed specifically. Goal 5 was, “By the year 2000, United States students will be first in the world in mathematics and science achievement” (DOEd, 1989). Eleven years later, when the department published a definitive study of science and mathematics teaching in the United States, the conclusion was that little progress had been made toward reaching that goal (DOEd, 2000). In the past few years, many studies, such as Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future, have argued that improving science and mathematics education will require substantial reform (NAS et al., 2007). Many of these reports include data from the National Assessment of Educational Progress (NAEP) and two ongoing international comparative assessments, the Trends in International Mathematics and Science Study (TIMSS) and the Programme for International Student Assessment (PISA), to support the contention that U.S. K–12 students, particularly secondary students, simply do not measure up. Although TIMSS and PISA data are often used as indicators, some have argued that most interpretations of these data overstate the U.S. achievement problem, in part because they do not account for differences in the educational systems of the participating countries (Lowell and Salzman, 2007). In 2007, the Department of Education published a review of all federally funded programs with a math or science education focus, looking at their effectiveness and at ways to integrate and coordinate them. The report focused on 115 programs that it considered to have the best evaluations and concluded that there was very little hard evidence as to which programs were effective and which were not (DOEd, 2007). mathematics achievement tests1 (Hotaling et al., 2007). Statistically significant gains in science and mathematics scores have also been reported by the 1 In this study, the results were not disaggregated, and no measure of variance was provided. Thus we cannot know if the gains were uniform or if some subgroups were more or less impacted.
OCR for page 53
Engineering in K–12 Education: Understanding the Status and Improving the Prospects Center for Innovation in Engineering and Science Education at Stevens Institute of Technology, which has created a variety of online, problem-based K–12 engineering curricula (McKay and McGrath, 2007). Students who had participated in “Engineering is Elementary,” a program developed by the Boston Museum of Science that integrates engineering with science content for elementary students, showed improvement in a post-test measuring science and engineering knowledge (Lachapelle and Cunningham, 2007). Unfortunately, there was no control group for comparison in this study. Engineering design has been shown to encourage mathematical thinking. Akins and Burghardt (2006) studied teams of middle and high school students who applied mathematical reasoning to solve problems in a design challenge. Pre-test results were used to disaggregate students into quartiles, and all quartiles showed improvement on math and science tests. (No tests of significance were conducted, but post-test scores were 21 percent to 125 percent higher than pre-test scores.) The authors noted that the lowest scoring teams had the highest score gains, which suggests that engineering design has the potential to narrow achievement gaps; this possibility was not noted by the researchers, however. In some cases, standardized test scores were not impacted by student participation in engineering activities, but other measures, such as the ability to explain, analyze, predict, or reason about science, mathematics, or technology, demonstrate that the students had learned a great deal. For example, a program at one inner-city school involved designing remote-control vehicles. Although the scores of students who participated in the program did not show improvements on district-wide physics achievement tests, prepost measures showed that the students had a better understanding of the physics related to their vehicles (Barnett, 2005). In the “Integrated Mathematics, Science, and Technology” (IMaST) curriculum project, participating students and non-IMaST students had similar gains on state mathematics and science achievement tests, but IMaST students scored higher on TIMSS math items than students in a control group (Satchwell and Loepp, 2002). Notably, IMaST students scored higher on measures related to “process” (i.e., mathematical problem solving and science inquiry) whereas the control students scored higher on measures related to “knowing” (i.e., understanding routine mathematics operations and scientific information). A few studies have been done on the potential of K–12 engineering to differentially affect math and science achievement among girls and underrepresented minorities. In a middle-school study of modules in which engi-
OCR for page 54
Engineering in K–12 Education: Understanding the Status and Improving the Prospects neering design and science were integrated, pre-post data showed that the achievement gap for African American and Latino/a students was narrowed, but the achievement gap for girls was increased (Cantrel et al., 2006). It is not clear in this case whether the students engaged in a truly iterative design process, which has been shown to encourage science learning for girls and students from families of low socio-economic status (SES) (Kolodner et al., 2003). Barnett (2005) reported on a study of inner-city, low SES, predominantly ethnic-minority high school students that included a significant population of English language learners and many students with disabilities. All of these students had participated in a project that involved designing remotely operated vehicles. Pre-post data revealed that, overall, the students’ understanding of physics had improved. However, the improvement did not translate to higher scores on a district-wide final exam in physics. So-called challenge-based environments can mimic design or motivate students to solve problems in order to learn engineering, science, and mathematics content. In a three-year study of this approach, “legacy cycles,” Klein and Sherwood (2005) found that students in the experimental group had statistically larger gains in measures of relevant science knowledge and concepts. Although most of the modules did not involve design, they did require problem solving in the context of engineering and had many design elements. The researchers argue that design challenges embedded in science activities increase the likelihood that students will explore variables rather than stopping their inquiries as soon as the design criteria have been satisfied. The “Math out of the Box” program uses a modified legacy cycle in which engineering provides a context for learning applied mathematics (Diaz and King, 2007). This program has been implemented in several schools; the ones that have continued to use it have found that achievement scores in mathematics have risen, particularly for low-SES and African American students. The schools that discontinued the program found that mathematics scores fell. Qualitative research in the learning sciences provides some insights into how and why science and mathematics learning may be impacted by participation in engineering activities, particularly design activities. Fortus et al. (2004) recorded significant increases in science knowledge among ninth graders engaged in the “Designed-Based Science” curriculum. The researchers suggest that this effect can be explained in part by students’ personal ownership of science content as compared with consensus-driven ownership in other forms of inquiry. Students using this curriculum were also able to transfer their understanding of a concept from the original context to a different context (Fortus et al., 2005), which the researchers
OCR for page 55
Engineering in K–12 Education: Understanding the Status and Improving the Prospects attribute to the way the design activity is structured to support learning for understanding in the context of solving a problem. Roth (2001) suggests that design activities, which present distributed representations of ideas, can stimulate discussions about science concepts. Ideas represented through design can then be inspected and tested. Penner et al. (1998) explored how the design by elementary students of a physical model of an elbow can support science and mathematics learning related to the mechanics of motion. The success of the project depended on students having multiple opportunities to engage in and discuss their design experiences, teachers’ use of analogies, and sense-making based on data collection and interpretation. Redesign gives students a chance to explore connections between science and design, to test their ideas, and to decide how to correct their designs and then adjust the corresponding understanding of the relevant scientific principle or concept (Sadler et al., 2000). In summary, the available evidence suggests that under certain circumstances, engineering education can boost learning and achievement in science and mathematics. These effects may be more significant for certain populations, particularly underrepresented minority students. However, the positive effects are not universal and research has not clearly established the causal mechanism(s) to explain such benefits when they occur. Increased Awareness of Engineering and the Work of Engineers This goal, improving students’ awareness of engineering and the work of engineers, can be of great benefit to a society, because engineering is central to technology development, and technology influences the well-being of everyone. Conversely, a lack of awareness of engineering and misconceptions or ignorance about what engineers do can be detrimental to a society. On a practical level, young people who believe engineers drive trains or repair car engines or who have negative stereotypes of the profession are unlikely ever to consider studying engineering or pursuing it as a career. If enough youngsters feel this way, it may become increasingly difficult to attract and retain a technically proficient workforce. Generally, individuals who do not have a basic idea of what engineers do are unlikely to appreciate how engineering and science contribute to economic development, quality of life, national security, and health care; such awareness is one aspect of technological literacy (NAE and NRC, 2006). The engineering community, including engineering professional societies, schools of engineering, and firms that depend heavily on engineering
OCR for page 56
Engineering in K–12 Education: Understanding the Status and Improving the Prospects talent, have spent hundreds of millions of dollars annually on initiatives to raise the level of the public understanding of engineering (NAE, 2002), for the most part unsuccessfully. For example, researchers have found that K–12 teachers and students generally have a poor understanding of what engineers do (Cunningham and Knight, 2004; Cunningham et al., 2005; Oware et al., 2007). Survey data suggest that many adults in the United States believe that engineers, as compared with scientists, are not as responsive to societal and community concerns and are not as important in saving lives (Harris Interactive, 2004). This widespread misconception reveals a lack of awareness of the many ways engineering has dramatically improved the human condition (e.g., www.greatachievements.org). Teens and adults strongly associate engineering with skills in mathematics and science, according to recent online polling, but much more rarely with creativity, rewarding work, or a positive effect on the world (NAE, 2008). Findings like these have prompted advocates of K–12 engineering education to argue for the importance of young people having opportunities to learn about engineers, engineering, and technology. Research has shown that participation in engineering education activities can provide those opportunities. For example, assessments showed that students who participated in the “Engineering Our Future New Jersey” program were able to name significantly more types of engineers and to describe types of engineering activities (Hirsch et al., 2005). These students were also able to recognize technology and the work of engineers (Hotaling et al., 2007). Teachers, too, may be more aware of engineering career options after leading engineering design activities with students (McGrath et al., 2008). Pre-post tests found that young children who took part in the “Engineering is Elementary” program had a significantly broader conception of what technology is and were able to identify activities undertaken by engineers (Lachapelle and Cunningham, 2007). According to a study by graduate teaching fellows in K–12 education funded by the National Science Foundation, such changes in students’ awareness of engineers and engineering can be sustained over time (Lyons and Thompson, 2006). Understanding of and the Ability to Engage in Engineering Design The iterative, open-ended, problem-solving method known as engineering design is the central activity of engineers. For this reason, a good deal of K–12, as well as post-secondary, engineering education is spent on
OCR for page 57
Engineering in K–12 Education: Understanding the Status and Improving the Prospects developing students’ understanding and capabilities in this area. In addition, as was mentioned above, design activities provide a real-world focus for abstract concepts, which may have a positive impact on learning not only in engineering, but also in other subjects, such as mathematics and science. In this section, we consider the evidence related to how well students learn to understand and engage in engineering design. Data from a number of studies suggest that engineering design as practiced by engineers is neither quickly learned by students nor easily taught by teachers. Issues common to novice design, such as using trial-and-error methods (rather than a systematic approach) and spending too much time on defining the problem, have been well documented (e.g., Hill and Smith, 1998; Ressler and Ressler, 2004). Unless the teacher explicitly encourages a systematic approach, the design process can be overwhelmed by student excitement about hands-on activities (Seiler et al., 2001). Specific concepts integral to engineering design also pose challenges to students. For example, in a project in which undergraduate engineering and education students developed design activities for students in the seventh through twelfth grades, Bergin et al. (2007) found that the K–12 students had difficulty understanding the idea of constraints. Penner et al. (1997) found that elementary students struggled to use modeling in a way that reflects engineering practice. In this study, student pairs were asked to design a functional model of an elbow. At first, the children tended to see models as small versions of the thing itself, and their first design iterations copied the form of an elbow but could not perform the functions of an elbow. After some discussion, it was clear that students had not isolated the motion of the elbow but had inferred a great range of motion based on the pivot of the shoulder. After experimenting with real elbow movements, they began a second iteration of modeling. This time the models incorporated constraints but also included nonfunctional but physically similar details, such as a representation of veins. Interest in Pursuing Engineering as a Career As many reports and commentators have noted, the economic competitiveness of the United States depends in large part on our ability to attract, train, and retain a large corps of highly qualified, creative engineers in a variety of fields (e.g., NAS et al., 2007). Unfortunately, many students who are capable of becoming engineers never even enter the educational pipeline leading to an engineering career because they either do not understand what
OCR for page 58
Engineering in K–12 Education: Understanding the Status and Improving the Prospects engineers do or they believe that they do not have the necessary aptitude or interests to become engineers. This is particularly common for females and students from certain minorities, who are greatly underrepresented in engineering schools and in engineering practice (see Table 2-1) (Chang, 2002). Up to now, the primary strategy for ensuring that the engineering pipeline is filled has been to insist that high school graduates have a good grounding in science and mathematics. Thus students are not exposed to engineering until they enter college, frequently not until their junior year. K–12 engineering programs offer a different strategy. By introducing students to engineering in K–12 programs—in theory, at least—more of them, from a wider variety of backgrounds, will be attracted to the field. Although keeping the engineering pipeline flowing is an explicit goal of only a handful of the curricula we examined, the idea that exposure to engineering thinking, particularly design experiences, will attract more students to the pursuit of engineering or technology-related studies and careers seems intuitively sound. In this section, we examine the evidence for how K–12 engineering education affects student interest in engineering and related factors, such as school attendance, retention, and persistence. Research has shown that students who choose to participate in engineering-related activities and coursework may become more interested in pursuing careers in engineering. For instance, both girls and boys who attended the “Discover Engineering” summer camp at Ryerson University in Canada reported an increased interest in engineering as a career (Anderson and Northwood, 2002). A follow-up study showed that approximately one-third of camp participants actually went on to pursue engineering degrees (Anderson et al., 2005). However, without a comparison group we cannot know if this group of students was representative of the general population. Not all students respond the same way to educational interventions. Thus it is important to determine how specific groups tend to respond. For instance, in an engineering enrichment program for gifted students, participants completed small design projects as part of reaching a larger design goal (Bayles et al., 2007). Following the experience, 11 percent of students indicated that they felt less confident about their ability to become engineers, and 41 percent said they felt more confident. In a survey of students entering the “Discover Engineering” outreach program, Anderson and Gilbride (2003a) found that boys were significantly more interested than girls in pursuing engineering careers. Boys who claimed to have more knowledge of engineering were more interested than less-knowledgeable boys, but girls who claimed to be more knowledgeable were not more interested than their
OCR for page 59
Engineering in K–12 Education: Understanding the Status and Improving the Prospects less knowledgeable peers. An assessment of student interest in engineering following participation in the program showed an increase in interest among both boys and girls, but girls’ interest did not rise to parity (Anderson and Gilbride, 2003b). In a study of a different program, both boys and girls reported gains in confidence about engineering as a career after participating in engineering design activities, and girls and boys had equal scores (Zarske et al., 2007). An investigation of why the two studies produced different results could be potentially informative. Some evidence suggests that engineering activities have coincided with higher school attendance, perhaps a reflection of increased interest. Barnett (2005) reported that attendance increased for a group of inner-city high school science students (largely from low-SES ethnic minorities) who were randomly assigned to classes in which the major focus was on engineering design projects, compared to their peers who were taught the standard science curriculum. Studies have also been done on retention levels and persistence in engineering, primarily for students already interested in engineering. Most high school students who took an introductory engineering-design course based on a course for first-year college students, for example, went on to pursue engineering degrees in college (Bayles, 2005). Students who take courses from PLTW, a four-year college preparatory program, tend to take more advanced science and math courses and to consider them important to their future (Bottoms and Anthony, 2005; Bottoms and Uhn, 2007). Most PLTW students say they plan to attend college (Walcerz, 2007), although this cannot definitively be attributed to participation in PLTW because this is a self-selected group. The same students reported feeling confident about their career choices (mostly engineering and technology) because of the courses they took in high school. In addition, participation in PLTW has been shown to reduce attrition rates in college engineering programs and to increase the percentage of degrees attained (Taylor et al., 2006). These findings are positive, but the students who choose to take such courses cannot be considered a general population. In a study of the long-term impact of a two-week engineering camp for middle schoolers, participating students were likelier than a control group to take STEM courses (Hubelbank et al., 2007). This finding is significant because both groups had applied to attend the camp, and the participants were selected by lottery. The camp experience did not affect students’ interest in college-level engineering, however. Students in the control and experimental groups were equally likely to pursue engineering degrees.
OCR for page 60
Engineering in K–12 Education: Understanding the Status and Improving the Prospects Participation in K–12 engineering education programs may correlate with an increase in applications to engineering colleges. Zarske et al. (2007) found this to be true for a K–12 program in Colorado. However, although the number of applications increased, many applicants had not completed the coursework necessary for acceptance into the college program. One way of supporting these students is to provide a bridge program. Anderson-Rowland et al. (1999) demonstrated a significantly higher level of retention for students who attended the Summer Bridging Program (SBP) at Arizona State University, a program for entering minority freshmen. However, the effects of SBP were difficult to determine because participants were also required to enroll in an Academic Success Seminar during their freshman year. Increased Technological Literacy Many have argued that K–12 engineering classes improve students’ technological literacy. Although this argument might not have been compelling 20 years ago, there is a growing appreciation today of the importance of technological literacy to individuals and to society as a whole. As defined in Technically Speaking: Why All Americans Need to Know More About Technology, “technological literacy combines basic knowledge about the various technologies in our world with the ability to think critically about technology and to make well-informed decisions about technological issues” (NAE and NRC, 2006). A technologically literate person understands the essential characteristics of technology and how it influences society and the factors that shape technology, including engineering. Concepts central to engineering, such as systems, trade-offs, and intended and unintended consequences, provide a foundation for making informed decisions in a technologically dependent society like ours. In Technically Speaking, the case for technological literacy is spelled out in detail. A technologically literate person can make informed decisions about his or her use of personal technologies, for example. Technologically literate citizens can be effective participants in decision-making processes involving technology—for instance, whether a city should support the building of a coal-fired power plant. In a society with a growing number of jobs that require technological skills and savvy, employers are more likely to find technologically competent workers if the general population is technologically literate.
OCR for page 61
Engineering in K–12 Education: Understanding the Status and Improving the Prospects In K–12 schools, technological literacy is largely the purview of technology education teachers. In the United States, 25,000 to 35,000 such teachers work in K–12 schools, mostly middle schools and high schools (Dugger, 2007). In 2000, the International Technology Education Association (ITEA) published Standards for Technological Literacy: Content for the Study of Technology, which accelerated an ongoing shift in the field of technology education away from its beginnings in industrial arts toward an emphasis on a broad understanding of the concept of technology. The standards in the ITEA document, developed with input from the National Academy of Engineering and National Research Council, include benchmarks related to engineering design (Box 3-3). ITEA and others have also produced curricular materials (e.g., “Engineering by Design,” “Engineering is Elementary”) that attempt to meet the learning goals spelled out in the standards. Research shows that many Americans—children and adults—have a narrow, sometimes incorrect, view of technology. In one study, students BOX 3-3 Selected Engineering-Design-Related Benchmarks, by Grade Band To comprehend engineering design, students should learn that: The engineering design process includes identifying a problem, looking for ideas, developing solutions, and sharing solutions with others. (Grades K–2) Models are used to communicate and test design ideas and processes. (Grades 3–5) Design involves a series of steps, which can be performed in different sequences and repeated as necessary. (Grades 6–8) Engineering design is influenced by personal characteristics, such as creativity, resourcefulness, and the ability to visualize and think abstractly. (Grades 9–12) SOURCE: ITEA, 2000.
OCR for page 62
Engineering in K–12 Education: Understanding the Status and Improving the Prospects in lower elementary grades associated technology mostly with things that require electricity (they conflated technology with lightning) (Cunningham, et al., 2005). Only a few children recognized bridges and bandages, for example, as technologies. First graders identified parrots as a technology nearly as often as they did cups. Surveys of adults have shown that the vast majority associate technology primarily with computers (ITEA, 2004). Several studies have shown that students who have been exposed to engineering education have a broader conception of technology and have corrected some misconceptions (Hotaling et al., 2007; Lachapelle and Cunningham, 2007). Being able to recognize technology is a basic prerequisite for technological literacy. The committee did not find any published research that explicitly ties K–12 engineering education to improvements in other aspects of technological literacy. One reason may be that technological literacy, unlike science literacy, is a relatively new idea in education. In addition, there are significant challenges associated with the development of assessments of technological literacy. In an extended discussion of the latter problem, Tech Tally: Approaches to Assessing Technological Literacy (NAE and NRC, 2006) pointed out that the “capabilities” dimension of technological literacy may be especially difficult to measure. In that report the study committee reviewed 28 existing assessment instruments for measuring some aspect of technological literacy, even if they were not designed for that purpose. The committee found that none of these instruments was completely adequate for measuring technological literacy and that only two explicitly targeted engineering learning; one was developed for students in “The Infinity Project,” and the other was an achievement test for fifth, eighth, and tenth graders in Massachusetts. Interest on the national level in the technological literacy of K–12 students and improvements in measuring instruments, such as assessments, may increase in coming years. For example, when a revised version of the science portion of NAEP is administered for the first time in 2009, 10 percent of test items will focus on technological design (NAGB, 2008a). In addition, the National Assessment Governing Board, which oversees NAEP, has recently funded a feasibility study for an assessment of technological literacy (NAGB, 2008b). If the study, which runs until 2012, finds that technological literacy can be validly and reliably measured, NAGB may add an assessment of technological literacy to its portfolio of tests.
OCR for page 63
Engineering in K–12 Education: Understanding the Status and Improving the Prospects LIMITATIONS OF THE DATA Besides the relatively small number of studies on the impacts of teaching engineering concepts and skills to K–12 students, our review of the literature revealed a number of weaknesses in the methodologies used in some studies. Several of these are highlighted below in hopes that they will be addressed in future research on the impacts of this emerging area of education. An overarching concern with the data is that assessments of whether these well-intentioned initiatives achieve their desired goals frequently appear to be an afterthought. Assessments require advanced planning and viable pre-tests. Although pre- and post-tests cannot replace longitudinal data, they do indicate changes over time. Follow-up surveys can be used to determine the persistence of these changes. Another problem is that the data are not “generalizable.” For example, students who participate in engineering camps, clubs, and courses have chosen to do so. Thus the findings about the effectiveness of these activities cannot be generalized to students who do not choose to participate in these programs. This issue involves not only methodology. Because the findings do not provide information about the specific impacts on women and underrepresented minorities or on students who are not initially interested in learning about engineering, these assessments tell us little or nothing about the effectiveness of engineering education on general student populations. This can be a serious problem, because a goal of many of these programs is to increase the number of women and underrepresented minorities in engineering classes and ultimately in engineering practice. When data on K–12 engineering education initiatives are collected, they often indicate only if participants enjoyed the program and include self-reported changes. It is known that participants in studies sometimes report positive results simply because they are in a study, the so-called Hawthorne effect (Landsberger, 1958). This methodological weakness could be addressed by measuring learning on pre-and post-tests. Most of the studies we reviewed did not assess the impact of engineering education on student subgroups. The problem arises because in presenting data, it is critical to provide measures of central tendency and distribution. For example, the same average may be found for tightly clustered data, indicating that most respondents have similar scores, or for widely distributed data, indicating that approximately equal numbers of people had scores above and below the mean. The critical factor is the meaning of the spread. For instance, did minority students or students who most need to learn fall below the mean? The simple solution is to disaggregate data. This is only
OCR for page 64
Engineering in K–12 Education: Understanding the Status and Improving the Prospects viable, of course, if the number of subgroup members in the study is sufficient to permit statistically valid comparisons. These problems are not limited to studies of engineering education. In fact, definitive data about the impacts of educational interventions in most subjects are hard to come by. Even for the best-studied areas, such as reading and mathematics, little convincing evidence is available about the effectiveness of teaching approaches. In 2007, for example, the U.S. Department of Education published a review of all federally funded programs with a math or science education focus with the intent of determining their effectiveness as a basis for integrating and coordinating them. The report focused on 115 programs, 24 of them K–12 programs for which the “best” evaluations were done (DOEd, 2007). [D]espite decades of significant federal investment in science and math education, there is a general dearth of evidence of effective practices and activities in STEM education. Even the 10 well-designed studies [that the review identified] would require replication and validation to be used as the basis for decisions about education policy or classroom practice. In short, the lack of a strong evidence base for the benefits of K–12 engineering education is consistent with the situation for much educational research in the STEM arena. This is another reason, if any were needed, for those who promote K–12 engineering education to pursue empirical, methodologically sound impact studies. REFERENCES Akins, L., and D. Burghardt. 2006. Work in Progress: Improving K–12 Mathematics Understanding with Engineering Design Projects. Pp. 13–14 in Frontiers in Education Conference, 36th Annual. New York: Institute of Electrical and Electronics Engineers. Anderson, L.S., and K.A. Gilbride. 2003a. Bringing Engineering to K–12 Classrooms: Initiatives and Results. Proceedings of the 2003 American Society for Engineering Education Annual Conference, Nashville, Tennessee, June 22–25, 2003. Available online at http://soa.asee.org/paper/conference/paper-view.cfm?id=18386 (accessed January 16, 2009). Anderson, L.S., and K.A. Gilbride. 2003b. Pre-university Outreach: Encouraging Students to Consider Engineering Careers. Paper presented at 3rd Global Congress on Engineering Education, Glasgow, Scotland, June 30–July 5, 2002. Available online at http://www.discoverengineering.ryerson.ca/publications/2002UICEE.pdf (accessed January 16, 2009). Anderson, L.S., and D.O. Northwood. 2002. Recruitment and Retention Strategies to Increase Diversity in Engineering. Paper presented at the International Conference on Engineering Education ICEE, Manchester, U.K., August 18–21, 2002. Available online at http://www.ineer.org/Events/ICEE2002/Proceedings/Papers/Index/O065-O070/O069.pdf (accessed January 14, 2009).
OCR for page 65
Engineering in K–12 Education: Understanding the Status and Improving the Prospects Anderson, L.S., K.A. Gilbride, and N. Bajaj. 2005. Discover Engineering Follow-up Surveys: Assessment/Evaluation of Recruitment Programs. Paper presented at the WEPAN/NAMEPA Joint Conference, Las Vegas, Nevada, April 11, 2005. Available online at http://www.x-cd.com/wepan05/24.pdf (accessed January 13, 2009). Anderson-Rowland, M.R., S.L. Blaisdell, S.L. Fletcher, P.A. Fussell, M.A. McCartney, M.A. Reyes, et al. 1999. A collaborative effort to recruit and retain underrepresented engineering students. Journal of Women and Minorities in Science and Engineering 5(4): 323–349. Augustine, N.R. 2007. Is America Falling off the Flat Earth? Washington, D.C.: The National Academies Press. Also available online at http://www.nap.edu/catalog.php?record_id=12021 (accessed May 2, 2008). Barnett, M. 2005. Engaging inner city students in learning through designing remote operated vehicles. Journal of Science Education and Technology 14(1): 87–100. Bayles, T.M. 2005. Project-Based Learning Design Projects for Introduction to Engineering Design Courses. Paper presented at the American Society for Engineering Education Annual Conference and Exposition Portland, Oregon, June 12–15, 2005. Available online at http://soa.asee.org/paper/conference/paper-view.cfm?id=21005 (accessed January 14, 2009). Bayles, T.M., J. Rice, G. Russ, and T. Monterastelli. 2007. High School Outreach: A Look at Renewable Energy. Paper presented at the American Society for Engineering Education, Honolulu, Hawaii, June 27, 2007. Available online at http://soa.asee.org/paper/conference/paper-view.cfm?id=4795 (accessed January 14, 2009). Bergin, D., S.K. Khanna, and J. Lynch. 2007. Infusing design into the G7–12 curriculum: two example cases. International Journal of Engineering Education 23(1): 43–49. Bottoms, G., and K. Anthony. 2005. Project Lead the Way: A Pre-Engineering Curriculum That Works. Southern Regional Education Board. Available online at http://www.sreb.org/programs/hstw/publications/briefs/05V08_Research_PLTW.pdf (accessed May 9, 2008). Bottoms, G., and J. Uhn. 2007. Project Lead the Way Works: A New Type of Career and Technical Program. Southern Educational Review Board. Available online at http://www.sreb.org/programs/hstw/publications/2007pubs/07V29_Research_Brief_PLTW.pdf (accessed January 15, 2009). BSCS (Biological Sciences Curriculum Study). 2007. A Decade of Action: Sustaining Global Competitiveness. Colorado Springs, Colo.: BSCS. Cantrell, P., G. Pekcan, A. Itani, and N. Velasquez-Bryant. 2006. The effects of engineering modules on student learning in middle school science classrooms. Journal of Engineering Education 95(4): 301–310. Chang, J.C. 2002. Women and Minorities in the Science, Mathematics, and Engineering Pipeline. ERIC Digest. Available online at http://eric.ed.gov/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/1a/52/ec.pdf (accessed May 13, 2008). Cunningham, C., and M. Knight. 2004. Draw an Engineer Test: Development of a Tool to Investigate Students’ Ideas about Engineers and Engineering. Paper presented at the American Society for Engineering Education Annual Conference and Exposition. Salt Lake City, Utah, June 20–23, 2004. Available online at http://soa.asee.org/paper/conference/paper-view.cfm?id=19444 (accessed January 16, 2009).
OCR for page 66
Engineering in K–12 Education: Understanding the Status and Improving the Prospects Cunningham, C., C. Lachapelle, and A. Lindgren-Streicher. 2005. Assessing Elementary School Students’ Conceptions of Engineering and Technology. Paper presented at the American Society for Engineering Education Annual Conference and Exposition, Portland, Oregon, June 12–15, 2005. Available online at http://soa.asee.org/paper/conference/paper-view.cfm?id=21451 (accessed January 16, 2009). Diaz, D. and P. King. 2007. Adapting a Post-Secondary STEM Instructional Model to K–5 Mathematics Instruction. Paper presented at the American Society of Engineering Education Annual Conference and Exposition, Honolulu, Hawaii, June 24–27, 2007. Available online at http://www.mathoutofthebox.org/aseepaperdiaz.pdf (accessed January 15, 2009). DOEd (U.S. Department of Education). 1989. Report of the National Education Goals Panel. Washington, D.C.: DOEd. DOEd. 2000. Before It’s Too Late: A Report to the Nation from the National Commission on Mathematics and Science Teaching for the 21st Century. Washington, D.C.: DOEd. Also available online at http://www.ed.gov/inits/Math/glenn/report.pdf (accessed May 2, 2008). DOEd. 2007. Report of the Academic Competitiveness Council. Washington, D.C.: DOEd. Also available at http://www.ed.gov/about/inits/ed/competitiveness/acc-mathscience/index.html (accessed April 15, 2008). Dugger, W.E., Jr. 2007. The status of technology education in the United States: a triennial finding of the findings from the states. The Technology Teacher 67(1): 14–21. Fortus, D., R.C. Dershimer, J.S. Krajcik, R.W. Marx, and R. Mamlok-Naaman. 2004. Design-based science and student learning. Journal of Research in Science Teaching 41(10): 1081–1110. Fortus, D., J.S. Krajcik, R.C. Dershimer, R.W. Marx, and R. Mamlok-Naaman. 2005. Design-based science and real-world problem-solving. International Journal of Science Education, 27(7): 855–879. Harris Interactive. 2004. American Perspectives on Engineers and Engineering. Poll conducted for the American Association of Engineering Societies. Final report, February 13, 2004. Available online at http://www.aaes.org/harris_2004_files/frame.htm (accessed July 6, 2007). Hill, A.M., and H. Smith. 1998. Practice meets theory in technology education: a case of authentic learning in the high school setting. Journal of Technology Education 9(2): 29–45. Hirsch, L.S., H. Kimmel, R. Rockland, and J. Bloom. 2005. Implementing Pre-engineering Curricula in High School Science and Mathematics. Paper presented at the 35th Annual Frontiers in Education Conference, Indianapolis, Indiana, October 19–22, 2005. Available online at http://fie-conference.org/fie2005/index.htm (accessed January 15, 2009). Hotaling, L., B. McGrath, M. McKay, C. Shields, S. Lowes, C.M. Cunningham, et al. 2007. Engineering Our Future New Jersey. Paper presented at the American Society for Engineering Education Annual Conference and Exposition Proceedings, Honolulu, Hawaii, June 24–27, 2007. Available online at http://soa.asee.org/paper/conference/paper-view.cfm?id=4611 (accessed January 15, 2009). Hubelbank, J., C. Demetry, S.E. Nicholson, S. Blaisdell, P. Quinn, E. Rosenthal, et al. 2007. Long-Term Effects of a Middle School Engineering Outreach Program for Girls: A
OCR for page 67
Engineering in K–12 Education: Understanding the Status and Improving the Prospects Controlled Study. Paper presented at the American Society for Engineering Education, Honolulu, Hawaii, June 24–27, 2007. Available online at http://soa.asee.org/paper/conference/paper-view.cfm?id=4368 (accessed January 15, 2009). ITEA (International Technology Education Association). 2000. Standards for Technological Literacy: Content for the Study of Technology. Reston, Va.: ITEA. ITEA. 2004. The Second Installment of the ITEA/Gallup Poll and What It Reveals as to How Americans Think about Technology: A Report of the Second Survey Conducted by the Gallup Organization for the International Technology Education Association. Available online at http://www.iteaconnect.org/TAA/PDFs/GallupPoll2004.pdf (accessed December 22, 2008). Klein, S., and R.D. Sherwood. 2005. Biomedical engineering and cognitive science as the basis for secondary science curriculum development: a three year study. School Science and Mathematics 105(8): 384–401. Kolodner, J.L., P.J. Camp, D. Crismond, B. Fasse, J. Gray, and J. Holbrook. 2003. Problem-based learning meets case-based reasoning in the science classroom: putting learning-by-design into practice. Journal of the Learning Sciences 12(4): 495–547. Lachapelle, C.P., and C.M. Cunningham. 2007. Engineering is Elementary: Children’s Changing Understandings of Science and Engineering. Paper presented at the American Society for Engineering Education Annual Conference and Exposition, Honolulu, Hawaii, June 24–27, 2007. Available online at http://soa.asee.org/paper/conference/paper-view.cfm?id=3247 (accessed January 15, 2009). Landsberger, H.A. 1958. Hawthorne Revisited, Management and the Worker: Its Critics, and Developments in Human Relations in Industry. Ithaca, N.Y.: Cornell University Press. Lowell, L.B., and H. Salzman. 2007. Into the Eye of the Storm: Assessing the Evidence on Science and Engineering Education, Quality, and Workforce Demand. October 2007. Washington, D.C.: The Urban Institute. Lyons, J., and S. Thompson. 2006. Investigating the Long-Term Impact of an Engineering-Based GK–12 Program on Students’ Perceptions of Engineering. 2006 ASEE Annual Conference and Exposition: Excellence in Education, Chicago, Illinois, June 18–21, 2006. Available online at http://soa.asee.org/paper/conference/paper-view.cfm?id=2184 (accessed January 15, 2009). McGrath, B., J. Sayres, S. Lowes, and P. Lin. 2008. Underwater LEGO Robotics as the Vehicle to Engage Students in STEM: The BUILD IT Project’s First Year of Classroom Implementation. Paper presented at the fall 2008 American Society for Engineering Education Mid-Atlantic Conference, October 18, 2008, Stevens Institute of Technology, Hoboken, N.J. Available online at http://www.stevens.edu/asee/fileadmin/asee/pdf/Mc-Grath_-_final.pdf (accessed January 15, 2009). McKay, M., and B. McGrath. 2007. Real-world problem-solving using real-time data. International Journal of Engineering Education 23(1): 36–42. NAE (National Academy of Engineering). 2002. Technically Speaking: Why All Americans Need to Know More About Technology. Washington, D.C.: National Academy Press. Also available online at http://www.nap.edu/openbook.php?record_id=10250 (accessed May 7, 2008). NAE. 2008. Changing the Conversation: Messages for Improving Public Understanding of Engineering. Washington, D.C.: The National Academies Press.
OCR for page 68
Engineering in K–12 Education: Understanding the Status and Improving the Prospects NAE and NRC (National Research Council). 2006. Tech Tally: Approaches to Assessing Technological Literacy. Washington, D.C.: The National Academies Press. NAGB. (National Assessment Governing Board). 2008a. Science Framework for the 2009 National Assessment of Educational Progress. September 2008, Exhibit 18. Distribution of items by science practice and grade, p. 113. Available online at http://www.nagb.org/frameworks/science-09.pdf (accessed September 19, 2008). NAGB. 2008b. Conducting a Special Study in Technological Literacy for the 2012 National Assessment of Educational Progress. Solicitation ED-08-R-0032. Available online at https://www.fbo.gov/download/a0d/a0dd95e5669a97e35fd90eedcc609428/TECH_LIT_RFP.pdf (accessed September 19, 2008). NAS, NAE, and IOM (National Academy of Sciences, National Academy of Engineering, and Institute of Medicine). 2007. Rising Above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future. Washington, D.C.: The National Academies Press. Oware, E., B. Capobianco, and H. Diefes-Dux. 2007. Gifted Students’ Perceptions of Engineers? A Study of Students in a Summer Outreach Program. Paper presented at the American Society of Engineering Education Annual Conference and Exposition, Honolulu, Hawaii, June 24–27, 2007. Available online at http://soa.asee.org/paper/conference/paper-view.cfm?id=5401 (accessed January 15, 2009). Penner, D.E., N.D. Giles, R. Lehrer, and L. Schauble. 1997. Building functional models: designing an elbow. Journal of Research in Science Teaching 34(2): 125–143. Penner, D.E., R. Lehrer, and L. Schauble. 1998. From physical models to biomechanics: a design-based modeling approach. Journal of the Learning Sciences 7(3-4): 429–449. Ressler, S.J., and E.K. Ressler. 2004. Using a nationwide Internet-based bridge design contest as a vehicle for engineering outreach. Journal of Engineering Education 93(2): 117-128. Roth, W.M. 2001. Learning science through technological design. Journal of Research in Science Teaching 38(7): 768–790. Sadler, P.M., H.P. Coyle, and M. Schwartz. 2000. Engineering competitions in the middle school classroom: key elements in developing effective design challenges. Journal of the Learning Sciences 9(3): 299–327. Satchwell, R.E., and F.L. Loepp. 2002. Designing and implementing an integrated mathematics, science, and technology curriculum for the middle school. Journal of Industrial Teacher Education 39(3): 41–66. Seiler, G., K. Tobin, and J. Sokolic. 2001. Design, technology, and science: sites for learning, resistance, and social reproduction in urban schools. Journal of Research in Science Teaching 38(7): 746–767. Svihla, V., J. Marshall, and A.J. Petrosino. Unpublished. K–12 Engineering Education Impacts. Paper prepared for the National Academy of Engineering and National Research Council Committee on K–12 Engineering Education. Taylor, K.D., G.N. Foster, and M.M. Ratcliff. 2006. Project Lead the Way: Raising the Technical Literacy of High School Graduates. Paper presented at the 9th International Conference on Engineering Education, San Juan, Puerto Rico, June 27–July 1, 2006. Available online at http://www.icee.usm.edu/icee/conferences/icee2006/papers/3603.pdf (accessed January 16, 2009).
OCR for page 69
Engineering in K–12 Education: Understanding the Status and Improving the Prospects Tran, N., and M. Nathan. In press. An investigation of the relationship between pre-college engineering studies and student achievement in science and mathematics. Journal of Engineering Education. Tran, N., and M. Nathan. In press. Effects of pre-college engineering studies on mathematics and science achievements for high school students. International Journal of Engineering Education (Special Issue on Applications of Engineering Education Research). Walcerz, D. 2007. Report on the Third Year of Implementation of the TrueOutcomes Assessment System for Project Lead the Way. Available online at http://www.pltw.org/pdfs/AnnualReport-2007-Public-Release.pdf (accessed January 16, 2009). Zarske, M., J.L. Yowell, J.F. Sullivan, D. Knight, and D. Wiant. 2007. The TEAMS Program: A Study of a Grades 3–12 Engineering Continuum. Paper presented at the American Society of Engineering Education Annual Conference and Exposition, Honolulu, Hawaii, June 24–27, 2007. Available online at http://www.pltw.org/pdfs/AnnualReport-2007-Public-Release.pdf (accessed January 16, 2009).
OCR for page 70
Engineering in K–12 Education: Understanding the Status and Improving the Prospects This page intentionally left blank.