Tech Tally Approaches to Assessing Technological Literacy (2006) / Chapter Skim
Currently Skimming:
6 From Theory to Practice: Five Sample Cases
6 From Theory to Practice: Five Sample Cases
Pages 127-160
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 127... ...
Many of the sample cases inform one or more of the recommendations in Chapter 8. For example, Case 2, a national sample-based assessment, addresses some of the same issues designers of the National Assessment of Educational Progress, Trends in Mathematics and Science Study, and Programme for International Student Assessment may face in adapting those instruments to measuring technological literacy (Recommendations 1 and 2)
|
From page 128... ...
. And the cases related to broad populations (Case 4)
|
From page 129... ...
For instance, a statewide assessment generally falls somewhere between a national assessment and a school-based assessment in terms of the timeliness of results and the breadth and depth of knowledge covered. But the most important difference is that a statewide assessment provides an opportunity for assessors to calculate individual, subgroup, and group-level scores.
|
From page 130... ...
However, an assessment on such a large scale conducted in even a single state could demonstrate the feasibility and value of determining what students know and can do with respect to technology and could provide momentum for changes in standards, cur riculum, and teacher education across the country. Purpose In this example, the primary purpose of the statewide assessment Typically, of technological literacy is to improve teaching and learning related to statewide technology.
|
From page 131... ...
, the NRC National Science Education Standards (NRC, 1996) , and especially state-specific content standards, would be logical starting points for determining the content of the assessment.
|
From page 132... ...
If the assessment in cludes items that measure student attitudes and opinions about technol ogy or technology careers, that information could be correlated with performance data. In this way, the data could be used by K12 educators to assist with course planning and career counseling.
|
From page 133... ...
Without the threat of punitive consequences for poor outcomes, teachers may be less inclined to spend time preparing students for the assessment, and students may be less inclined to take the test seriously. A statewide assessment of technological literacy would also have resource constraints, especially today, when states are already spending considerable sums to meet the assessment and reporting requirements of 2Matrix sampling and census testing are explained in Chapter 4 in the section on Measurement Issues.
|
From page 134... ...
, it would still be difficult to convince states that are already "feeling the pinch" of NCLB to add a statewide assessment of technological literacy. Furthermore, traditional paper-and-pencil tests alone generally do not provide an adequate measure of capabilities related to technological design.
|
From page 135... ...
3For a statewide assessment, items would be based on a framework derived from rigorously developed content standards. In this example, items were derived from content specified for grades 6 through 8 in the ITEA Standards for Technological Literacy.
|
From page 136... ...
Another important feature of a matrix sample is that the large number of questions ensures that all three dimensions of technological literacy are assessed. The assessment de scribed here is similar in structure to assessments conducted through the National Assessment of Education Progress (NAEP)
|
From page 137... ...
Content Specifications The ITEA Standards for Technological Literacy, AAAS Benchmarks for Science Literacy, and the NRC National Science Education Standards would be useful starting points for determining the content of a national sample assessment, just as they would be for the statewide assessment described in Case 1. Each of these documents suggests "benchmarks" F R O M T H E O R Y T O P R A C T I C E 137
|
From page 138... ...
Test developers would also have to create a detailed test- and item-specifications document. Performance Levels The development of performance levels would be as important for a national sample-based assessment as it would be for the statewide assessment described in Case 1.
|
From page 139... ...
Many decision makers, whose support would be necessary for the development of a national sample-based assessment, might have a limited understanding of technological literacy themselves. In addition, as might be true for a statewide assessment, the mathematics and science education communities might object to a separate assessment of technological literacy on the grounds that it would divert attention and resources from their efforts to improve student learning in their content areas.
|
From page 140... ...
Materials Provided 15 plastic drinking straws, each one 10 inches long 10 inches of masking tape 5 spice containers (~4 cm in diameter, 10 cm tall) filled with sand or water 2 large cardboard bricks log sheets Time Limit 25 minutes Case 3: National-Sample Assessment of Teachers Description and Rationale Case 3 involves an assessment of technological literacy for a national sample of pre-service and in-service K12 teachers.
|
From page 141... ...
Student selects a solution Student selects solutions Student provides detailed based on limited attention on the basis of efficiency reasons for selecting a to criteria. The solution and effectiveness.
|
From page 142... ...
. Content The ITEA Standards for Technological Literacy, the AAAS Bench marks for Science Literacy, and the NRC National Science Education Stan dards would be useful starting points for determining the content of a teacher assessment.
|
From page 143... ...
Third, setting discipline-specific benchmarks would require the involvement of experts in various dimensions of technological literacy and experts familiar with K12 curricula in the subjects of interest. For all of these reasons, setting performance levels and reporting results for this assessment must be approached with considerable care and sensitivity.
|
From page 144... ...
Sample Assessment Items Test Items for Generalists 1. An electric generator is used to convert what into what?
|
From page 145... ...
Focus on either the ethical, social, political, or environmental impact of this significant technological change, and identify the negative and positive consequences. (critical thinking and decision making dimension)
|
From page 146... ...
Broad population assessments might also provide opportunities to gauge how the dimensions of technological literacy play out in the situations and environments of everyday life, rather than in the somewhat artificial environment of the classroom. Researchers, policy makers, and the education and business communities might all benefit from informa tion about the nature of technological literacy outside the formal educa tion environment.
|
From page 147... ...
The people in this group tend to seek out information about specific technologies -- for example, technologies related to health and medical issues -- but they are generally more interested in the value and risks of specific technologies than in general issues of public policy. As Assessments of a group, technology consumers have been studied intensely by technology broad populations developers and manufacturers, who routinely conduct studies of users -- or might be potential users -- of specific technologies.
|
From page 148... ...
In some cases, rather than relying on earlier assessment instruments or surveys, assessment developers might consult with subject-matter experts in tech nology, the history of technology, and science, technology and society studies, as well as representatives of populations participating in the assessment and groups that are expected to make use of the results. The dimensions of technological literacy must be approached from a different angle in the context of broad populations.
|
From page 149... ...
Citizens also need to have enough background information to be able to absorb new information and form reasoned opinions about technological issues. Levels of understanding for various broad populations may range from a general appreciation of the importance of technology to a deeper understanding based on historical examples.
|
From page 150... ...
A number of measurement methods, strategies, and practices have been developed for studying broad populations. Deciding which of these to use will depend on the population of interest and the goals of the study.
|
From page 151... ...
(Answer choices: completely agree; agree; neither agree nor disagree; disagree; completely agree; not sure.)
|
From page 152... ...
2. Please indicate for each of the following sentences the extent to which you believe it is absolutely true, probably true, probably false, or absolutely false.
|
From page 153... ...
. This number is consistent with NSB survey data indicating that 61 percent of adult Americans visit an informal science institution (e.g., a zoo, aquarium, science center, natural history museum, or arboretum)
|
From page 154... ...
logical decisions, including career changes required by new technologies, many times in their lives. For this reason, "lifelong learning," which can take place formally in settings like community colleges and the workplace, or informally through independent reading, visits to museums and science centers, or exposure to radio, television, and the Internet, has become critical to technological literacy.
|
From page 155... ...
Performance Levels Assessments of visitors to informal-learning institutions would be most useful for identifying a spectrum of technological literacy rather than specific levels of literacy. Changes in the spectrum, for example, movement -- up or down -- of the entire curve or changes in the shape of the curve, would provide valuable information.
|
From page 156... ...
. In addition, compared to the population at large, patrons of science centers, zoos, and related institutions tend to have higher socio economic parameters, although institutions in urban areas attract more diverse patrons.
|
From page 157... ...
and can be programmed to select assessment items based on the visitor's characteristics and physical location in an exhibit space. Sample Test Items10 Give an example of a type of technology you like.
|
From page 158... ...
1997. Achieving Scientific Literacy: From Purposes to Practices.
|
From page 159... ...
2000. Civic Scientific Literacy and Attitude to Science and Technology: A Comparative Analysis of the European Union, the United States, Japan, and Canada.
|
From page 160... ...
2003a. Public school student, staff, and graduate counts by state: School year 200102.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.