|
Items in cart [1] |
Questions? Call 888-624-8373 |
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 1
Summary and Recommendations
The Committee on Indicators of Precollege Science and Mathe-
matics Education was established by the National Research Council
to develop indicators of the condition of science and mathematics
education in the nation's schools. The committee's first report con-
centrated on conventional indicators. In this report, the committee
makes recommendations for improved ways of monitoring the condi-
tion of education in these critical fields. Our recommendations are
based on two premises:
.
All students need the knowledge and reasoning skills that
good science and mathematics education provides. Not only should
students leave school scientifically and mathematically literate, but
they should also have acquired the mental tools with which they can
renew that literacy throughout their lives.
. What teachers and students do in schools determines how
much learning takes place. Student and teacher behaviors are influ-
enced by a variety of incentives and constraints. Among the many
influences on behavior are curriculum mandates and curricular sup-
port materials, working conditions for teachers, and resources at the
classroom level. This simplified model of the educational process
provides the framework for our report.
This chapter presents the committee's recommendations. Sup-
port for the recommendations, definitions of what the committee
1
OCR for page 2
2
INDICATORS OF SCIENCE AND MATNEMA TICS EDUCATION
means by high-quality education and by the term indicators, and
caveats about the interpretation of indicators are provided in the
subsequent chapters.
INDICATORS OF SCIENCE AND
MATHEMATICS EDUCATION
The committee makes three kinds of recommendations. We rec-
ommend a number of key indicators, to which we assign the highest
priority. We recommend supplementary indicators, which, although
of lesser importance, would also improve knowledge of the quality of
mathematics and science education. The key indicators we recom-
mend are listed below, and the supplementary indicators are listed
on page 4. We also make a number of proposals for research, either to
validate the recommended indicators or to lead to the development
of additional indicators.
For two of the key types of indicators dealing with assessment
of student learning and assessment of curriculum content important
development work needs to be done before they can become useful in-
dicators. For the assessment of learning in mathematics and science,
tests and exercises need to be developed that will allow assessment
of conceptual knowledge, process skills, and higher-order thinking in
addition to the factual knowledge and skills assessed by tests in cur-
rent use. For the assessment of curriculum, exemplary frameworks
containing substantive content and desirable learning goals need to
be constructed, each spanning several grade levels, to provide refer-
ents against which textbooks and other curriculum components can
be evaluated.
Recommended Key Indicators
Extent of student learning in mathematics and sci-
ence
Extent of scientific and mathematical literacy of
adults
Enrollment data for mathematics and science courses
taken by students in high school and the amount of
time spent on the study of science and mathematics
in elementary and middIe/junior high school
Nature of student activities during science and math-
ematics instruction
OCR for page 3
SUMMARY AND RECOMMENDATIONS
3
Extent of teachers' knowledge in the subject matter
that they are expected to teach
Salaries paid to college graduates with particular
subject-matter specialties who choose to enter var-
ious occupations
Quality of the curriculum content in state guidelines,
textbooks and associated materials, tests, and actual
classroom instruction in science and mathematics
through matching to exemplary curriculum frame-
works along four dimensions: breadth and depth of
treatment and scientific and pedagogic soundness
For each key indicator, unless otherwise noted in a specific rec-
ommendation, data should be collected in four-year cycles. For in-
dicators dealing with student learning, student behavior, teaching
effectiveness, and quality of the curriculum, information should be
collected and analyzed so it can be reported by student subgroup-
that is, data should be aggregated not only by region (or state, if
current efforts in that direction proceed), but also by students' age or
grade level, gender, race, ethnicity, socioeconomic status, and type
of community (urban, suburban, rural). The reason for aggregating
by student demographic variables is to establish to what extent there
are systematic inequities in the distribution of resources devoted
to science and mathematics education and systematic differences in
student learning.
Some of the proposed indicators are most appropriate at the
national level, for example, assessment of the scientific and math-
ematical literacy of the general population. Others may be most
policy-relevant at the school level, for example, the information oh
tained by observing student activities during instruction. Still others
are relevant at the national, state, and local levels, for example,
assessments of student learning or teacher knowledge of subject mat-
ter. When appropriate, the recommendations note the policy level
for which an indicator is intended. Recommendations for research
are addressed both to the research community and to those fund-
ing agencies concerned with better understanding and monitoring of
science and mathematics education.
OCR for page 4
4
INDICATORS OF SCIENCE AND MATHEMATICS EDUCATION
Recommended Supplementary Indicators
Amount of time spent on science and mathematics
homework
Teacher preparation college courses in mathematics
and science, majors and minors, advanced degrees
Teachers' use of time outside the classroom spent
on professional activities related to their teaching of
mathematics and science
Materials, facilities, and supplies available and used
by teachers in mathematics and science instruction
Level of federal financial support for science and
mathematics education
Commitment of resources by scientific bodies for the
improvement of mathematics and science education
in the schools
In the chapters that follow, our recommendations for improving
indicators in current use and developing new ones appear within the
appropriate domain of science and mathematics education: student
learning (Chapter 4) and student behavior (Chapter 5), teaching
quality (Chapter 6), quality of the curriculum (Chapter 7), and
financial and leadership support (Chapter 83. A consequence of
this organization is that recommendations for key indicators are
intermingled with recommendations for supplementary indicators
and with recommendations for research.
The remainder of this chapter presents all the recommendations
with amplifying material as they appear in the report, spelling out
the recommended key and supplementary indicators in greater detail
as well as needed research. The recommendations are organized by
domain of mathematics and science education, as they are in the
chapters of this report.
RECOMMENDATIONS
Indicators of Learning in Science and Mathematics
Indicators of student learning at the national, state, and local
levels should be based on scores on tests that are consonant with the
OCR for page 5
SUMAŁ4RY AND RECOMMENDATIONS
5
curriculum and all major curricular objectives, including the learning
of factual and conceptual knowledge, process skills, and higher-order
thinking in specific content areas. Measuring progress toward this
last objective is especially important, since it is possible that pres-
sures on school practitioners to increase student scores on multiple-
choice tests emphasizing recall of factual information may result in
diminished attention paid to the development of higher-order think-
ing skills. In order to establish how well major curricular objectives
are being met, test items used to assess students' mathematics and
science learning should not be exclusively in a multiple-choice format.
A significant number of items using an open-ended pencil-and-paper
format and a hands-on problem-solving format should also be used.
Research and Development: To provide the requisite
tests for use as indicators of student learning, the committee
recommends that a greatly accelerated program of research
and development be undertaken aimed at the construction
of free-response materials anct techniques that measure skills
not measured with multiple-choice tests. The committee
urges that the development of science tests at the K-5 level
receive immediate attention.
Techniques to be developed include problem-solving tasks, as
exemplified by the College Board Advanced Placement Tests; pencil-
and-paper tests of hypothesis formulation, experimental design, and
other tasks requiring productive-thinking skills, as exemplified by
questions in the British Assessment of Performance Unit Series;
hands-on experimental exercises, as exemplified by some test materi-
als administered by the National Assessment of Educational Progress
(NAEP) and the International Association for the Evaluation of Edu-
cational Achievement (TEA); and simulations of scientific phenomena
with classroom microcomputers that give students opportunities for
experunental manipulations and prediction of results.
The creation of new science tests for grades K-5 should be done
by teams that include personnel from the school districts that have
been developing hands-oI1 curricula to ensure that the new tests
match the objectives of this type of instruction. In addition to
providing valid national indicators of learning in areas of great im-
portance, such new assessment materials for science in grades K-5
OCR for page 6
6
INDICATORS OF SCIENCE AND MATHEAfA TICS EDUCATION
will provide models of tests that state and local school officials may
want to adopt and use.
Eey Indicator: The committee recommends that assess-
ment of student learning using the best available tests and
testing methods continue to be pursued in order to provide
periodic indicators of the quality of science and mathematics
education.
Tests should be given to students in upper-elementary, middle,
and senior high school (for example, in grades 4, 8, and 12~. Because
of the rapid changes taking place in science instruction in grades K-S,
assessment at this level should be carried out every two years, using
exercises developed according to the preceding recommendation. For
higher levels, a four-year cycle is appropriate. The tests should be
given to a national sample, using matrix-sampling techniques. Test
scores should be available for each test item or exercise and should
be reported over time and by student subgroups (e.g., gender, race,
ethnicity, type of community). Similar procedures are appropriate
for indicators of state or district assessments of student learning.
Research and Development: The committee recommends
that a research and development center be established to
provide for the efficient production, evaluation, and distribu-
tion of assessment materials for use as indicators of student
learning at district, state, and national levels and for use by
teachers in instruction.
The center should function as a centralized resource and clear-
inghouse that would make it possible for school people to survey the
available assessment materials and obtain those desired. The center
might be called the National Science and Mathematics Assessment
Resource Center.
Key Indicator: The committee recommends that, starting
in 1989, the scientific and mathematical literacy of a random
sample of adults (including 17-year-olds) be assessed. The
OCR for page 7
SUMMARY AND RECOMMENDATIONS
assessment should tap the dimensions of literacy discussed
in Chapter 2 and should be carried out every four years.
7
To make the desired types of assessment possible, effort should
be devoted over the next two years to developing interim assessment
tools that use some free-response and some problem-solving compo-
nents; these assessment tools should be used until more innovative
assessment techniques are available. The data collected should be
aggregated and reported by age, gender, race, ethnicity, socioeco-
nomic status, and geographic region so as to establish to what extent
there are systematic inequities in the distribution of scientific and
mathematical literacy.
Indicators of Student Behavior
Key Indicator: The committee recommends that data on
secondary school course enrollment be gathered on a four-
year cycle for both mathematics and science. The specific
data to be gathered are the number of semesters of science
and mathematics taken by students and total enrollment in
the variety of science and mathematics courses offered in
secondary schools.
Courses should be identified as to level of difficulty (e.g., for
eighth-grade mathematics: remedial, typical, enriched, algebra).
The indicators to be constructed from these data are the average
number of mathematics and science courses taken and the percent-
age of students enrolled in specific courses.
Key Indicator: The committee recommends that the data
to be gathered at the elementary- and middle-school level,
equivalent to course enrollment data, be the number of min-
utes per week devoted to the study of science and mathemat-
ics. The indicator should also be expressed both as a ratio
of all instructional time and of total time spent in school.
At each policy level-national, state, and local experts may
wish to define the minimum amount of class time necessary in each
grade, particularly for science. Because of the importance of possible
OCR for page 8
8
INDICATORS OF SCIENCE AND MATHEMATICS EDUCATION
differences among various groups (ethnic and racial, gender, socio-
economic status, etc.) we recommend that the data be collected at
the level of both the school and the individual student.
Key Indicator: The committee recommends development
of a time-use study involving external observers to obtain
some indication of the quality of the science and mathemat-
ics instruction being received. In science classes, this would
include, in addition to the teaching of conceptual and fac-
tual knowledge, the percentage of time spent by students
involved in the processes of science (observing, measuring,
conducting experiments, asking questions, etc.~. A similar
study is recommended for mathematics classes; a panel of
mathematics educators should deterrn~ne the nature of the
student behaviors sought.
Supplementary Indicator: The committee recommends
the collection of information on minutes per week spent on
science and mathematics homework.
The frequency and detail necessary for gathering data on home-
work are the same as for in-school activities-that is, the information
should be gathered every four years and allow analysis by ethnicity,
race, gender, grade level, and size and type of community. Na-
tional data are important for comparisons over time and with other
countries; states and local districts may also wish to have this infor-
mation. Care must be taken that homework done in school is not
double counted as both homework time and instructional time.
Research and Development: The committee recommends
further research and development on possible supplemen-
tary indicators in the following three areas of out-of-school
behaviors, with the goal of clarifying their relationships to
student mathematics and science learning:
.
Amount of time (minutes) devoted to out-of-school sci-
ence and mathematics activities, for example, going to
zoos and science museums, watching science programs
on television, reading science books, playing with a
OCR for page 9
SUMMARY AND RECOMMENDATIONS
.
computer at home, voluntarily doing science projects
or mathematics puzzles.
Percentage of students reporting that they use (apply)
the concepts of science and mathematics from time to
time in their own lives. One way to implement this in-
dicator is to conduct a survey on the number of times
students faced a personal decision and relied on some-
thing that they learned in science or mathematics to
help them make that decision.
Percentage of students reporting that they use the con-
cepts of science and mathematics to help them address
some persistent societal problem.
Research and Development: The committee recommends
continued research on linkages between student learning and
various student activities, on more effective ways of assess-
ing activities that affect learning, and on the factors that
influence individuals to engage in these activities.
Research and Development: Given the importance at-
tached by science and mathematics educators to the devel-
opment of attitudes that will foster continuing engagement
with science and mathematics, the committee recommends
that research be conducted to establish which attitudes af-
fect future student and adult behavior in this regard and to
develop unambiguous measures for those that matter most.
Research and Development: The committee recommends
research to identify and validate constructs related to the
continuing involvement of students and adults with science
and mathematics throughout their lives. In addition to the
refinement of these constructs, strategies should be explored
for obtaining indicators of the relevant constructs and asso-
ciated behaviors.
Indicators of Teaching Quality
Key Indicator: The committee recommends that samples
of teachers be selected to take tests that probe the same
9
OCR for page 10
10 INDICATORS OF SCIENCE AND ~TH~TICS EDUCATION
content and skills that their students are expected to master.
For this purpose, tests for teachers should be developed to
include the same kinds of improvements that the committee
recommends for tests of student learning.
The distribution of teachers' test scores should be reported by
student background and characteristics. This will provide informa-
tion about the distribution across different student groups of teachers
who are in command of the mathematics and science they are ex-
pected to teach. Both current distribution and change over time
are of interest; therefore, tests should be given every four years to a
sample of all teachers and every two years to a sample of newly hired
secondary school mathematics and science teachers.
Supplementary Indicator: The committee recommends
reorganization of the information currently being collected
on teacher preparation (college courses in mathematics and
science, majors and minors, advanced degrees), using various
student groups taught as the reporting groups of interest.
The information reported should display the percentage of stu-
dents with particular backgrounds and characteristics who are being
taught mathematics and science in elementary school as well as
courses in these domains in secondary school by teachers with spe-
cific college preparation. For this indicator also, four-year cycles are
appropriate for collection and analysis of information.
Research and Development: The committee recommends
that research be undertaken on two issues: the impact of
teachers' knowledge of subject matter on their effectiveness
in teaching these subjects to students, and the role of early
home and school experiences in determining the decisions to
become a teacher and on how and what to teach.
Supplementary Indicator: The committee recommends
that time-budget studies be conducted, asking teachers to
record how they spend time related professionally to their
OCR for page 11
SUMMARY AND RECOMMENDATIONS
present or future classroom activities, other than in the
classroom itself, during a particular period, perhaps a week.
11
The information collected should be evaluated against sets of
activities identified by experts as advancing effectiveness in the cIass-
room in teaching mathematics or science. Investigations of the
relationships between professional activities reported by teachers
and teaching effectiveness should be conducted to help refine this
indicator.
Research and Development: The committee recommends
research on the following aspects of the behavior of teachers
in science and mathematics instruction:
.
the factors affecting teacher responses to changes in the
intended curriculum;
the use of hands-on experiences involving concrete ma-
terials, laboratory experiments, and computers; and
allowing an adequate period of time for students to for-
mulate responses to questions.
Supplementary Indicator: The committee recommends
that data be collected on a four-year cycle through open-
ended surveys on the materials, facilities, and supplies avail-
able and used by teachers in mathematics and science in-
struction.
An indicator can be constructed from this information by report-
ing on the levels of resources being used in the classroom by student
subgroups of different backgrounds and competencies.
Key Indicator: The committee recommends collection at
least every three years (preferably every two years) of de-
tailed information on the salaries paid to college graduates
with particular subject matter specialties who choose to en-
ter various occupations.
OCR for page 12
12 INDICATORS OF SCIENCE AND MATHEMATICS EDUCATION
The information should include data on starting salaries and on
salaries after 15 years of experience. These data should be reported
in a manner that facilitates comparisons of salaries in teaching with
salaries in other occupations for college graduates trained in partic-
ular sciences and mathematics.
indicators of curriculum Quality
Research and Development: In order to develop indica-
tors of breadth of content coverage in the science and math-
ematics curriculum, the committee recommends that exem-
plary frameworks be constructed for the following
curriculum blocks: grades K-5 science, grades K-5 mathe-
matics, grades - 8 science, grades 6-8 mathematics, grades
9-12 literacy in science, grades - 12 literacy in mathematics,
grades 9-12 science for college-bound students, and grades
9-12 mathematics for college-bound students. The frame-
works for grades K-5 and 6-8 science should be accorded
the highest priority.
The frameworks must represent the structures of the subject
matter and desirable learning goals, or alternatives among desirable
goals.
Key Indicator: Once the frameworks are constructed, the
committee recommends that three elements of the intended
curriculum should be matched and rated against them for
content coverage: state guidelines, textbooks and such as-
sociated materials as computer software and laboratory ex-
ercises, and tests. The frameworks should also be used to
analyze the content coverage of the implemented curriculum
(i.e., the content presented to the student as reported by
classroom teachers).
The ratings obtained through analysis of the three elements of
the intended curriculum and analysis of the implemented curriculum
will provide the raw material for the construction of indicators of
content coverage. The ratings should be carried out every four years
OCR for page 13
SUMMARY AND RECOMMENDATIONS
13
at the national level in synchronization with the student assessments
recommended above so that the indicators can be used together.
Research and Development: The committee recommends
that research be carried out to establish the validity of
teacher-reported information regarding content coverage in
the classroom.
Research and Development: Standards of excellence
should be developed based on the best of curricula in current
use.
High-quality programs encompassing the curriculum blocks sug-
gested above should be selected, profiled, and analyzed to provide
models of excellence in depth of content coverage, scientific accuracy,
and pedagogic soundness of science and mathematics curricula.
Key Indicator: The quality of the curriculum should be
assessed by expert panels along three dimensions: depth of
content treatment, scientific accuracy, and pedagogic sound-
ness. Ratings for each of these quality dimensions should be
assigned to the three elements of the intended curriculum
(i.e., state guidelines, texts and associated materials, and
tests). Assessments regarding depth of treatment should
also be made of the implemented curriculum through teacher
and student surveys and classroom observation.
To assess the depth of content treatment, the frameworks devel-
oped according to the recommendation made above should be used
to identify the critical topics that constitute a coherent curriculum.
Weights assigned by each rating panel regarding the depth of treat-
ment desired for a given topic must be made explicit in reporting
results.
The assessment of the scientific accuracy of the intended cur-
riculum should be carried out by scientists in the relevant disciplines.
The scientific content of the frameworks shoud be used to construct
the tests of teacher competency of subject matter recommended in
OCR for page 14
14 INDICATORS OF SCIENCE AND MATHEMATICS EDUCATION
Chapter 6 and such tests used as a minimum measure of the scientific
accuracy of the actual curriculum experienced by students.
Research and Develop~nent: The committee recommends
research to provide validity checks on the standards being
used to assess depth of treatment, scientific accuracy, and
pedagogic soundness of science and mathematics curricula.
For example, research should be undertaken to establish what
pedagogic knowledge teachers need to have and need to know how to
use in order to teach science and mathematics effectively to students
of different ages, backgrounds, and competencies.
[r`dicators of Financial and Leadership Support
Supplementary Indicator: The committee recommends
the construction of a set of accounts detailing the level and
type of support for science and mathematics education from
all departments and agencies of the federal government that
fund relevant programs.
The importance of having reliable annual data on the level of
federal financial support merits the investment necessary to con-
struct such a set of accounts. Agencies should be encouraged to
report budget and funding data by categories identifiable as precol-
lege mathematics and science education, and funds should be made
available (possibly through NSF) to perform the necessary analy-
ses. The kind of disaggregation of financial support for science and
mathematics education found in the NSF budget could be used as
a mode! for developing the recommended cross-agency indicator of
federal support.
Supplementary Indicator: The committee recommends
that indicators be designed using budgetary data of scien-
tific bodies and information on stab time and volunteer time
devoted to education and that these indicators be routinely
available to reflect the commitment of resources by scien-
tific bodies for the improvement of mathematics and science
education in the schools.
Representative terms from entire chapter:
student learning
