Questions? Call 888-624-8373

PAPERBACK + PDF
your price: $64.50
add to cart

PAPERBACK
list:$55.00
Web:$49.50
add to cart

PDF BOOK
your price: $42.50
add to cart

PDF CHAPTERS
your price: $4.60
select

Rights & Permissions

topleft topright

Methodological Advances in Cross-National Surveys of Educational Achievement (2002)
Board on Testing and Assessment (BOTA)
 Center for Education (CFE)

Page
363
bottomleft bottomright
  E-mail
 Facebook
 Digg
 Stumble
 Twitter
More

The following HTML text is provided to enhance online readability. Many aspects of typography translate only awkwardly to HTML. Please use the page image as the authoritative form to ensure accuracy.

Methodological Advances in Cross-National Surveys of Educational Achievement

Index

Accountability, opportunity to learn measures, 232

Achievement motivation theory, 118, 119-145, 205

Adaptation of tests, see Translating and adapting tests

Administration of tests, see Standardized administration

Administrators

opportunity to learn measures, 231-232

principals, 4, 90-91

sampling, 90-91, 109-110

Age and grade factors, 18, 19, 29, 31, 38, 271-272, 284-287, 331-333, 344-345

curriculum and instructional materials, 38

dropouts, 10, 292, 333

International Adult Literacy Survey (IALS), 14, 85, 86, 190(n.3), 209, 268-269, 273-274, 292

International Assessment of Educational Progress (IAEP), 31, 94

opportunity to learn, 236-237, 242, 243, 247, 250, 259

sampling issues, 81, 87-88, 91-92, 94, 95, 99, 101, 102, 104-105, 107-108, 276-278, 292

Second International Science Study (SISS), 47, 177

Third International Mathematics and Science Study (TIMSS), 36-37, 45, 51, 95, 218, 219, 271-272, 276-278, 282-283, 301, 306-312, 314-315

triangulation of data, 218

Analogy questions, 61

Attitudes and beliefs, 7, 118, 119, 223

see also Cultural factors;

Emotional factors;

Public opinion achievement motivation , 118, 119-145, 205

consensus by participating nations, 40-42

emic beliefs, 132-140

emotional factors, 122

family factors, 7, 119, 124-125, 134

self-esteem, 120, 127-128

teachers, 201, 252

Attribution theory, 131


Background questionnaires, 7, 12, 18-19

see also Attitudes and beliefs;

Cultural factors;

Family factors;

Social factors

Beginning Teacher Evaluation Study (BTES), 235, 258, 306

Beliefs, see Attitudes and beliefs

 Page
363

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 363
Methodological Advances in Cross-National Surveys of Educational Achievement Index Accountability, opportunity to learn measures, 232 Achievement motivation theory, 118, 119-145, 205 Adaptation of tests, see Translating and adapting tests Administration of tests, see Standardized administration Administrators opportunity to learn measures, 231-232 principals, 4, 90-91 sampling, 90-91, 109-110 Age and grade factors, 18, 19, 29, 31, 38, 271-272, 284-287, 331-333, 344-345 curriculum and instructional materials, 38 dropouts, 10, 292, 333 International Adult Literacy Survey (IALS), 14, 85, 86, 190(n.3), 209, 268-269, 273-274, 292 International Assessment of Educational Progress (IAEP), 31, 94 opportunity to learn, 236-237, 242, 243, 247, 250, 259 sampling issues, 81, 87-88, 91-92, 94, 95, 99, 101, 102, 104-105, 107-108, 276-278, 292 Second International Science Study (SISS), 47, 177 Third International Mathematics and Science Study (TIMSS), 36-37, 45, 51, 95, 218, 219, 271-272, 276-278, 282-283, 301, 306-312, 314-315 triangulation of data, 218 Analogy questions, 61 Attitudes and beliefs, 7, 118, 119, 223 see also Cultural factors; Emotional factors; Public opinion achievement motivation , 118, 119-145, 205 consensus by participating nations, 40-42 emic beliefs, 132-140 emotional factors, 122 family factors, 7, 119, 124-125, 134 self-esteem, 120, 127-128 teachers, 201, 252 Attribution theory, 131 Background questionnaires, 7, 12, 18-19 see also Attitudes and beliefs; Cultural factors; Family factors; Social factors Beginning Teacher Evaluation Study (BTES), 235, 258, 306 Beliefs, see Attitudes and beliefs

OCR for page 364
Methodological Advances in Cross-National Surveys of Educational Achievement Benchmarking, 4-5, 16, 20, 305-306, 313, 323-333 Board on International Comparative Studies in Education (BICSE), 212, 311, 313, 314-315, 322, 334-335, 336, 337, 342, 344 member biographical sketches, 358-362 sampling issues, 81, 82-84, 102, 103, 105, 112 Canada, 41 Case studies, 4, 7, 12, 21, 219, 343, 345 Civic Education Study, 203-204, 209-210, 211, 214-217, 220 cultural-cognitive issues, 123, 140 cultural factors, other, 12, 13, 123, 140, 199, 203, 209-219, 222 classroom videos, 21, 220; see also “Third International Mathematics and Science Study” below interviews, 121, 132, 133, 134, 136, 204, 205, 206, 220 Third International Mathematics and Science Study (TIMSS), 12, 13, 123, 209-219, 222 classroom videos, 4, 6, 13, 17-18, 123, 144, 210, 211, 218, 221, 222, 306, 336-337, 340-341 CASMIN, see Comparative Analysis of Social Mobility in Industrial Nations Causal inferences, 5, 15, 19, 231-318, 334, 338, 339, 342, 347, 354-355 benchmarking, 4-5, 16, 20, 305-306, 313, 323, 323-333 cultural factors, 122, 131-132, 142, 200, 205, 206, 208, 210, 213, 219-220 family background, 173 national policy applications, 231-232, 267, 295-317; see also “benchmarking” above opportunity to learn, 231-266 statistical issues, 268-269, 272-273, 276, 281, 283-290, 291, 292; see also Statistical analyses Chinese, 315 cultural-cognitive factors, 130, 143 family background, 156-157 translation issues, 59, 66, 67, 69 Civic education, 4, 321, 322 sampling, 85, 86 Civic Education Study, 203-204, 209-210, 211, 214-216, 220 Classroom observations see also Opportunity to learn Beginning Teacher Evaluation Study (BTES), 235, 258, 306 videotapes, 21, 218, 220, 222 Third International Mathematics and Science Study (TIMSS), 4, 6, 13, 17-18, 123, 144, 210, 211, 218, 221, 222, 306, 336-337, 340-341 Class size, 167, 202 Cognitive processes, 6, 11, 33-35, 328 case studies, 123, 140 cultural, 117-149, 354 mathematics, 118, 119-120, 137, 138 science, 118, 138 First International Mathematics Study (FIMS), 33, 35 First International Science Study (FISS), 33 International Assessment of Educational Progress (IAEP), 31, 34 item field testing, 44 item formats, 28, 37 opportunity to learn, 243 Second International Mathematics Study (SIMS), 34 Second International Science Study (SISS), 33 statistical analyses, 142, 267 Third International Mathematics and Science Study (TIMSS/TIMSS-R), 34-35, 137, 139-140, 142, 144 Cohort studies, 6, 9-10, 19, 52-53, 104, 174, 269-270, 278-283, 284-288 Third International Mathematics and Science Study (TIMSS/TIMSS-R), 9-10, 278-281, 282-283 Comparative Analysis of Social Mobility in Industrial Nations (CASMIN), 164-165 Comparative and International Education Society (CIES), 224(n.2) Computer-assisted testing, 54, 344 Computer science, other, 4 see also Internet databases, 16, 118, 188 qualitative, 13, 199, 212, 216, 217, 218, 219, 221, 223, 224

OCR for page 365
Methodological Advances in Cross-National Surveys of Educational Achievement family ownership of PC, 182-183, 287 opportunity to learn, 248 sampling, 85 Computer Study (IEA), 209 Confidentiality issues, 222 Cost and cost-effectiveness sampling issues, 106-107 temporal sequencing, 220 translation of tests, 70 triangulation of data, 218 Cultural capital, 150, 153-154, 169, 170-172, 174, 177, 178, 182-183, 189, 332, 344 Cultural factors, 4, 5, 6, 7, 10-13, 150, 198-229, 354 see also Attitudes and beliefs; Family factors; Language factors; Political factors; Race/ethnicity; Social factors; Translating and adapting tests achievement motivation theory, 118, 119-145, 205 bias, 45-46, 208 case studies, 12, 13, 123, 140, 199, 203, 209-219, 222 Civic Education Study, 203-204, 209-210, 211, 214-217, 220 classroom videos, 21, 220 TIMSS classroom videos, 4, 6, 13, 17-18, 123, 144, 210, 211, 218, 221, 222, 306, 336-337, 340-341 causal inferences, 122, 131-132, 142, 200, 205, 206, 208, 210, 213, 219-220 civic education, 4, 85, 86, 321, 322 Civic Education Study, 203-204, 209-210, 211, 214-216, 220 cognitive, 117-149, 354 committee study methodology, 6, 7 definition of culture, 199-203, 207-208 differential item functioning, 8-9, 45-46, 69 educational attainment, 139, 159 immigrants, 152, 171, 177, 183, 283 International Association for the Evaluation of Educational Achievement (IEA), 45, 123, 141, 142, 198-99, 200-201, 203-204, 209-222 mathematics assessments, general, 118, 119-120, 137, 138 teaching methodology, 119, 124, 135-136 Third International Mathematics and Science Study (TIMSS), 4, 6, 12-13, 17-18, 123, 144, 210, 211, 218, 221, 222, 306, 336-337, 340-341 Curriculum and instructional materials, 13, 14, 17, 20, 28, 29-30, 32-33, 37-40, 53, 123, 289-290, 292, 297, 325, 328-329, 343 see also Opportunity to learn; Teaching methodology; Textbooks age factors, 38 consensus by participating nations, 42 English, influence of, 33, 61, 62 International Association for the Evaluation of Educational Achievement (IEA), 37-38, 39-40 First International Mathematics Study, 31, 38 Second International Mathematics Study (SIMS), 17, 31, 290, 308, 315 Second International Science Study (SISS), 38 Third International Mathematics and Science Study (TIMSS), 5, 17, 35, 38-39, 51, 53, 54, 210, 211, 213, 290-291, 296, 298-314, 343 summary scores, 50 Databases, 16, 118, 188 see also Internet qualitative, 13, 199, 212, 216, 217, 218, 219, 221, 223, 224 Demographic and Health Surveys, 180 Developing countries, 152, 161-162, 167 Differential item functioning, 8-9, 45-46, 69 Disabled students, sampling issues, 96, 102, 107 Documentation, 4, 9, 327-330 committee study methodology, 7 translation guidelines, 78-79 Dropouts, 10, 292, 333 Early Childhood Longitudinal Study (ECLS), 291 Economic factors, 269, 296 see also Cost and cost-effectiveness; Developing countries; Socioeconomic status cultural capital, 150, 153-154, 169, 170-172, 174, 177, 178, 182-183, 189, 332, 344

OCR for page 366
Methodological Advances in Cross-National Surveys of Educational Achievement human capital, 154, 297, 315, 331, 333, 344 social capital, 150, 160, 170-172, 183, 189 teacher salaries, 167 Educational attainment, 10, 268, 287, 297, 305 cultural factors, 139, 159 family background, 153-172, 175, 255-256, 287 opportunity to learn, 250-251, 252, 255-256 Educational Testing Service (ETS) see also International Assessment of Educational Progress consensus by participating nations, 40 sampling, 85 translations, 74 Emotional factors, 122 see also Attitudes and beliefs England, 51, 63, 134, 156-159, 166, 271, 335 Estimation procedures, 14, 69, 80, 82, 84, 91, 95, 109 Ethnicity, see Race/ethnicity Expert opinion see also Board on International Comparative Studies in Education consensus by participating nations, 42 field testing and, 44 Extended-answer problems, 30, 35-36, 37, 43, 53 Family factors, 6, 11-12, 150-197, 205-206, 344-345, 354 attitudes and beliefs, 7, 119, 124-125, 134 causal inferences, 173 computer ownership, 182-183, 287 cultural capital, 150, 153-154, 169, 170-172, 174, 177, 178, 182-183, 189, 332, 344 educational attainment of parents, 153-172, 175, 255-256, 287 human capital, 154, 297, 315, 331, 333, 344 International Assessment of Educational Progress (IAEP), 190(n.3) International Association for the Evaluation of Educational Achievement (IEA), 150, 155, 166-167, 172-189 (passim) First International Mathematics Study (FIMS), 150, 173-178 (passim), 182 First International Science Study (FISS), 150, 160-161, 174, 175, 176-177, 182, 184 Second International Mathematics Study (SIMS), 150, 160-163, 173, 174, 176-177, 178, 182-183, 188, 256-257 Second International Science Study (SISS), 150, 174, 175, 176-177, 182, 188 Third International Mathematics and Science Study (TIMSS/ TIMSS-R), 12-13, 150, 173-188 (passim), 190-191(n.4), 212, 257 language spoken at home, 162-163, 177, 183 opportunity to learn, 239, 255-257 Program for International Student Assessment (PISA), 12-13, 150-151, 173, 174, 175, 178-179, 182, 183, 188, 189-190, 190-191(n.4) social capital, 150, 160, 170-172, 183 socioeconomic status, 12-13, 19, 150, 153-169, 171-172, 175-184, 186, 188-189, 269, 287, 338 opportunity to learn, 239, 255-257 statistical analyses, 15-16, 168, 267-268, 269 structure of family, 169-170, 176-177, 182 Field tests, 8-9, 40-41, 43-45 qualitative assessments, 221 teacher logs, 246 translations, 59, 64, 68, 70 First International Mathematics Study (FIMS), 31, 47, 52, 321 cognitive processes, 33, 35 consensus by participating nations, 40 curricula and instructional materials, 31, 38 family factors, 150, 173-178 (passim), 182 opportunity to learn, 232, 237-238, 239-242, 250, 252, 253, 259, 260 sampling, 84, 85, 86, 87 First International Science Study (FISS), 52 cognitive processes, 33 family factors, 150, 160-161, 174, 175, 176-177, 182, 184 sampling, 84, 85, 86, 87-88 Formats of items, see Item formats

OCR for page 367
Methodological Advances in Cross-National Surveys of Educational Achievement Gender factors, 269 item field testing, 43 opportunity to learn, 239 parent’s socioeconomic status, 155-164, 176-179 sampling, 90 General equivalency diploma (GED), 60 Germany, 63, 123, 158-159, 187, 306 Grade level, see Age and grade factors Grain size, content specification, 14-15, 30-31, 32 Handicapped students, see Disabled students Hispanics, 60, 61-62, 65-66 Human capital, 154, 297, 315, 331, 333, 344 see also Cultural capital; Social capital; Socioeconomic status Immigrants, 152, 171, 177, 183, 283 see also Hispanics Inferences, see Causal inferences Instructional materials, see Curriculum and instructional materials; Textbooks Intelligence and intelligence assessments cultural factors, 118, 122, 130, 134-135, 143-144 translation, 63, 74 International Adult Literacy Survey (IALS), 14, 85, 86, 190(n.3), 209, 268-269, 273-274, 292 International Assessment of Educational Progress (IAEP), 4, 31-32, 47, 53 age/grade factors, 31, 94 cognitive processes, 31, 34 consensus by participating nations, 41 cultural bias, 45, 46 family factors, 190(n.3) item field testing, 43 sampling, 86, 93-95, 104 International Association for the Evaluation of Educational Achievement (IEA), 4, 52, 54, 117 see also First International Mathematics Study; First International Science Study; Second International Mathematics Study; Second International Science Study; Third International Mathematics and Science Study case studies and, 12, 123 Civic Education Study, 203-204, 209-210, 211, 214-216, 220 Computer Study (IEA), 209 cultural factors, 45, 123, 141, 142, 198-99, 200-201, 203-204, 209-222 curricula, 37-38, 39-40 family factors, 150, 155, 166-167, 172-189 (passim) item formats, 36 opportunity to learn, 237 policy implications, 29, 296-297 sampling, 85, 93, 104, 105 International Labor Organization, 164 International Socioeconomic Index (ISEI), 164 International Standard Classification of Education (ISCED), 164-165, 175 International Standard Classification of Occupations (ISCO), 164 International Test Commission, 65 Internet access at home, 287 teacher logs, opportunity to learn, 245-246 Third International Mathematics and Science Study (TIMSS), 188, 218, 221, 245-246, 315(n.2) Interviews, 121, 132, 133, 134, 136, 204, 205, 206, 220 ISCED, see International Standard Classification of Education Item formats, 27-28, 35-37, 47-48 analogy questions, 61 extended-answer problems, 30, 35-36, 37, 43, 53 multiple-choice items, 9, 20, 30, 35-37, 40, 43, 71 short-answer items, 35, 37, 43, 53 student familiarity with, 28, 30 Item response models, 14, 48-50, 53 Japan, 11, 15, 224(n.4), 274-278, 289, 306, 315 achievement motivation, 119, 120, 123, 124-128, 130, 134-136, 144, 206 family background, 156-159, 162-163, 187 opportunity to learn, 254, 255 Language factors, 321 see also Hispanics; Reading comprehension; Translating and adapting tests; specific countries cultural capital, 171

OCR for page 368
Methodological Advances in Cross-National Surveys of Educational Achievement culture defined, 201 family background, home language, 162-163, 177, 183 sampling issues, 85, 90, 96, 102, 107 second language achievement, 4, 102 Mass media, 3, 28, 295 Mathematics assessments, 4, 16, 54, 308-309, 321 see also First International Mathematics Study; Second International Mathematics Study; Third International Mathematics and Science Study cultural-cognitive factors, 118, 119-120, 137, 138 National Council of Teachers of Mathematics (NCTM), 247-249 opportunity to learn measures, 232, 237, 243-244, 247-250, 262 translation issues, 59, 63 Matrix procedures, 8, 40, 46-48, 53, 325-326 Multiple-choice items, 9, 20, 30, 35-37 consensus by participating nations, 40 field testing, 43 reading literacy, 37 translations, 71 National Assessment of Educational Progress (NAEP), 5, 16-17, 18, 28, 31, 47, 53, 307, 323 sampling, 93, 110 National Center for Education Statistics (NCES), 311, 333, 334, 337 National Council of Teachers of Mathematics (NCTM), 247-249 National Educational Longitudinal Study (NELS), 291 National Institute for Testing and Evaluation, 67 A Nation At Risk, 5, 298, 305-306, 323, 324 Nonresponse, see Response rates NUD*IST database, 216 Opportunity to learn (OTL), 6, 13-14, 17, 39-40, 45, 50, 53, 201, 202, 211, 231-266, 336 accountability, 232 administrators role, 231-232 age/grade factors, 236-237, 242, 243, 247, 250, 259 causal inferences, 231-266 computer science, 248 educational attainment and, 250-251, 252, 255-256 family factors, 239, 255-257 gender factors, 239 International Association for the Evaluation of Educational Achievement (IEA) , 237 First International Mathematics Study (FIMS), 232, 237-238, 239-242, 250, 252, 253, 259, 260 Second International Mathematics Study (SIMS), 238, 239, 241, 242, 250, 251-252, 253-254, 256-257, 259-260 Third International Mathematics and Science Study (TIMSS/ TIMSS-R), 231, 241-243, 244-245, 250, 252, 254, 257, 288, 336 mathematics assessments, general, 232, 237, 243-244, 247-250, 262 memorization, 243 policy, 231-232 problem-solving skills, 232, 243, 248 reading comprehension, 247, 255-256, 258-259 science assessments, general, 232, 249-250 social factors, general, 239, 255-257, 258-259, 262 socioeconomic status, 239, 255-257 statistical analysis, 233-234, 243, 251-255, 258, 288, 290 teacher logs, 218, 221, 222, 236, 242-249, 258 Organization for Economic Cooperation and Development (OECD), 4, 21-22 see also Program for International Student Assessment International Adult Literacy Survey (IALS), 14, 85, 86, 190(n.3), 209, 268-269, 273-274, 292 Pedagogy, see Opportunity to learn; Teaching methodology Political factors, 323 see also Public opinion causal inferences, benchmarking, 4-5, 16, 20, 305-306, 313, 323-333 causal inferences, other, 231-232, 267, 295-317

OCR for page 369
Methodological Advances in Cross-National Surveys of Educational Achievement civic education, 4, 85, 86, 321, 322 Civic Education Study, 203-204, 209-210, 211, 214-216, 220 consensus by participating nations, 40-42 opportunity to learn measures, 231-232 sampling issues, 106-107 state-level, 295-296, 305-306, 308, 311-312 statistical analyses, 231-232, 267 Third International Mathematics and Science Study (TIMSS/TIMSS-R), 12, 296, 298-314 Principals, 4, 90-91 Privacy, see Confidentiality issues Problem-solving skills, 35 cultural-cognitive factors, 119 item formats, 37 opportunity to learn measures, 232, 243, 248 Program for International Student Assessment (PISA), 4, 8, 9 administration standardization, 58 age vs grade cohorts, 19 construct definition, 58 family background, 12-13, 150-151, 173, 174, 175, 178-179, 182, 183, 188, 189-190, 190-191(n.4) sampling, 58, 85, 86, 100-102, 105, 106, 108, 109, 110 translations, 59, 63, 65, 66, 69, 70-73, 74 Psychometrics, 8-9, 58 see also Item formats differential item functioning, 8-9, 45-46, 69 First International Mathematics Study (FIMS), 31 item response models, 14, 48-50, 53 Public opinion, 29, 267, 332 see also Mass media; Political factors civic education, 4, 85, 86, 321, 322 Civic Education Study, 203-204, 209- 210, 211, 214-216, 220 mass media, 3, 28, 295 Qualitative data, 21, 136, 143-144, 203-224, 341-342 see also Case studies; Cultural factors; Social factors; Videotapes databases, 13, 199, 212, 216, 217, 218, 219, 221, 223, 224 interviews, 121, 132, 133, 134, 136, 204, 205, 206, 220 teachers, 21, 201, 202, 205-206; see Classroom observations; Videotapes Twenty Statements Test, 128 Race/ethnicity, 19, 269 see also specific countries Hispanics, 60, 61-62, 65-66 immigrants, 152, 171, 177, 183, 283 Reading comprehension, 4, 17, 287, 321-322 International Adult Literacy Survey (IALS), 14, 85, 86, 190(n.3), 209, 268-269, 273-274, 292 International Assessment of Educational Progress (IAEP), 31 item formats, 37 opportunity to learn, 247, 255-256, 258-259 sampling, 85 Reporting, see Documentation Response rates, 9, 81, 93, 94, 96-97, 99, 100, 103, 106, 110-111 Rural areas, 271 Russia, 134, 158-159 Sampling, 4, 6, 8-9, 10, 14, 52-53, 80-114, 269-270, 272-278, 325-326, 330-333, 343-344, 346 see also Age and grade factors; Cohort studies; Response rates administrators, 90-91, 109-110 age/grade factors, 81, 87-88, 91-92, 94, 95, 99, 101, 102, 104-105, 107-108, 276-278, 292 Board on International Comparative Studies in Education (BICSE), 81, 82-84, 102, 103, 105, 112 civic education assessment, 85, 86 computer science, 85 cost issues, 106-107 disabled students, 96, 102, 107 dropouts, 10, 292, 333 gender factors, 90 International Assessment of Educational Progress (IAEP), 86, 93-95, 104 International Association for the Evaluation of Educational Achievement (IEA), 85, 93, 104, 105

OCR for page 370
Methodological Advances in Cross-National Surveys of Educational Achievement First International Mathematics Study (FIMS) , 84, 85, 86, 87 First International Science Study (FISS), 84, 85, 86, 87-88 Second International Mathematics Study (SIMS), 84, 85, 86, 88-91, 93, 105 Second International Science Study (SISS), 84, 85, 86, 91-93 Third International Mathematics and Science Study (TIMSS/ TIMSS-R), 58, 84, 85, 86, 95-100, 101, 104, 105, 106, 108, 109, 110, 111-112, 272-278 language factors, 85, 90, 96, 102, 107 matrix procedures, 8, 40, 46-48, 53, 325-326 missing data, 12, 14, 180, 181, 183, 274, 279, 281, 284, 292, 347 National Assessment of Educational Progress (NAEP), 93, 110 political factors, 106-107 principals, 90-91 Program for International Student Assessment (PISA), 58, 85, 86, 100-102, 105, 106, 108, 109, 110 qualitative studies, 222-223 reading comprehension assessment, 85 response rates, 9, 81, 93, 94, 96-97, 99, 100, 103, 106, 110-111 state-level assessments, 98 teaching methodology, 81, 97 translations, 69 vocational education, 88, 89, 98, 101, 107 Scholastic Aptitude Test (SAT), 5, 60, 297 Science achievement, 16-17, 118, 271, 321 see also First International Science Study; Second International Science Study; Third International Mathematics and Science Study cognitive psychology, 118, 138 item formats, 36 opportunity to learn measures, 232, 249-250 Program for International Student Assessment (PISA), 4 Scoring protocols, translation and adaptation, 58, 78-79 Second International Mathematics Study (SIMS), 9, 13-14, 31, 47, 53, 298, 308-309 cognitive processes, 34 curriculum analysis based on, 17, 31, 290, 308, 315 family factors, 150, 160-163, 173, 174, 176-177, 178, 182-183, 188, 256-257 multiple scores, 59 opportunity to learn, 238, 239, 241, 242, 250, 251-252, 253-254, 256-257, 259-260 sampling, 84, 85, 86, 88-91, 93, 105 textbooks, 162-163 Second International Science Study (SISS), 47, 53 age/grade factors, 47, 177 cognitive processes, 33 consensus by participating nations, 40 curriculum and instructional materials, 38 family factors, 150, 174, 175, 176-177, 182, 188 multiple scores, 59 sampling, 84, 85, 86, 91-93 Secretary’s Commission on Necessary Skills (SCANS), 315 Self-esteem, 120, 127-128 Short-answer items, 35, 37, 43, 53 see also Multiple-choice items Six Subject Study, 150 Social capital, 150, 160, 170-172, 183, 189 see also Cultural capital; Human capital Social factors, 4, 5, 10-13, 205, 332, 339-341, 345 see also Attitudes and beliefs; Cultural factors; Economic factors; Family factors; Political factors; Public opinion; Race/ethnicity; Rural areas; Urban areas achievement motivation, 118, 119, 124-127 case studies and, 12, 13, 123, 140, 199, 203, 209-219, 222 Civic Education Study, 203-204, 209-210, 211, 214-217, 220 classroom videos, 21, 220 TIMSS classroom video component, 4, 6, 13, 17-18, 123, 144, 210, 211, 218, 221, 222, 306, 336-337, 340-341

OCR for page 371
Methodological Advances in Cross-National Surveys of Educational Achievement civic education, 4, 85, 86, 321, 322 Civic Education Study, 203-204, 209-210, 211, 214-216, 220 cognitive, 117-149, 354 mathematics, 118, 119-120, 137, 138 science, 118, 138 competition, 207 opportunity to learn, 239, 255-257, 258-259, 262 statistical analyses, 15-16, 168, 267-268 Socioeconomic status, 12-13, 19, 150, 153-169, 171-172, 175-184, 186, 188-189, 269, 287, 338 see also Educational attainment opportunity to learn, 239, 255-257 parental occupation by sex, 155-164, 176-179 teacher salaries, 167 Southern Africa Consortium for Monitoring Educational Quality (SACMEQ) , 190-191(n.4) Spanish-language translations, 60, 61-62, 65-66 Standard International Occupational Prestige Index (SIOP), 164 Standardized administration, 28, 54 committee study methodology, 7 computer-assisted testing, 54, 344 translations and, 78 State-level assessments, 60 policy inferences, 295-296, 305-306, 308, 311-312 sampling issues, 98 Statistical analyses, 6, 9, 13-16, 267-294, 313-314 see also Causal inferences; Databases; Psychometrics; Sampling cultural-cognitive factors, 142, 267 cultural factors, other, 200-201, 224 differential item functioning, 8-9, 45-46, 69 estimation procedures, 14, 69, 80, 82, 84, 91, 95, 109 family background, 15-16, 168, 267-268, 269 grain size, content specification, 14-15, 30-31, 32 historical perspectives, 4, 13-14 item field testing, 43-44 item standards, 9 missing data, 12, 14, 180, 181, 183, 274, 279, 281, 284, 292, 347 National Center for Education Statistics (NCES), 311, 333, 334, 337 opportunity to learn, 233-234, 243, 251-255, 258, 288, 290 social factors, 15-16, 168, 267-268 translations, 74-75, 78 Statistics Canada, 85 Summary scores, 48-50 item response models, 14, 48-50, 53 Survey of Enacted Curriculum (SEC), 263(n.7) Taiwan, 130, 158-159 Teachers, 4, 218, 220-221 attitudes and beliefs, 201, 252 Beginning Teacher Evaluation Study (BTES), 235, 258, 306 cultural-cognitive factors, 119 homework assigned by, 217 logs, 218, 221, 222, 236, 242-249, 258 National Council of Teachers of Mathematics (NCTM), 247-249 preparation of, 123, 167 qualitative data, 21, 201, 202, 205-206; see also Classroom observations; Videotapes salaries, 167 time on task, 201, 202; see also Opportunity to learn Teaching methodology, 5, 17, 213, 288-290 see also Curriculum and instructional materials; Opportunity to learn cultural-cognitive factors, 119, 124, 135-136 end-of-unit tests, 27 sampling issues, 81, 97 small studies, 7 Third International Mathematics and Science Study (TIMSS), general, 303 TIMSS video component, 4, 6, 13, 17-18, 123, 144, 210, 211, 218, 221, 222, 306, 336-337, 340-341 tutoring, 12, 15 Textbooks, 167, 177 opportunity to learn, 248 Second International Mathematics Study (SIMS), 162-163

OCR for page 372
Methodological Advances in Cross-National Surveys of Educational Achievement Third International Mathematics and Science Study (TIMSS), 39, 211 Third International Mathematics and Science Study (TIMSS/TIMSS-R), 8, 13-15, 20-21, 29, 53, 117, 268, 272, 288, 291, 292, 335, 338-339, 342 administration standardization, 58 age/grade factors, 36-37, 45, 51, 95, 218, 219, 271-272, 276-278, 282-283, 301, 306-312, 314-315 case studies and, 12, 13, 123, 209-219, 222 classroom video component, 4, 6, 13, 17-18, 123, 144, 210, 211, 218, 221, 222, 306, 336-337, 340-341 cognitive processes, 34-35, 137, 139-140, 142, 144 cohort studies, 9-10, 278-281, 282-283 consensus by participating nations, 42 construct definition, 58 content areas, general, 31-32, 137, 139-140, 142, 144 cultural factors, 12-13, 46, 201, 203-204, 209-222, 223, 304, 336-337 curriculum analysis, 5, 17, 35, 38-39, 51, 53, 54, 210, 211, 213, 290-291, 296, 298-314, 343 differential item functioning, 46, 69 family background, 12-13, 150, 173-188 (passim), 190-191(n.4), 212, 257 Internet, 188, 218, 221, 245-246, 315(n.2) item field testing, 44-45 item formats, 36-37 multiple scores, 59-52 national policy inferences, 12, 296, 298-314 opportunity to learn, 231, 241-243, 244-245, 250, 252, 254, 257, 288, 336 sampling, 58, 84, 85, 86, 95-100, 101, 104, 105, 106, 108, 109, 110, 111-112, 272-278 summary scores, 48-49 textbooks, 39, 211 translations, 59, 62, 65, 69, 70-73, 74 Time factors, 267, 325-326, 327, 345-347 see also Age and grade factors; Cohort studies cognitive processes, measurement, 35 cultural-cognitive issues, 142 educational attainment, research on, 155 International Adult Literacy Survey (IALS), 273-274 item formats, 35-36, 37 teacher variables, time on task, 201, 202; see also Opportunity to learn test length, 329 TIMSS timeline, case study data, 213 translation of test items, 63 videos of classroom practices, 221 Translating and adapting tests, 6, 7, 8-9, 14, 28, 42, 58-79, 325 cost of, 70 field tests, 59, 64, 68, 70 guidelines, 77-79 intelligence tests, 63, 74 scoring protocols, 58, 78-79 Spanish translations, 60 statistical analysis, 74-75, 78 Third International Mathematics and Science Study (TIMSS/TIMSS-R), 59, 62, 65, 69, 70-73, 74 Tutoring, 12, 15 Twenty Statements Test, 128 United Nations Educational, Scientific, and Cultural Organization (UNESCO), 164, 190-191(n.4) Urban areas, 19, 271 Videotapes, 21, 218, 220, 222 Third International Mathematics and Science Study (TIMSS), 4, 6, 13, 17-18, 123, 144, 210, 211, 218, 221, 222, 306, 336-337, 340-341 Vocational education, sampling, 88, 89, 98, 101, 107 World Wide Web, see Internet

Representative terms from entire chapter:

science study