NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competencies and with regard for appropriate balance.
This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Frank Press is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Frank Press and Dr. Robert M. White are chairman and vice chairman, respectively, of the National Research Council.
This project was supported by funds from the U.S. Department of Education.
Library of Congress Catalog Card No. 92-64011
International Standard Book Number 0-309-04728-5
Additional Copies of this report are available from:
National Academy Press
2101 Consitution Avenue N.W. Washington, D.C. 20418
S621
Printed in the United States of America
First Printing, June 1992
Second Printing, November 1992
Panel to Review Evaluation Studies of Bilingual Education
STEPHEN E. FIENBERG (Chair),
Department of Mathematics and Statistics and Osgoode Hall Law School, York University
BARBARA F. FREED,
College of Humanities and Social Sciences, Carnegie Mellon University
KENJI HAKUTA,
School of Education, Stanford University
LYLE V. JONES,
Department of Psychology, University of North Carolina
KATHRYN B. LASKEY,
School of Information Technology and Engineering, George Mason University
LUIS C. MOLL,
School of Education, University of Arizona
P. DAVID PEARSON,
College of Education, University of Illinois
JOHN E. ROLPH,
The RAND Corporation, Santa Monica, California
PAUL R. ROSENBAUM,
Department of Statistics, The Wharton School, University of Pennsylvania
DONALD B. RUBIN,
Department of Statistics, Harvard University
KEITH F. RUST,
Westat, Inc., Rockville, Maryland
BURTON H. SINGER,
Department of Epidemiology and Public Health, Yale University
HERBERT L SMITH,
Population Studies Center, University of Pennsylvania
MICHAEL M. MEYER, Study Director (Carnegie Mellon University)
MICHELE L. CONRAD, Senior Project Assistant
Committee on National Statistics 1991–1992
BURTON H. SINGER (Chair),
Department of Epidemiology and Public Health, Yale University
NORMAN M. BRADBURN,
National Opinion Research Center, University of Chicago
MARTIN H. DAVID,
Department of Economics, University of Wisconsin
ANGUS S. DEATON,
Woodrow Wilson School of Public and International Affairs, Princeton University
NOREEN GOLDMAN,
Office of Population Research, Princeton University
LOUIS GORDON,
Department of Mathematics, University of Southern California
JOEL B. GREENHOUSE,
Department of Statistics, Carnegie Mellon University
ROBERT M. HAUSER,
Department of Sociology, University of Wisconsin
GRAHAM KALTON,
Westat, Inc., Rockville, Maryland
WILLIAM A. MORRILL,
Mathtech, Inc., Princeton, New Jersey
JANET L. NORWOOD,
The Urban Institute, Washington, D.C.
DOROTHY P. RICE,
Department of Social and Behavioral Sciences, School of Nursing University of California, San Francisco
JOHN E. ROLPH,
The RAND Corporation, Santa Monica, California
DONALD B. RUBIN,
Department of Statistics, Harvard University
MIRON L. STRAF, Director
Preface
In October of 1990 the United States Department of Education requested the National Research Council to convene a panel to review two major evaluation studies of bilingual education. The panel was formed in early 1991 and given the following charge:
The Panel to Review Evaluation Studies of Bilingual Education will review and assess the methodology of data collection and analysis of two major studies to evaluate bilingual education programs: a national longitudinal study of the effectiveness of instruction of limited-English-proficient students and a study to compare the effectiveness of three different instructional strategies for bilingual education for such students. The three strategies are immersion (teachers understand Spanish, but respond in English), early exit (students are placed in classes conducted in English as soon as possible), and late exit (both languages are maintained and a proficiency in English is developed over time).
In the first phase of its study, the panel will:
-
review the methods of data collection and analysis for potential sources of error and assess implications for some of the principal findings;
-
assess whether additional analyses of the data from either study would strengthen or broaden the findings and, if so, recommend some analyses that could be carried out; and
-
suggest alternative ways to compare the different instructional strategies and provide advice to the Department of Education Planning and Evaluation Service on commissioning and managing similar evaluation studies in the future.
If, in the panel's assessment, additional data analyses are warranted, and if the Department of Education and the panel agree, the panel would, in the second phase of its study, commission and review the analyses.
The two studies reviewed are The National Longitudinal Study of the Evaluation of the Effectiveness of Services for Language-Minority Limited-English-Proficient Students and The Longitudinal Study of Immersion Strategy, Early-exit and Late-exit Transitional Bilingual Education Programs for Language-Minority Children. Throughout the report the panel has adopted the words Longitudinal Study to describe the first study and Immersion Study to describe the second study. These terms are misleading, but they were the language that the Department of Education adopted in its description of the studies. For the record, the panel notes that both of the studies are longitudinal and that the “immersion” study does not have a real immersion component.
The panel's conclusions and recommendations are presented in Chapter 6. This is a very short chapter and readers who have a limited amount of time may wish to turn there first.
The panel first met in March of 1991, shortly after the final report for the Immersion Study was made available. Subsequent meetings were held in June and December of 1991. This report is the result of the panel's deliberations.
At the outset we thank the panel for its dedication to this project and its willingness to work in a very short time frame.
Many people made generous contributions to the work of the panel. We thank the many staff members of the U.S. Department of Education who attended our meetings, answered our innumerable questions, and provided mountains of reports, interim reports and background material. In particular, the efforts of James English and David Moguel were invaluable. The contractors for the two studies, Development Associates and Research Triangle Institute for the Longitudinal Study and Aguirre International for the Immersion Study, attended our first meeting, provided us with background materials and responded to many questions. Graham Burkheimer of Research Triangle Institute, Malcolm Young of Development Associates, and David Ramirez of Aguirre International were especially helpful. Wallace Lambert of McGill University shared with the panel his experiences in the Canadian immersion studies; his discussions helped to set the stage for many of our own deliberations.
The Departments of Statistics and of Academic Computing and Media at Carnegie Mellon University provided release time to allow Michael Meyer to direct the study and also provided excellent computing support.
Miron Straf, director of Committee on National Statistics, was always available to answer our questions and to assist us through the operations of the National Research Council.
The Committee on National Statistics provided valuable input into the report, and in particular encouraged us to expand the scope of our comments to the general domain of educational research.
Finally, we would like to thank the staff of the Committee on National Statistics, especially our senior project assistant, Michele Conrad. Michele kept the panel informed, organized meetings, provided editorial assistance, suffered our changing schedules, and still managed to keep a cheery disposition. The report would have taken a lot longer to complete without her assistance.
Stephen E. Fienberg, Chair
Michael M. Meyer, Study Director
Panel to Review Evaluation Studies of Bilingual Education