5
Comparison of American Community Survey Estimates and State Counts

The previous chapters have described various aspects of the American Community Survey (ACS) and have documented state policies, practices, and criteria that affect the counts of English language learner (ELL) students that are reported by state education agencies. It is readily apparent that these two allowable sources of data for use in allocating Title III funds to states have distinct strengths and weaknesses. In this chapter, we first briefly discuss the concepts and methods that underlie the two counts. We then consider in depth the differences between and the ACS estimates and state-provided counts on several dimensions.

CONCEPTUAL DIFFERENCES IN THE TWO SOURCES

The ACS estimates and state-provided counts of ELL students are two very different mechanisms for determining the number of school-age children in a state likely to have difficulty with English. As shown in Table 5-1, they differ along a number of dimensions.

The ACS is an indirect and subjective measure in that a parent or other adult household member provides an assessment for each child in the home. Since the question only asks about spoken English, it focuses on a single modality, and no context for English use is specified. The respondent may be considering the child’s proficiency with English in any number of settings (i.e., family life, community, social, academic), and the child may have different levels of proficiency in different settings. However, the ACS questions and criteria are consistent across states.

In contrast, the state-provided counts are based on direct, relatively objective measures of students’ English language proficiency. The counts are based on comprehensive processes established by state and local education agencies that consider



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 103
5 Comparison of American Community Survey Estimates and State Counts The previous chapters have described various aspects of the American Commu - nity Survey (ACS) and have documented state policies, practices, and criteria that affect the counts of English language learner (ELL) students that are reported by state education agencies. It is readily apparent that these two allowable sources of data for use in allocating Title III funds to states have distinct strengths and weaknesses. In this chapter, we first briefly discuss the concepts and methods that underlie the two counts. We then consider in depth the differences between and the ACS estimates and state-provided counts on several dimensions. CONCEPTUAL DIFFERENCES IN THE TWO SOURCES The ACS estimates and state-provided counts of ELL students are two very different mechanisms for determining the number of school-age children in a state likely to have difficulty with English. As shown in Table 5-1, they differ along a number of dimensions. The ACS is an indirect and subjective measure in that a parent or other adult household member provides an assessment for each child in the home. Since the question only asks about spoken English, it focuses on a single modality, and no context for English use is specified. The respondent may be considering the child’s proficiency with English in any number of settings (i.e., family life, community, social, academic), and the child may have different levels of proficiency in different settings. However, the ACS questions and criteria are consistent across states. In contrast, the state-provided counts are based on direct, relatively objective measures of students’ English language proficiency. The counts are based on com - prehensive processes established by state and local education agencies that consider 103

OCR for page 103
104 ALLOCATING FEDERAL FUNDS TABLE 5-1 Differences Between the ACS Estimates and State-Provided Counts of ELL Students Type of Difference ACS Estimate State-Provided Count Age Range 5-21 years of age Not specified (elementary and secondary school-aged population is usually defined as 5-18 years of age) School Enrollment Enrollment status not specified Newly and continually enrolled in (i.e., includes public and elementary and secondary schools private schools) for which Consolidated State Performance Reports are submitted by state education agencies (i.e., public schools including charter schools) Assessment Method Single question regarding Comprehensive assessment that spoken English ability incorporates information from multiple sources Mode of Response Indirect and subjective Direct evaluation based on a measure, based on the response student’s performance in acquiring of a parent (or other adult English proficiency in the household) to a single question Modality(ies) Assessed Speaking Speaking, listening, reading, and writing Context Assessed Not specified: likely to be Classroom setting community and family setting Basis for Distinguishing Single national cut score State- or local-determined criteria Proficient from Not Proficient Comparability Across States Item is identically presented States use different assessments, across the nation; estimates procedures, cut scores, and criteria; based on a uniform estimates based on different methodology across the states methodologies language proficiency across multiple modalities (listening, speaking, reading, and writing). The measures of language proficiency explicitly address both academic and social contexts. Unlike the ACS estimates, the information from the states varies because the policies, practices, and criteria used by the states are not uniform. CONCLUSION 5-1 The criteria used by the states for counts of English language learner students are more conceptually sound than the criteria on which American Community Survey (ACS) estimates are based. However, the policies, practices, and criteria used by the states differ from state to

OCR for page 103
105 COMPARISON OF COMMUNITY SURVEY ESTIMATES AND STATE COUNTS state, while the ACS provides estimates on the basis of a uniform methodol- ogy across the country. Despite their differences, the ACS estimates and state counts represent concep - tually similar entities—both measure the number of school-age children in the state that have not mastered English. Thus, some level of correspondence between the two measures would be expected. We conducted a series of analyses to evaluate the consistency of the ACS and state-provided percentages of ELL students. In order to facilitate comparisons be - tween the ACS estimates and state-provided counts we limit the ACS population to those aged 5-18 and only to those enrolled in public school. It is important to point out that by limiting the ACS estimate to this comparison group, we have created an ACS-based variable that is more limited than the legal definition of ELL students used by the U.S. Department of Education (DoEd). As detailed in Chapters 2 and 4, there are two ways to calculate the percentages: the number of ELL children in the state as a percentage of the total number of ELL children in the country, which is the state’s share of ELL children; and the proportion that ELL students constitute of the total number of enrolled students, which is the rate of ELL students. We also conducted a series of multiple regression analyses to evaluate the correspondence between the ACS and state estimates. In these analyses, we focus on rates, rather than shares, in order to assess the degree of consistency of the two data sources in a manner that is relatively independent of state population. That is, analyses that focus on state counts or shares are dominated by the agreement between the ACS and state-provided numbers for some states, suggesting only that certain states (notably, California and Texas) are large and others are small, a trivial finding that provides little information about how well the two measures agree on estimation of ELL students. COMPARISON OF SHARES OF ELL STUDENTS In this section we compare the state shares (of Title III funding) based on ACS estimates with those based on state-provided counts. Since the funding allocations are based on each state’s share of ELL students in the country, this analysis allows us to evaluate how the allocations would be affected on the basis of which measure was used, as well as the ways that the measures would result in different funding deci - sions. We compare the shares in three ways: (1) the percentage shares themselves, (2) the ratio of the shares, and (3) the absolute differences in the shares across the states. State Percentage Shares Table 5-2 shows each state’s share of ELL students based on the two data sources. The first four columns on the left-hand side of the table show the shares based on the ACS estimates. Included are 1-year estimates for 2006, 2007, and 2008

OCR for page 103
TABLE 5-2 Shares of ELL Students Based on ACS and State-Provided Counts (in percentage) 106 State-Provided Count ACS Estimate All ELLa Tested Not, Proficientb State 2006 2007 2008 2006-2008 2006-2007 2007-2008 2008-2009 2007-2008 2008-2009 Alabama 0.43 0.40 0.40 0.40 0.43 0.46 0.43 0.53 0.40 Alaska 0.17 0.17 0.15 0.17 0.48 0.37 0.27 0.46 0.44 Arizona 3.66 3.90 3.69 3.74 3.56 3.31 2.79 4.15 2.86 Arkansas 0.45 0.48 0.37 0.44 0.55 0.57 0.61 0.77 0.80 Californiac 29.12 28.31 27.25 28.12 36.35 34.32 33.68 29.37 28.63c Colorado 1.70 1.74 1.66 1.70 2.10 1.89 1.98 1.75 2.70 Connecticut 0.88 0.65 0.58 0.71 0.61 0.66 0.66 0.61 0.54 Delaware 0.17 0.16 0.15 0.16 0.15 0.16 0.16 0.10 0.16 District of Columbia 0.08 0.06 0.07 0.06 0.11 0.11 0.13 0.15 0.15 Florida 5.54 5.51 5.25 5.38 5.47 5.11 5.03 5.42 5.16 Georgia 2.10 2.20 1.97 2.09 1.73 1.77 1.80 2.05 2.01 Hawaii 0.30 0.22 0.38 0.29 0.37 0.41 0.41 0.49 0.50 Idaho 0.28 0.25 0.32 0.29 0.39 0.37 0.39 0.46 0.34 Illinois 4.52 4.81 4.74 4.67 4.03 4.20 4.55 3.56 3.24 Indiana 0.91 0.81 0.89 0.88 0.99 1.02 1.02 1.26 1.33 Iowa 0.44 0.40 0.40 0.43 0.42 0.43 0.45 0.47 0.45 Kansas 0.51 0.49 0.54 0.52 0.67 0.70 0.76 0.93 0.86 Kentucky 0.31 0.39 0.45 0.41 0.25 0.28 0.32 0.38 0.41 Louisiana 0.26 0.38 0.39 0.36 0.20 0.25 0.28 0.38 0.33 Maine 0.14 0.11 0.08 0.12 0.09 0.09 0.09 0.10 0.12 Maryland 0.96 1.10 0.98 1.03 0.80 0.89 0.89 0.65 1.07 Massachusetts 1.71 1.57 1.61 1.64 1.26 1.23 1.09 0.86 1.16 Michigan 1.56 1.51 1.40 1.49 1.63 1.14 1.35 1.86 1.23 Minnesota 1.18 1.22 1.35 1.27 1.49 1.35 1.37 1.17 1.67 Mississippi 0.18 0.16 0.16 0.20 0.12 0.12 0.15 0.04 0.18

OCR for page 103
Missouri 0.56 0.69 0.60 0.64 0.52 0.42 0.36 0.40 0.52 Montana 0.06 0.05 0.04 0.06 0.16 0.15 0.10 0.05 0.03 Nebraska 0.41 0.37 0.39 0.39 0.42 0.42 0.41 0.40 0.38 Nevada 1.02 1.15 1.38 1.19 1.64 1.04 1.69 2.08 2.12 New Hampshire 0.06 0.11 0.09 0.09 0.07 0.07 0.09 0.09 0.11 New Jersey 2.55 2.29 2.57 2.48 1.27 1.20 1.20 1.28 1.30 New Mexico 0.97 0.84 0.74 0.83 1.42 1.35 1.20 1.47 1.40 New York 6.63 6.36 6.80 6.61 4.57 4.65 4.09 5.59 5.31 North Carolina 2.06 1.88 2.25 2.04 2.05 2.82 2.53 3.33 2.97 North Dakota 0.04 0.07 0.06 0.05 0.06 0.10 0.09 0.14 0.06 Ohio 1.03 0.93 1.02 1.00 0.68 0.77 0.81 0.97 1.00 Oklahoma 0.45 0.53 0.47 0.50 0.89 0.83 0.85 0.97 0.91 Oregon 1.25 1.23 1.14 1.21 1.44 1.37 1.40 1.81 1.77 Pennsylvania 1.48 1.52 1.60 1.53 1.06 1.02 1.06 1.18 1.02 Rhode Island 0.19 0.29 0.27 0.25 0.23 0.16 0.16 0.19 0.21 South Carolina 0.51 0.57 0.51 0.56 0.70 0.63 0.70 0.86 0.89 South Dakota 0.05 0.05 0.09 0.07 0.08 0.09 0.08 0.09 0.09 Tennessee 0.61 0.67 0.66 0.68 0.54 0.57 0.61 0.63 0.59 Texas 16.98 17.75 18.55 17.63 11.69 15.31 15.97 15.09 15.25 Utah 0.75 0.77 0.71 0.75 1.13 1.03 0.99 0.91 0.88 Vermont 0.04 0.03 0.03 0.03 0.04 0.03 0.03 0.04 0.04 Virginia 1.57 1.22 1.45 1.42 2.01 1.86 1.93 1.73 2.67 Washington 2.04 2.31 2.35 2.22 1.98 1.78 1.84 2.21 2.22 West Virginia 0.12 0.09 0.09 0.11 0.05 0.05 0.04 0.04 0.03 Wisconsin 0.96 1.18 0.86 1.01 0.96 0.97 1.06 0.42 1.43 Wyoming 0.04 0.03 0.05 0.04 0.07 0.05 0.05 0.06 0.06 Total Count in the U.S. 2,491,160 2,482,420 2,433,845 2,462,330 4,289,046 4,525,892 4,499,072 3,052,983 3,131,685 aThe total number of ELL students was obtained from the EDEN database. bThe number of tested, not proficient students was computed for each state from the state Consolidated State Performance Reports by subtracting the number of all LEP students proficient or above on state annual ELP assessments (1.6.3.1.2) from the number of all LEP students tested on state annual ELP assessments (1.6.3.1.1). 107 cCounts for California for 2008-2009 were unavailable, percentage is based on the 2007-2008 count.

OCR for page 103
108 ALLOCATING FEDERAL FUNDS and the 3-year estimate across these years. The table includes two types of shares calculated from the state-provided counts. Three of the columns show the state- provided counts of all ELL students for the 2006-2007, 2007-2008, and 2008-2009 school years. The other two columns show the shares based on the state-provided counts of ELL students who were determined to be not proficient on the English language proficiency (ELP) test for the 2007-2008 and 2008-2009 school years. Ratios of the Shares To help compare the percentages from the two measures, we calculated the ratio of the share based on the ACS estimate to the share based on the state-provided counts. These ratios are shown in Table 5-3. The tables include the ratios of the ACS 1-year estimate to the state counts for each school year, as well as the ratios of the ACS 3-year estimate to the most recent state school year data. Ratios higher than 1.00 indicate that the share based on the ACS estimate was higher than the share based on the state-provided count, and ratios that are less than 1.00 indicate that the state-provided count was higher than the ACS estimate. Scan - ning the ratios across the time spans and the type of state-provided counts reveals considerable consistency. That is, for a given state, the ratios were generally consis - tently above 1.00 (ACS estimate higher than state-provided count) or consistently below 1.00 (state-provided count higher than ACS estimate). It is difficult to discern any explanatory factors from this comparison. No patterns appear to be evident due to region of the country or type of proficiency test used. For example, about half of the states that used the ACCESS for ELLs test developed by the World-Class Instructional Design and Assessment Consortium (see Table 4-1 in Chapter 4) had ACS rates higher than the state rates and half were lower. Absolute Differences in the Shares To quantify the potential effects of the differences between the two data sources in terms of the distribution of Title III funds, we calculated the total absolute differ - ences between the shares based on ACS estimates and those based on state-provided counts. The differences are shown in Table 5-4. The left-hand side of the table shows the values for the differences between ACS estimates and the state-provided counts of all ELL students, for both 1-year and 3-year ACS estimates; the right-hand side of the table shows the values of the differences between ACS estimates and the state- provided counts of tested, not proficient students. This quantity varies from about 20 percent to 26 percent of the total allocation, depending on the years considered. Because every dollar moved is counted twice in this total (once when it is taken from a state with a reduced share and once when added to one with an increased share), it means that from 10 percent to 13 percent of the total dollars would be moved by switching from one allocation to another, a substantial change in allocations.

OCR for page 103
TABLE 5-3 Ratio of State Shares Based on ACS Estimate to Shares Based on State-Provided Counts All ELL Studentsa Tested, Not Proficient Studentsb ACS 2006 to ACS 2007 to ACS 2008 to ACS 2006-2008 to ACS 2007 to ACS 2008 to ACS 2006-2008 to State State 2006-2007 State 2007-2008 State 2008-2009 State 2008-2009 State 2007-2008 State 2008-2009 State 2008-2009 Alabama 1.00 0.87 0.91 0.91 0.76 0.99 0.99 Alaska 0.35 0.47 0.56 0.65 0.37 0.34 0.39 Arizona 1.03 1.18 1.32 1.34 0.94 1.29 1.31 Arkansas 0.81 0.84 0.61 0.72 0.62 0.47 0.55 Californiac 0.80 0.82 0.81 0.83 0.96 0.95 0.98c Colorado 0.81 0.93 0.84 0.86 1.00 0.61 0.63 Connecticut 1.43 0.98 0.87 1.08 1.07 1.07 1.33 Delaware 1.10 1.01 0.93 1.02 1.60 0.93 1.02 District of Columbia 0.71 0.56 0.51 0.46 0.41 0.45 0.40 Florida 1.01 1.08 1.05 1.07 1.02 1.02 1.04 Georgia 1.21 1.25 1.10 1.16 1.07 0.98 1.04 Hawaii 0.83 0.54 0.92 0.71 0.45 0.76 0.58 Idaho 0.72 0.68 0.80 0.74 0.54 0.94 0.87 Illinois 1.12 1.15 1.04 1.03 1.35 1.46 1.44 Indiana 0.92 0.79 0.88 0.87 0.65 0.67 0.66 Iowa 1.04 0.93 0.89 0.96 0.86 0.89 0.96 Kansas 0.76 0.69 0.71 0.69 0.52 0.62 0.61 Kentucky 1.22 1.38 1.40 1.27 1.04 1.11 1.01 Louisiana 1.27 1.52 1.41 1.30 1.02 1.20 1.11 Maine 1.68 1.19 0.88 1.30 1.03 0.68 1.01 Maryland 1.20 1.23 1.10 1.16 1.70 0.92 0.96 Massachusetts 1.36 1.28 1.48 1.50 1.83 1.39 1.41 Michigan 0.96 1.32 1.04 1.10 0.81 1.15 1.22 Minnesota 0.79 0.90 0.99 0.93 1.04 0.81 0.76 Mississippi 1.53 1.34 1.07 1.37 4.24 0.86 1.11 Missouri 1.08 1.63 1.66 1.77 1.72 1.16 1.23 Montana 0.35 0.34 0.39 0.61 0.99 1.19 1.84 109 continued

OCR for page 103
TABLE 5-3 Continued 110 All ELL Studentsa Tested, Not Proficient Studentsb ACS 2006 to ACS 2007 to ACS 2008 to ACS 2006-2008 to ACS 2007 to ACS 2008 to ACS 2006-2008 to State State 2006-2007 State 2007-2008 State 2008-2009 State 2008-2009 State 2007-2008 State 2008-2009 State 2008-2009 Nebraska 0.96 0.87 0.96 0.96 0.92 1.02 1.02 Nevada 0.62 1.10 0.82 0.71 0.55 0.65 0.56 New Hampshire 0.78 1.55 0.98 0.99 1.18 0.83 0.84 New Jersey 2.01 1.90 2.13 2.06 1.79 1.98 1.91 New Mexico 0.68 0.62 0.62 0.69 0.57 0.53 0.59 New York 1.45 1.37 1.66 1.62 1.14 1.28 1.25 North Carolina 1.01 0.67 0.89 0.81 0.56 0.76 0.69 North Dakota 0.69 0.66 0.63 0.56 0.49 0.92 0.82 Ohio 1.51 1.20 1.27 1.23 0.96 1.03 1.00 Oklahoma 0.51 0.64 0.56 0.59 0.55 0.52 0.55 Oregon 0.86 0.90 0.82 0.87 0.68 0.65 0.69 Pennsylvania 1.39 1.48 1.51 1.45 1.29 1.58 1.50 Rhode Island 0.82 1.85 1.71 1.59 1.55 1.31 1.22 South Carolina 0.73 0.91 0.73 0.81 0.66 0.57 0.63 South Dakota 0.71 0.56 1.12 0.91 0.56 1.00 0.81 Tennessee 1.14 1.19 1.09 1.11 1.06 1.12 1.14 Texas 1.45 1.16 1.16 1.10 1.18 1.22 1.16 Utah 0.66 0.75 0.72 0.76 0.85 0.81 0.85 Vermont 0.96 0.88 0.94 0.98 0.72 0.81 0.84 Virginia 0.78 0.65 0.75 0.73 0.70 0.54 0.53 Washington 1.03 1.30 1.28 1.21 1.05 1.06 1.00 West Virginia 2.35 1.83 2.46 3.03 2.51 3.24 4.01 Wisconsin 1.00 1.21 0.81 0.95 2.79 0.60 0.71 Wyoming 0.56 0.54 0.95 0.83 0.47 0.81 0.71 aThe total number of ELL students was obtained from the Education Data Exchange Network (EDEN) database. bThe number of tested, not proficient students was computed for each state from the state Consolidated State Performance Reports by subtracting the number of all LEP (ELL) students proficient or above on state annual ELP assessments (1.6.3.1.2) from the number of all LEP (ELL) students tested on state annual ELP assessments (1.6.3.1.1). cCounts for California for 2008-2009 were unavailable; the percentage is based on the 2007-2008 count.

OCR for page 103
111 TABLE 5-4 Total Absolute Difference Between Shares Based on ACS Estimates and Shares Based on State-Provided Counts ALL ELL Students Tested, Not Proficient Students ACS ACS ACS ACS 2006 ACS 2007 ACS 2008 2006-2008 2006-08 2006-2008 Type of ACS and State and State and State and State and State and State Estimate 2006-2007 2007-2008 2008-2009 2006-2007 2007-2008 2008-2009 1-year 23.68 21.13 21.57 N/A 18.21 20.55 3-year 25.94 20.44 19.73 N/A 17.94 18.66 COMPARISON OF RATES OF ELL STUDENTS As noted above, comparison of state rates removes the simple effect of size from the analyses and thereby focuses attention on differences in measurement. State Rates Table 5-5 shows each state’s rate of ELL students based on the two data sources. The first four columns on the left-hand side of the table show the rates based on the ACS estimates, including 1-year estimates for 2006, 2007, and 2008 and the 3-year estimate across these years. For the state-provided counts, two types of rates calculated are shown: the rates based on state-provided counts of all ELL students for the 2006-2007, 2007-2008, and 2008-2009 school years and the rates based on the state-provided counts of tested, not proficient students for the 2007-2008 and 2008-2009 school years. The rates derived from the ACS were lower than the rates derived from state- provided counts in all but two states (New Jersey and West Virginia), when state- provided counts were based on all ELL students. When the state-provided count was based on the number of tested, not proficient students, the ACS estimates were consistently lower in five states (Illinois, Massachusetts, New Jersey, Pennsylvania, and West Virginia). In the most recent period, the average percentage of ELL stu- dents for the nation was about 5 percent for the ACS and about 9 percent for the state-provided rates; for the state-provided rate of tested, not proficient students, the rate was 6 percent. Ratio of the Rates To compare the percentages from the two measures, we calculated the ratio of the rate based on the ACS estimate to the rate based on each of the state-provided counts. These ratios are shown in Table 5-6. The left-hand side of the table shows

OCR for page 103
TABLE 5-5 Rate of ELL Students by State Based on ACS Estimates and State-Provided Counts (in percentage) 112 State-Provided Count ACS Estimate All ELLa Tested, Not Proficientb State 2006 2007 2008 2006-2008 2006-2007 2007-2008 2008-2009 2007-2008 2008-2009 Alabama 1.44 1.36 1.31 1.33 2.47 2.81 2.62 2.16 1.68 Alaska 3.34 3.56 3.09 3.53 15.66 12.84 9.21 10.82 10.61 Arizona 8.50 8.82 8.01 8.40 14.30 13.77 11.55 11.65 8.23 Arkansas 2.40 2.54 1.94 2.32 4.96 5.41 5.77 4.93 5.24 Californiac 11.34 11.13 10.54 11.00 24.34 24.48 24.23 14.13 14.34c Colorado 5.55 5.53 5.14 5.37 11.32 10.64 10.86 6.65 10.34 Connecticut 3.90 2.91 2.55 3.16 4.58 5.26 5.25 3.25 2.98 Delaware 3.51 3.20 2.93 3.26 5.44 5.92 5.73 2.52 3.99 District of Columbia 2.91 2.60 2.57 2.34 6.47 6.54 8.52 5.94 6.79 Florida 5.35 5.33 4.99 5.16 8.78 8.68 8.59 6.20 6.15 Georgia 3.24 3.31 2.89 3.15 4.55 4.85 4.89 3.79 3.80 Hawaii 4.40 3.40 5.60 4.36 8.66 10.38 10.34 8.39 8.72 Idaho 2.69 2.30 2.84 2.70 6.25 6.13 6.42 5.20 3.83 Illinois 5.41 5.72 5.57 5.54 8.16 8.99 9.66 5.15 4.79 Indiana 2.24 1.98 2.12 2.13 4.07 4.42 4.37 3.66 3.97 Iowa 2.24 2.07 2.03 2.22 3.75 4.01 4.17 2.93 2.91 Kansas 2.75 2.60 2.85 2.81 6.16 6.78 7.24 6.08 5.73 Kentucky 1.18 1.48 1.67 1.54 1.58 1.94 2.18 1.73 1.91 Louisiana 0.96 1.44 1.40 1.29 1.28 1.68 1.82 1.68 1.49 Maine 1.81 1.35 1.05 1.55 1.90 2.06 2.19 1.60 1.97 Maryland 2.85 3.25 2.87 3.02 4.03 4.78 4.75 2.33 3.97 Massachusetts 4.51 4.16 4.15 4.27 5.58 5.79 5.12 2.72 3.79 Michigan 2.27 2.23 2.06 2.18 4.05 3.04 3.67 3.36 2.31 Minnesota 3.52 3.67 4.07 3.81 7.60 7.31 7.35 4.28 6.27 Mississippi 0.87 0.79 0.74 0.98 1.01 1.10 1.33 0.23 1.15

OCR for page 103
Missouri 1.57 1.88 1.63 1.75 2.43 2.08 1.78 1.33 1.78 Montana 0.94 0.86 0.66 1.02 4.84 4.71 3.21 1.10 0.74 Nebraska 3.56 3.22 3.40 3.42 6.33 6.57 6.29 4.20 4.12 Nevada 5.99 6.45 7.62 6.79 16.61 10.96 17.53 14.82 15.31 New Hampshire 0.70 1.35 1.09 1.09 1.55 1.59 2.06 1.41 1.69 New Jersey 4.68 4.26 4.65 4.56 3.92 3.94 3.92 2.82 2.94 New Mexico 7.03 6.26 5.43 6.14 18.50 18.60 16.34 13.64 13.27 New York 5.72 5.59 5.87 5.72 6.98 7.61 6.71 6.17 6.06 North Carolina 3.54 3.19 3.73 3.47 6.07 8.56 7.65 6.82 6.26 North Dakota 0.95 1.70 1.44 1.27 2.48 4.89 4.29 4.48 2.03 Ohio 1.43 1.29 1.42 1.38 1.59 1.92 2.00 1.62 1.72 Oklahoma 1.84 2.17 1.89 2.02 5.96 5.88 5.90 4.59 4.41 Oregon 5.47 5.40 4.90 5.28 11.01 10.98 11.19 9.79 9.82 Pennsylvania 2.05 2.11 2.22 2.13 2.43 2.57 2.69 2.00 1.80 Rhode Island 3.11 4.71 4.52 4.12 6.62 4.84 6.47 3.89 4.48 South Carolina 1.84 2.03 1.77 1.99 4.26 3.98 4.38 3.67 3.89 South Dakota 1.06 0.99 1.66 1.38 2.72 3.47 2.84 2.34 2.23 Tennessee 1.62 1.76 1.71 1.76 2.35 2.66 2.82 2.01 1.91 Texas 9.69 10.02 10.07 9.89 10.90 14.82 15.12 9.85 10.05 Utah 3.54 3.52 3.15 3.42 9.25 8.12 7.94 4.81 4.94 Vermont 1.03 0.77 0.85 0.87 1.83 1.55 1.62 1.29 1.31 Virginia 3.28 2.51 2.94 2.92 7.08 6.85 7.04 4.30 6.76 Washington 4.94 5.64 5.60 5.38 8.26 7.83 7.98 6.54 6.71 West Virginia 1.13 0.87 0.79 0.99 0.80 0.83 0.57 0.41 0.30 Wisconsin 2.82 3.39 2.45 2.91 4.71 5.01 5.48 1.47 5.12 Wyoming 1.17 0.82 1.34 1.20 3.53 2.77 2.61 2.17 2.13 United States 5.10 5.10 5.00 5.10 8.70 9.20 9.10 6.19 6.37 aThe total number of ELL students was obtained from the EDEN database. bThe total number of tested, not proficient was computed for each state from the state Consolidated State Performance Reports by subtracting the number of all LEP students proficient or above on state annual ELP assessments (1.6.3.1.2) from the number of all LEP students tested on state annual ELP assessments (1.6.3.1.1). cCounts for California for 2008-2009 were unavailable; percentage is based on the 2007-2008 count. 113

OCR for page 103
122 ALLOCATING FEDERAL FUNDS TABLE 5-8 Analysis of Using ACS 3-Year Estimate and Other Variables to Predict State-Provided Rate of Tested, Not Proficient ELL Students 2007-2008 2008-2009 R2 Model 1 Adjusted .66** .66** Regression Coefficients (Standard Error): ACS 2006-2008 1.300** (.1300) 1.2871 (.1293)** Intercept .0034 (.0052) .0057 (.0051) Adjusted R2 Model 2 .66** .66** Regression Coefficients (Standard Error): ACS 2006-2008 1.320 (.1773)** 1.2752 (.1763)** Percent immigrants –0.1566 (.920) .0918 (.9152) Intercept .0039 (.0060) .0052 (.0062) Adjusted R2 Model 3 .67** .66* Regression Coefficients (Standard Error): ACS 2006-2008 0.8173 (.4597) 1.0598 (.4614)* Percent immigrants in poverty 1.612 (1.4722) .7589 (1.4778) Intercept .0040 (.0052) .0060 (.0053) Adjusted R2 Model 4 .67** .66* Regression Coefficients (Standard Error): ACS 2006-2008 0.9594 (.2950)** 0.9758(.2941)** Percent unauthorized immigrants 0.5396 (.4201) .4930 (.4188) Intercept .0049 (.0053) .0071(.0053) Adjusted R2 Model 5 .71** .68** Regression Coefficients (Standard Error): ACS 2006-2008 0.8221 (.2102)** 0.9773 (.2185)** Percent unauthorized Mexicans 1.1009 (.3943)** .7134 (.4099) Intercept .0090 (.0053)* .0094 (.0055) Adjusted R2 Model 6 .70** .67** Regression Coefficients (Standard Error): ACS 2006-2008 0.9105 (.1862)** 1.0807 (.1954)** States with high percent unauthorized .0387 (.0140)** .0205 (.0146) Intercept .0127 (.0059)* .0107 (.0062)* Adjusted R2 Model 7 .71** .67** Regression Coefficients (Standard Error): ACS 2006-2008 0.7279 (.2154)** 0.9395 (.2290)** Percent unauthorized Mexican .7306 (.4537) .5649 (.4822) States with high percent unauthorized .0253 (.0160) .0102 (.0170) Intercept .0132 (.0058)* .0110 (.0062) Adjusted R2 Model 8 .66** .66** Regression Coefficients (Standard Error): ACS 2006-2008 1.2327(.1449)** 1.2208 (.1444)** ACCESS user –.0080(.0068) –.0067 (.0067) ELDA user –.0056 (.0095) –.0074 (.0094) Intercept .0093 (.0074) .0114 (.0074)

OCR for page 103
123 COMPARISON OF COMMUNITY SURVEY ESTIMATES AND STATE COUNTS TABLE 5-8 Continued 2007-2008 2008-2009 R2 Model 9 Adjusted .65** .64** Regression Coefficients (Standard Error): ACS 2006-2008 0.7988 (.4795) 1.0263 (.4833)* Percent immigrants –.3598 (.9523) –.1165 (.9598) Percent immigrants in poverty 1.5989 (1.5016) .6992 (1.5134) ACCESS user –.0083 (.0070) –.0068 (.0070) ELDA user –.0057 (.0098) –.0074 (.0099) Intercept .0111 (.0084) .0120 (.0084) NOTES: *p < .05; **p < .01. All R2 values were adjusted for the sample size (n = 51). on the ACS language item appears to best approximate school assessments of English language learner status. It should be noted that the small number of units (50 states and the District of Columbia) limited the number of variables that could be considered. Furthermore, the concentration of ELL populations in relatively few states further limited our ability to infer systematic correlates of the discrepancies between the data sources. Within-State Analyses We also conducted a series of regression analyses to examine the relationships between ACS estimates and state-provided counts for school districts (local educa - tion agencies or LEAs) within each state. The purpose of this analysis was to assess how well the ACS and state-provided numbers tracked each other under a consistent set of procedures, criteria, and tests, that is, those of a single state. We obtained the 3-year (2006-2008) ACS estimates and the state-provided (2007-2008) counts of all ELL students for each unified school district for which they were available, which limited us to school districts with total populations of at least 20,000 (due to ACS release restrictions for small areas). Also excluded were several states for which LEA- level data were unavailable (California, New Jersey, and South Dakota). This analysis could be conducted only with rates based on state counts of all ELL students, since LEA counts of tested, not proficient students were not available. We formed the rate for each district (that is, we divided each of the counts by the number of K-12 students enrolled in public schools in the state). For smaller units of analyses, such as most school districts, the sampling vari- ability of ACS estimates of rates is generally greater than for states. Simple sample correlations would be attenuated by this error, underestimating the strength of the underlying relationship between the ACS and state-provided measures. We therefore used hierarchical models that adjust for the sampling variability of the ACS data to estimate this relationship. For these analyses, the dependent variable was the ACS

OCR for page 103
124 ALLOCATING FEDERAL FUNDS estimate of the school district’s ELL rate, and the explanatory variable was the state- provided estimate of the district rate. (Making the ACS rate the dependent variable facilitated specification of a hierarchical model in which the difference between the ACS estimate and its linear prediction from the state-reported rate is modeled as the sum of two random effects, one for ACS sampling error with known variance, and one for the discrepancy between the ACS and the state rate with variance to be estimated.) Table 5-9 shows descriptive information for each of the states included in the analysis. The first two columns show the school enrollment and the number of uni - fied districts in the state. The third column shows the overall rate of ELL students in the state based on state-provided information. The next three columns provide distributional information about the LEA rates within the state (based on the state- provided information): the average rate across the districts and the 20th and 80th percentiles of the LEA rates in the state. The seventh column shows the overall rate of ELL students in the state based on the ACS information. The eighth column presents the ratio of the ACS rate to the state-provided rate. The final column shows the sample correlation of the rates based on ACS estimates and state-provided counts for the unified school districts within a state. For instance, the correlation between the two sets of rates for the 58 unified school districts in Alabama was 0.697. The correlation is labeled “unadjusted” because it has not been corrected for sampling error associated with the ACS estimates. Table 5-10 presents the results of the within-state regressions in states with at least 10 eligible LEA units, incorporating a correction for sampling error in the ACS estimates. The first four columns show the results from regressions that include the intercept in the model. The first two columns show, respectively, the regression coefficients for the intercept and for the rate based on the state-provided estimate. The third column shows the root mean square residual error (RMSE) of the model, which quantifies the amount by which the ACS estimates by LEA vary around the regression line. The fourth column shows the correlations after adjustment for sam - pling error. The median of these estimated correlation coefficients is 0.949, and the coefficient exceeds 0.90 in 30 of 41 states, although there are also a few states for which these LEA-level correlations are relatively low. The fifth and sixth columns show parallel results (regression coefficients and RMSE) from the regressions that did not include the intercept in the model. The final column is the ratio of the errors from the two models (with and without intercepts). This ratio is usually not far from 1.0 (except in a few states where the denominator is very small due to an extremely good model fit), suggesting that the no-intercept (proportional) model fits the data almost as well as the unconstrained linear model. As noted previously, the proportional model implies that ACS-based and state-data-based allocations would be equivalent. In general, the results suggest very good consistency between the ACS and state-provided numbers within states. This greater consistency, relative to similar models fitted at the state level, might be attributed to two features of the within-

OCR for page 103
TABLE 5-9 Descriptive Summaries of LEA-Level Data on Rate of ELL Students, by State State Counts State ELL Rates ACS School Number Overall Rate Mean of LEAs 20%-tile 80%-tile Overall Rate Ratio of Unadjusted State Enrollment of LEAs (%) (%) (%) (%) (%) ACS/State Correlation Alabama 579,913 58 3.0 2.8 0.4 4.7 1.3 0.44 0.697 Alaska 93,838 5 7.4 5.8 2.4 10.6 2.6 0.34 0.207 Arizona 872,395 72 14.2 14.5 3.7 26.2 9.6 0.68 0.798 Arkansas 227,292 30 8.7 5.9 0.6 7.3 3.1 0.36 0.918 Colorado 684,657 35 11.3 11.9 2.5 21.4 5.8 0.51 0.920 Connecticut 378,744 56 7.3 5.3 1.4 10.1 3.9 0.54 0.877 Delaware 102,396 13 6.7 5.8 2.1 9.0 3.3 0.49 0.529 District of Columbia 57,877 1 7.1 7.1 7.1 7.1 2.3 0.33 NA Florida 2,619,362 54 8.8 5.2 0.9 9.4 5.2 0.59 0.682 Georgia 1,487,247 97 5.2 3.5 0.7 5.4 3.3 0.64 0.885 Hawaii 179,897 1 10.4 10.4 10.4 10.4 4.4 0.42 NA Idaho 180,200 20 5.2 5.5 0.3 11.3 2.4 0.46 0.904 Illinois 1,519,448 202 11.4 7.2 0.8 10.6 6.4 0.56 0.769 Indiana 705,862 87 5.5 4.9 1.0 7.5 2.4 0.44 0.776 Iowa 220,538 29 6.0 4.8 0.8 7.2 2.5 0.42 0.646 Kansas 272,573 28 9.6 9.2 1.6 12.9 4.1 0.42 0.907 Kentucky 463,556 54 2.4 1.4 0.2 1.8 1.6 0.67 0.647 Louisiana 606,547 49 1.8 1.2 0.1 1.6 1.4 0.76 0.418 Maine 45,917 12 5.8 4.3 0.4 6.3 3.7 0.63 0.797 Maryland 843,426 23 4.8 2.6 0.6 3.4 3.0 0.63 0.862 Massachusetts 639,309 110 8.1 4.8 0.6 9.3 5.3 0.65 0.819 Michigan 830,996 103 4.6 3.5 0.4 5.1 2.6 0.55 0.718 Minnesota 537,291 60 9.2 6.5 1.3 10.1 4.7 0.51 0.779 Mississippi 300,235 44 1.4 1.4 0.3 2.1 1.0 0.71 0.069 125 continued

OCR for page 103
TABLE 5-9 Continued 126 State Counts State ELL Rates ACS School Number Overall Rate Mean of LEAs 20%-tile 80%-tile Overall Rate Ratio of Unadjusted State Enrollment of LEAs (%) (%) (%) (%) (%) ACS/State Correlation Missouri 551,434 65 2.7 2.2 0.5 2.9 2.1 0.79 0.671 Montana 58,413 15 2.3 1.9 0.2 3.5 1.3 0.54 0.290 Nebraska 169,074 15 8.6 7.2 1.4 11.8 4.0 0.47 0.781 Nevada 419,488 8 10.9 12.5 10.3 13.9 6.9 0.63 –0.550 New Hampshire 86,972 17 2.8 1.7 0.3 2.6 1.6 0.56 0.407 New Mexico 270,081 22 18.8 18.3 6.0 33.8 6.6 0.35 0.658 New York 1,063,809 175 4.9 4.2 0.8 7.1 3.1 0.64 0.746 North Carolina 1,373,592 89 9.1 8.0 2.9 12.4 3.5 0.39 0.826 North Dakota 44,279 6 4.2 3.6 0.3 7.0 1.9 0.45 0.782 Ohio 1,026,648 151 2.9 2.1 0.3 2.8 1.6 0.56 0.726 Oklahoma 325,060 30 8.4 5.6 2.3 8.1 2.7 0.32 0.717 Oregon 430,730 45 12.2 11.4 2.2 18.7 5.9 0.48 0.921 Pennsylvania 107,022 18 4.5 2.6 0.2 5.3 3.1 0.70 0.841 Rhode Island 112,975 19 5.3 3.0 0.6 3.7 4.9 0.92 0.801 South Carolina 649,424 53 4.2 3.5 1.2 5.1 2.1 0.50 0.685 Tennessee 848,210 70 2.9 1.8 0.3 3.0 1.9 0.65 0.733 Texas 3,770,908 207 16.4 12.3 4.4 19.0 11.0 0.67 0.901 Utah 513,430 19 8.7 9.2 3.1 16.1 3.4 0.39 0.950 Virginia 1,125,896 77 7.2 4.3 0.5 5.7 3.0 0.42 0.738 Washington 837,945 78 7.5 6.4 1.5 9.9 5.4 0.72 0.859 West Virginia 239,577 32 0.9 0.6 0.0 0.7 1.0 1.06 –0.163 Wisconsin 498,915 66 7.3 4.6 0.7 7.1 3.8 0.52 0.729 Wyoming 43,609 6 1.8 2.2 0.5 4.5 1.3 0.73 0.951 NOTE: Data include only eligible LEAs, as described in the text.

OCR for page 103
TABLE 5-10 Results of Within-State Regressions Model with Intercept, State Data Rate No-Intercept Model Ratio of Intercept State Adjusted State Number RMSE Coefficient Coefficient RMSE Correlation Coefficient RMSE of LEAs Estimates Alabama 0.0026 0.2508 0.0036 0.9040 0.3084 0.0033 58 0.93 Arizona 0.0216 0.3852 0.0316 0.8433 0.4765 0.0346 72 1.09 Arkansas 0.0049 0.2747 0.0003 0.9999 0.3099 0.0003 30 1.11 Colorado 0.0029 0.4477 0.0004 1.0000 0.4693 0.0013 35 3.20 Connecticut 0.0043 0.4038 0.0065 0.9596 0.4602 0.0067 56 1.03 Delaware 0.0171 0.1587 0.0027 0.8904 0.3816 0.0069 13 2.56 Florida 0.0073 0.4762 0.0127 0.8737 0.5540 0.0134 54 1.06 Georgia 0.0044 0.5006 0.0043 0.9814 0.5723 0.0048 97 1.13 Idaho 0.0053 0.3292 0.0005 0.9997 0.3958 0.0036 20 7.73 Illinois 0.0139 0.3808 0.0189 0.8834 0.4739 0.0220 202 1.17 Indiana 0.0034 0.2750 0.0035 0.9778 0.3224 0.0033 87 0.92 Iowa 0.0095 0.1688 0.0066 0.8514 0.2623 0.0096 29 1.46 Kansas 0.0017 0.3394 0.0102 0.9675 0.3508 0.0102 28 1.00 Kentucky 0.0046 0.4069 0.0033 0.9360 0.5251 0.0021 54 0.65 Louisiana 0.0061 0.2929 0.0020 0.9192 0.4801 0.0047 49 2.38 Maine 0.0037 0.3091 0.0054 0.9590 0.3769 0.0043 12 0.79 Maryland 0.0049 0.4777 0.0067 0.8872 0.5681 0.0074 23 1.11 Massachusetts 0.0064 0.4551 0.0042 0.9891 0.5173 0.0047 110 1.11 Michigan 0.0075 0.2692 0.0052 0.9318 0.3665 0.0068 103 1.31 Minnesota 0.0092 0.3170 0.0107 0.9038 0.3984 0.0123 60 1.16 Mississippi 0.0051 0.0874 0.0024 0.6004 0.2673 0.0001 44 0.04 Missouri 0.0068 0.3750 0.0019 0.9820 0.5429 0.0038 65 2.04 Montana 0.0045 0.1244 0.0013 0.8678 0.2576 0.0038 15 2.91 Nebraska 0.0081 0.3081 0.0006 0.9996 0.3758 0.0063 15 10.44 127 continued

OCR for page 103
TABLE 5-10 Continued 128 Model with Intercept, State Data Rate No-Intercept Model Ratio of Intercept State Adjusted State Number RMSE Coefficient Coefficient RMSE Correlation Coefficient RMSE of LEAs Estimates New Hampshire 0.0060 0.3166 0.0004 0.9976 0.4780 0.0003 17 0.80 New Mexico 0.0118 0.1972 0.0271 0.7143 0.2418 0.0284 22 1.05 New York 0.0086 0.3293 0.0058 0.9533 0.4390 0.0078 175 1.33 North Carolina 0.0002 0.3369 0.0056 0.9625 0.3393 0.0056 89 1.00 Ohio 0.0057 0.2022 0.0028 0.9620 0.3289 0.0039 151 1.42 Oklahoma –0.0014 0.2894 0.0045 0.9483 0.2735 0.0048 30 1.05 Oregon 0.0026 0.4207 0.0040 0.9963 0.4425 0.0021 45 0.52 Pennsylvania 0.0021 0.5318 0.0058 0.9485 0.5606 0.0058 18 1.00 Rhode Island 0.0019 0.7080 0.0022 0.9960 0.7493 0.0030 19 1.36 South Carolina 0.0047 0.2788 0.0035 0.9254 0.3800 0.0035 53 0.99 Tennessee 0.0030 0.3389 0.0037 0.8840 0.4274 0.0042 70 1.14 Texas 0.0018 0.6058 0.0197 0.9603 0.6152 0.0197 207 1.00 Utah 0.0020 0.3175 0.0077 0.9591 0.3352 0.0079 19 1.02 Virginia 0.0082 0.2299 0.0059 0.9420 0.2931 0.0084 77 1.42 Washington 0.0024 0.5805 0.0097 0.9715 0.6075 0.0095 78 0.98 West Virginia 0.0069 –0.0525 0.0020 0.2609 0.2488 0.0036 32 1.79 Wisconsin 0.0100 0.2911 0.0061 0.9139 0.4134 0.0070 66 1.15 NOTE: Data include only eligible LEAs, as described in the text.

OCR for page 103
129 COMPARISON OF COMMUNITY SURVEY ESTIMATES AND STATE COUNTS state comparison: (1) the use of consistent procedures and criteria within most states but different ones in different states, and (2) the possibly greater similarity among immigrant populations within the same state than those in different states. The first of these reasons points to the difficulties in making present state-provided data comparable across states, while the second indicates possible difficulties in interstate comparability for ACS data. Nonetheless, the high degree of within-state consistency does give some reason for optimism that better consistency is achievable. CONCLUSION 5-3 In the absence of other factors, such as the legislated minimum allocation, the American Community Survey and state-provided data would yield broadly similar allocations to most states. However, the differences in allocations to a few states are substantial and not readily explainable by such factors as region of the country, demographic charac- teristics of the English language learner population, or the proficiency test used by the state. Temporal Variation Another criterion for comparison of the ACS estimates and state counts is the degree of variation over time of the estimates for each state. There are conflicting values in consideration of such variation. Responsiveness refers to the tendency of a set of estimates to respond quickly to changes in conditions, such as rapid growth of the population of immigrant children in a state from one year to the next. This term suggests a positive value in that resources will be more rapidly directed to states with growing needs if a more responsive measure is used. Volatility refers to the tendency of estimates to vary or fluctuate from year to year. It suggests a negative value since such funding fluctuations make it more difficult to plan and maintain program continuity. Responsiveness contributes directly to volatility when popula - tions are changing, but there are additional sources of volatility particular to each data source. Sampling variation contributes to purely random volatility in the ACS estimates. State data could become volatile when a state changes its tests, standards, or procedures from one year to the next or when there is an error or change in the mechanisms for reporting ELL counts from school districts to states to the DoEd. Table 5-11 summarizes the volatility of ACS and state-provided estimates of ELL counts in two ways (parallel to those used in the sensitivity analyses in Chapter 2). The first is the sum of absolute changes in state shares, equivalent to twice the portion of the total allocation that would be moved from one state to another in consecutive years. The second is the mean absolute value of relative changes in shares, which summarizes the amount by which allocations in each state change relative to the size of its allocation. As expected, the single-year ACS changes are about equal in the two pairs of years (2006 to 2007 and 2007 to 2008). As explained in Chapter 2, the 3-year ACS estimates are much more stable, both because of the greater reli - ability of 3 years of data and because only one out of the years changes in overlapping

OCR for page 103
130 ALLOCATING FEDERAL FUNDS TABLE 5-11 Comparison of Volatility in ACS Estimates and State-Provided Counts (in percentage) Sum of Absolute Difference Mean Absolute Relative in State Shares Difference in State Shares ACS 2006 to ACS 2007 6.07 14.84 All Large 3.02 5.77 Medium 2.27 10.99 Small 0.71 21.16 Minimum 0.07 26.11 ACS 2007 to ACS 2008 6.47 12.84 All Large 3.38 5.64 Medium 2.46 11.02 Small 0.54 14.17 Minimum 0.09 26.72 ACS 2005-2007 to ACS 2006-2008 3.64 4.83 All Large 2.36 2.46 Medium 1.04 4.34 Small 0.21 5.99 Minimum 0.02 7.18 State 2006-2007 to State 2007-2008 10.55 11.90 All Large 6.81 7.79 Medium 3.21 11.80 Small 0.42 9.83 Minimum 0.11 23.57 State 2007-2008 to State 2008-2009 5.53 9.98 All Large 2.91 5.61 Medium 2.11 8.38 Small 0.42 13.01 Minimum 0.09 14.58 NOTES: Large States: Arizona, California, Florida, Georgia, Illinois, New Jersey, New York, Texas, Washington. Medium States: Colorado, Connecticut, Indiana, Kansas, Maryland, Massachusetts, Michigan, Minne - sota, Missouri, North Carolina, New Mexico, Nevada, Pennsylvania, Oregon, Ohio, Oklahoma, South Carolina, Tennessee, Utah, Virginia, Wisconsin. Small States: Alabama, Alaska, Arkansas, Delaware, Hawaii, Idaho, Iowa, Kentucky, Louisiana, Maine, Mississippi, Nebraska, New Hampshire, Rhode Island, West Virginia. Minimum States: District of Columbia, Montana, North Dakota, South Dakota, Vermont, Wyoming.

OCR for page 103
131 COMPARISON OF COMMUNITY SURVEY ESTIMATES AND STATE COUNTS 3-year periods. (For the same reason, these estimates are also the least responsive.) Interestingly, the between-year changes in state-provided shares are much larger in 2006-2007 than in 2007-2008. We do not have enough detailed information about changes in state practices to identify specific reasons for the changes that might cause this variation and predict whether results would be similar in future years. The more detailed information by state share grouping sheds more light on patterns of volatility. In absolute terms, the largest part of annual changes in share occurs in the states with relatively large shares; as noted above, these encompass about 74 percent of allocations. However in relative terms, these states show the least volatility by any measure. Since volatility in ACS estimates is largely driven by sampling variation, it is consistently larger in relative terms for each group of suc - cessively smaller states. The pattern is less consistent in the state-provided estimates, although generally the larger states tend to have more stable numbers. This stability may reflect the greater effects on smaller states of rapid changes in ELL population in a few local areas, or it may reflect changes in reporting. Overall, the 3-year ACS estimates appear to be the most stable, at the cost of some loss of responsiveness. And as discussed in Chapter 2, 1-year ACS estimates do not capture year-to-year changes with acceptable precision. The evidence is ambiguous on comparative stability of single-year ACS estimates and state-based estimates. CONCLUSION 5-4 The superior precision and stability of the 3-year American Community Survey (ACS) estimates outweigh their slower re- sponsiveness to changes and make them superior to the ACS 1-year esti- mates as a basis for allocations.

OCR for page 103