Why Did Test Scores Go up in California? 
A Response to Unz/Reinhard*

by Stephen Krashen
University of Southern California 

David Reinhard of the Portland Oregonian recently published a column in which he attempted to rebut what he considers to be arguments of bilingual education defenders. It is, I think, a useful exercise to review the defenders' arguments, and especially Mr. Reinhard's responses: Mr. Reinhard makes it clear that he has accepted Ron Unz's point of view, and we can expect these arguments to resurface in many different parts of the country. 

The central issue in Reinhard's column is the so-called "skyrocketing" increase in test scores in California since the passage of Proposition 227, the initiative that dismantled California's bilingual education programs. According to Mr. Reinhard, supporters of bilingual education attribute this gain to a decline in class size and a new approach to reading. I don't. As Mr. Reinhard points out, studies have shown that the contribution of lowering class size was modest. There is, in addition, no evidence relating test score improvements to changes in reading instruction. The Reinhard/Unz view is, of course, that Proposition 227 deserves the credit for the increase. But there is also no evidence that dropping bilingual education increased test scores.

Why did test scores go up in California? Improvements in teaching can raise test scores, but one can also raise test scores by teaching to the test, special "test preparation" and by not allowing low scorers to take the test (for evidence suggesting that selective testing took place in California, see Asimov, 2000). This is like claiming to raise the temperature of the room by lighting a match under the thermometer. This use of such tactics is highly likely when states impose harsh penalties for lower scores and rich rewards for higher scores, as California has done.

Mr. Reinhard acknowledges that everyone gained in California over the last two years, but accepts Ron Unz's claim that limited-English-proficient (LEP) children improved far more than native speakers of English. Reinhard, quoting Unz, claims that for tests of English reading in grades 2 through 6, the overall gain for native speakers was 21 percent, but for LEP students it was 39 percent. A look at the actual scores reveals the flaws with this argument. 

California SAT9 Results: English reading percentile scores, grades 2-6
 

English language learners English-proficient students
grade 1998 2000 change 1998 2000 change
2 19 28 +9 39 48 +9
3 14 21 +7 36 44 +8
4 15 20 +4 40 45 +5
5 14 17 +5 40 44 +4
6 16 19 +6 43 47 +4

Source: California Department of Education
 
LEP children in California in grades 2-6 in 1998 averaged 15.6 on English reading. In 2000, they averaged 21, a difference of 5.4 points. Native speakers of English in grades 2-6 in 1998 averaged 39.6 points. In 2000, they averaged 45.6 points, a difference of 6.0 points. The gains were nearly identical. Mr. Unz makes the LEP students' gain look larger by misusing percentages: 5 or 6 points is a larger percentage of a small number than a larger number. Note also that the only clear difference between English language learners and English-proficient students is in grade 4, where English learners in 2000 scored 15 points higher than English learners in 1998.

Proposition 227 took force at the same time the new SAT9 test was introduced. Research
(Linn, Graue, and Sanders, 1990) has shown that when new tests are introduced, test scores go up, which is why commercial tests need to be recalibrated every few years. Typical test score inflation is about 1.5 to 2 points per year, which accounts for a great deal of the gains seen in grades 2-6 in California for both groups.

Mr. Reinhard also took Ron Unz's word that that "immersion's advantage over bilingual instruction are [sic] undeniable in district-to-district comparisons."

Not so. Mr. Unz has succeeded in directing nearly all attention on Oceanside, a district that dropped bilingual education and embraced English immersion. We have several reasons to suspect that Oceanside's previous bilingual program – the one that was dropped – was poorly conceived. In fact, it was not a bilingual education program: It was a Spanish-only program. In an article in the Washington Post (3 September 2000), Oceanside Superintendent Ken Noonan stated that, before Proposition 227, Oceanside's bilingual program was all-Spanish, lasting "for up to four years, even longer for some. Only after being designated fluent in English would a child's learning in English begin in earnest." Properly organized bilingual programs, by contrast, introduce children to English from day one, and academic subjects are taught in English as soon as they can be made comprehensible. Failing to provide any English instruction will naturally lead to miserable results on English-language achievement tests. This explains why Oceanside's test scores showed substantial improvement, especially for the youngest children, when English was introduced. 

In addition, in an article in the San Diego Union-Tribune (6 October 2000), it was revealed that, before Proposition 227, books were in very short supply in at least one Oceanside school with a significant number of LEP students: Before 227, "a lot of students (at Laurel Elementary School) didn't even have books." The Union Tribune article also gives the reader the impression that virtually any activity unrelated to test preparation was dropped from the school day. In addition, strong carrots (financial rewards) and sticks (threats of school closure) were instituted.

It thus appears that Oceanside dropped an inadequate bilingual program, and at the same time focused nearly all its energy on test preparation. In addition, according to research done by Prof. Kenji Hakuta of Stanford University, gains for Oceanside's English learners were similar to gains made in many California districts that retained bilingual education (Hakuta, 2000; Orr, Butler, Bousquet, & Hakuta, 2000).

Mr. Reinhard also accepted Unz's analysis of the situation in the Vista district in Southern California. Unz claimed that Vista, "a bilingual stronghold," improved by "only" 24 percent. According to a report in Education Week (6 September 2000), Vista erroneously lumped a large number of high-scoring LEP students with its fluent English speaking population. Removing the high-scoring children had the effect of artificially lowering the average for LEP children.

The only valid way to determine the effect of bilingual education is to perform controlled studies. In these studies, programs are compared in which the only difference is the use of the first language. Stanford 9 test score comparisons are not controlled studies. They often include English learners who are not in bilingual programs, and such comparisons do not consider a host of other factors that impact performance, such as poverty.

Scientifically valid controlled studies have been done, and they consistently show that students in properly organized bilingual programs acquire at least as much English as comparison students in all-English programs, and usually acquire more. The most recent review of this research is Greene (1997), which used statistical tools far more precise than those used in previous reviews. Greene concluded that the use of the native language in instructing LEP children has "moderate beneficial effects" and that "efforts to eliminate the use of the native language in instruction ... harm children by denying them access to beneficial approaches."

To conclude his column, Mr. Reinhard invoked the familiar argument of the immigrant who succeeded in school without bilingual education. He quotes José Campos, who immigrated to the United States at age 19 and started college six months after he arrived. We have studied many cases of immigrants who acquired English quickly and did well in school (Krashen, 1996, 1999; Ramos & Krashen, 1997; Ramos, 2001). In every case, they had had "bilingual education": quality education in their first language before arriving in the this country. Their solid foundation in subject-matter knowledge helped make the English they heard and read much more comprehensible, and it accelerated their English language development enormously. Also, these immigrants were already literate in their first language. It is well established that those who have previously developed literacy in the first language have a far easier time developing literacy in a second language (Krashen, 1996). Cases like these actually provide strong evidence in favor of bilingual education, because bilingual programs provide both subject matter knowledge and literacy in the first language.

The Reinhard/Unz claim has a great deal of popular appeal: Bilingual education was dropped in California and test scores went up. But dropping bilingual education had nothing to do with the increase. Test score increases in California appear to be a rest of normal "test score inflation" that occurs when new tests are introduced. Analysis of gains in individual districts shows that those that kept bilingual education improved and those that never did bilingual education improved; everybody improved in California (Hakuta, 2000) and there were no obvious differences between gain made by English learners and English proficient children. Gains in Unz's poster district, Oceanside, were, according to Hakuta (2000), not remarkable, and can be attributed to the poor bilingual program that was in place before 227, as well as intensive test preparation. 

Missing from nearly all discussions of the effectiveness of bilingual education is the consistent finding of controlled scientific studies: Bilingual education works.

References
Asimov, N. 2000. Test scores up, test-takers down: Link between participation, improvement on school exam prompts concern. San Francisco Chronicle, 22 July.
Greene, J. 1997. A Meta-Analysis of the Rossell and Baker review of bilingual education research. Bilingual Research Journal 21(2, 3): 103-122.
Hakuta, K. 2000. Points on SAT-9 performance and Proposition 227. 
Krashen, S. 1996. Under Attack: The Case Against Bilingual Education. Culver City, CA: Language Education Associates.
Krashen, S. 1999. Condemned without a Trial: Bogus Arguments Against Bilingual Education. Portsmouth: Heinemann.
Linn, R., Graue, E., & Sanders, N. 1990. Comparing state and district test results to national norms: The validity of claims that "everyone is above average." Educational Measurement: Issues and Practice 10: 5-14.
Orr, J., Butler, Y., Bousquet, M., & Hakuta, K. 2000. What can we learn about the impact of Proposition 227 from SAT9 scores? 
Ramos, F. 2001. The not-always considered importance of "de facto" bilingual education. Bilingual Basics 4(1): 9-11.
Ramos, F., & Krashen, S. 1997. Success without bilingual education? Some European cases of de facto bilingual education. CABE Newsletter 20(6): 7, 19.
Reinhard, D. 2001. End bilingual ed and save children. Portland Oregonian, 8 April.



*Forthcoming in the New York Association for Bilingual Education (NYSABE) Newsletter. Reprinted by permission.