Cheung, Alan C. K., & Slavin, Robert E. (2012). Effective reading programs for Spanish-dominant English Language Learners (ELLs) in the elementary grades: A synthesis of research. Review of Educational Research, 82(4), 351-395.
When a well-known researcher like Robert Slavin is listed as an author on a review of research in a prestigious journal, people are going to read that review. Findings of that review have weight when educators make decisions about literacy programs that they believe might be effective with learners from populations with a history of low reading achievement. The numbers presented in such a review may provide evidence for the implementation of large-scale programs in school districts where there are large low-achieving populations. Implementing these programs costs money, and some of that money will find its way to the programs’ developers, as well as provide employment for individuals who help facilitate program implementation. Financially, the stakes of having a program declared especially effective for low-achieving populations are high. If the program you have made a career of is seen as having the data to back up its effectiveness, you stand to gain a lot. If it is seen as less effective than others, you will not reap the rewards in terms of prestige and financial gain.
That’s why I am uncomfortable with a review of research like the one Cheung and Slavin present in this article. Whether the literacy program that is indelibly linked with Robert Slavin and the others who have joined his work is the most effective way of teaching English Learners is not what causes my discomfort. I know that program, Success for All, fairly well, because I have worked with teachers and teacher candidates at school sites in a large urban district that implemented Success for All for a few years. It did not prove to be the solution to that district’s underachievement woes, nor did it keep that district from losing its state accreditation. The district has since adopted yet another program, and to be fair, though some believe there are some glimmers of hope, and I want that to be true, that new reading program hasn’t solved the problems either. In any case, my own experience with Success for All has not given me any strong convictions that it is all that effective a program, and I need to be transparent about that. The reader of this annotation needs to know that background; in short, I am no great supporter of Success for All. When Jonathan Kozol’s scathing critique of Success for All, The Shame of the Nation, came out a few years ago, I read that critique and nodded knowingly, though Kozol’s book only confirmed what I had long before then independently concluded about Slavin’s program. Even though I myself am not a supporter of Success for All, I don’t think that is the principle reason for my discomfort.
The big question for me is whether it is a good thing for a reviewer of research to have “a horse in the race.” I wonder if I would be more comfortable with findings that support Success for All if they were produced by reviewers who do not have anything to gain from those supportive findings. I know that reviews by program developers that include those reviewers’ own programs and program effectiveness studies are not uncommon in today’s educational journals. I know that many reviews of research literature are not independent reviews, and a truly “disinterested” reviewer of a large-scale literacy program is probably a rarity. I will not go so far as to say there is anything unethical about that. I can find no reason to question the intentions of the reviewers here, and the review itself appears to be carefully and ethically constructed and reported. All that being said, when the findings of a research review clearly include support for a program associated with that review’s authors, that raises questions in the reader’s mind, and those questions are uncomfortable.
Perhaps it’s the way of the world to want to promote a program you have made a career of, especially if you truly believe it can help solve problems. Furthermore, what motivation could a disinterested reviewer possibly have, and what incentive, to undertake a purely disinterested review? Someone who wants to discredit a program like Success for All could do it, but that wouldn’t be any more disinterested. Am I being hopelessly naïve to think a disinterested review is ever possible? Probably so.
The journal in which the review appears is one of the most prestigious in the U.S., if not the world. It is sponsored by a large, respected educational research association. If such a journal would publish a review that clearly supports a program, and in fact would give it top billing as the lead article in that issue (which is where Cheung and Slavin’s article is placed), then those who make decisions about that journal must not have the same level of discomfort that I do about whether or not to trust the findings of a review like this one. As a consumer (and admittedly not a producer) of educational research findings, I believe the best way to manage my discomfort is to acknowledge it, and to keep on reading and questioning. As with any reading task, reading educational research is a process of meaning construction. Ultimately, the meaning I construct from what I read, and then what I do with that meaning, is up to me.
No comments:
Post a Comment