By John Thompson.

The latest report by the Education Research Association on New Orleans school reform, “Extreme Measures: When & How School Closures & Charter Takeovers Benefit Students,“ is consistent with the solid scholarship that informs its previous research. And it includes the important qualifying statement that, “prior evidence suggests that schools that are effective in generating high test scores are not consistently more effective with other student outcomes.” When the ERA released its landmark “What Effect Did the Post-Katrina Reforms Have on Student Outcomes?,” a diverse variety of scholars participated in the discussion of its findings. Moreover, previous research included qualitative research such as Huriya Jabbar’s “How Do School Leader’s Respond to Competition?” Fewer dissenting opinions were expressed when this new paper was released.

Fortunately, panelist Andre Perry articulated a broader view of the issues involved in evaluating school closures and takeovers in New Orleans and elsewhere. Perry explained that black people have already faced more than their share of punitive policies in terms of education, criminal justice, housing, and social welfare. He acknowledged that school closures and other state interventions are sometimes necessary. But, trust is required for school improvement and people of color can no longer trust the accountability-driven, technocratic experiments that are being imposed. Rather than tinker with the reward and punish approach of the last generation, we should focus on capacity-building.

Douglas Harris began his presentation with a mention of President Obama’s advocacy of school closures and takeovers. However, the President was not solely responsible for the misguided mass closure, turnaround, and transformation approach. It was the larger corporate reform community which pushed for the administration’s dubious Race to the Top (RttT), School Improvement Grant (SIG)s, and his other, risky, competition-driven policies. It was true-believers in charter schools and “disruptive innovation” that made the case to Obama that these experiments must be rapidly scaled up. Many of them will continue to use and misuse the ERA’s research to advance their agenda.

Harris’s characterization of closures and takeovers as “rare” is accurate, but they along with school turnarounds and transformations were integral parts of billions of dollars of RttT and SIG initiatives that produced very little positive change and significant amounts of harm, and drained urban educators of their energy and morale. Teachers have to worry that takeovers will become a growing part of the corporate school reform arsenal.

As explained here, I worry that the ERA is interpreting its data in ways that are too quick to equate test score increases with increases in learning. It is always annoying when researchers write of increased test scores as being increased “student performance.” It is bad enough when scholars get close to equating test outcomes with more “learning.” But, it would take a lot more documentation than can be offered by mostly quantitative methods before it could be concluded that increased bubble-in “outputs” mean that NOLA reforms “benefit students.” I don’t believe the new paper’s methodology can document the finding, “All future cohorts (of students) benefit, which means the net effect is clearly positive.”

The ERA’s single most important finding was illustrated by a graph showing elementary schools’ test score gains before, during, and after closures. It charted the gains of schools that were closed, as well gains by a matched group of students. The key conclusion was that two years after the interventions, tests scores in the schools that were taken over caught up and surpassed the matched group of students. So, if those gains reflect real learning, the ERA makes a strong case for taking over elementary schools.

But left unexplained is the time between just before the closure or takeover was announced and just after it was announced. The ERA chart shows that test score gains were just as dramatic during that period! And, if those transition gains were not real, then it’s not possible to conclude that future gains that grow from that policy are meaningful. The gains during that year were comparable to the difference between the matched group and the intervention group. No plausible explanation was given as to how the time between just before and just after closures were announced could have conceivably produced dramatic test score gains that reflect actual learning increases, as opposed to what targeted schools are likely to do – juke the stats.

To their credit, the ERA also reminds us of the oft-ignored principle which is especially important when experimenting with children’s lives, “the principle of ‘do no harm.'” If the ERA is serious about minimizing harm, however, it must tackle the following issue. During the ERA discussion, charter supporters insisted that they must have the right, preferably in conjunction with systems and patrons, to devise the school model that they will adopt. The ERA should accept the challenge of studying what that means in the real world. It’s likely that charters serving more affluent, whiter student bodies will encourage innovation and engaging instruction. It is doubtful that those schools will embrace the “No Excuses” behaviorist approach.

Conversely, competition-driven school policies are likely to result in much more teach-to-the-test in high-challenge schools. There is no secret about these dynamics. As was true when this mindset was known by the disgusting name of “earned autonomy,” education systems are pressured to offer respectful, holistic instruction in more privileged schools while reward and punish behaviorism is imposed on the poorest children of color. I hope the ERA will incorporate qualitative research to complement its quantitative studies in order to document how those dynamics are playing out in New Orleans.

Some new NOLA policies, such as placing students in closed schools at the top of the waiting list for the schools of their choice, show promise. But, as was asked in the discussion, how could their endless renewal process, even requiring schools to earn a grade of a C before a second renewal, not be a recipe for endless, fear-driven, teach-to-the-test, gimmick-driven schooling?

Even Neerav Kingsland, the former CEO of New Schools for New Orleans, now says, “I think much of New Orleans’ gains were driven by the phasing out of failing schools.” But, “It is much less clear to me that schools in New Orleans, to date, have figured out to crack the code of creating schools that are radically superior to your average functioning traditional school.” So, he has acted on the faith that “No Excuses Charter Schools May Allow Us to Eliminate Failing Schools and Raise the Aggregate Earnings of Low-Income Students in the United States.” Although it seems unlikely that Kingsland will walk away from his faith that No Excuses schools will bring future benefits to New Orleans students, at least he acknowledges, “A bunch of teaching to the test just jacks up crystallized knowledge but doesn’t really give kids the human capital qualities they need to succeed in the workforce.”

And that brings us back to the words of Andre Perry. A decade after Katrina, and after more than a generation of reward and punish programs in education and other sectors, where is the evidence that the lives of poor people of color have benefitted from technocratic policies that have such a punitive dynamic?

What do you think? If the ERA really believes in the rule “do know harm,” won’t it seek direct evidence about what is actually happening in New Orleans schools? Do the NOLA reforms guarantee a permanent second class status for the poorest children of color?