shadow

By John Thompson

Did you hear the one about “Voo Doo Economics?” President Ronald Reagan said that his “Supply Side Economics” would cut taxes, increase spending, and reduce the deficit!?!?

If a 22nd century historian were to uncover Reagan’s claim, and yet discover that all of the physical and digital records of the 1980s and several subsequent decades had been lost, it might be hard to prove that Reaganism didn’t raise the deficit.

Perhaps the same agnosticism could apply to claims that No Child Left Behind boosted student performance as measured by the reliable NAEP tests – except for one reason. NAEP records are readily obtainable by a quick Google search.

NAEP data may not prove what I believe to be the best summary of the evidence – that NCLB and subsequent NCLB-type testing caused more harm than good for students. But, NAEP metrics do prove the intellectual dishonesty of the true believers who claim that high stakes testing has improved so-called “student performance.”

In fact, NAEP scores were increasing before NCLB and their growth slowed after NCLB testing took effect.  The American Institutes of Research’s Mark Schneider, known by the conservative Fordham Institute as the “Statstud,” is just one scholar who documented this pattern, concluding “pre-NCLB gains were greater than the post-NCLB gains.”

Curiously, Schneider was also one of the true believers who first pushed the silly claim that NCLB deserves credit for test score gains that occurred before the law was enacted. Illogically, reformers claim test score increases from 1999 to the winter of 2002 were the result of a law that was enacted in the winter of 2002. The actual passage of NCLB high stakes testing was the tail of a “meteor” that was dubbed “consequential accountability.” And that brings us to the latest convoluted spin trying to deny that test-driven reform has failed. The most recent example is Tom Loveless’s “Measuring Effects of the Common Core.”

At least Loveless’s  approach to the pre-NCLB effects of NCLB is much more modest. He claims that it is “unlikely” that accountability efforts and increased reform-related spending did not “influence” pre-NCLB NAEP scores. Even so, Loveless offers no credible reason to believe that increases in 1999 test results should be attributed to stakes attached to tests that were imposed three years later.

So, what evidence does Loveless offer for his conclusion that NCLB might deserve credit for the test scores that preceded it? He cites Derek Neal and Diane Whitmore Schanzenbach who reported that “’with the passage of NCLB lurking on the horizon,’ Illinois placed hundreds of schools on a watch list and declared that future state testing would be high stakes.”

Neal and Schanzenbach were studying Chicago schools, however, and they concluded the opposite.  They report that “ISAT performance played a small role in the CPS rules for school accountability over this time (1999 to 2001).”  Neal and Schanzenbach explain that “in one year, the ISAT went from a relatively low-stakes state assessment to a decidedly high stakes exam.” But, “in the springs of 1999, 2000, and 2001, CPS took the ISAT with the expectation that the results would not have significant direct consequences in terms of the state accountability system.”

By the way, Neal’s and Schanzenbach’s title of “Left Behind by Design,” is not exactly a ringing endorsement of Schneider’s and Loveless’ spin in favor of No Child Left Behind.

Loveless cites two other papers in his footnotes, but a careful reading of provides evidence for both sides of the issue as to whether testing that preceded NCLB was actually high stakes, whether it changed behavior in positive or negative ways, or whether it contributed to improved test scores.

Rather than get bogged down in arcane details, policy-makers should remember that true believers in consequential accountability can’t even agree on which states had such systems. They acknowledge that the economy and demographic changes could explain changes in NAEP results, but they do not try to control for those factors. Moreover, test score growth of early adopters of accountability did not necessarily increase in comparison to their previous scores. And, the fact that early adaptors had the money to increase education spending may explain subsequent (though often short-lived) gains in student performance. Above all, even the papers cited by Loveless help explain why NCLB-type testing, whether it helped some students or not, probably damaged other students.

While Loveless and others debate whether the effects of the No Child Left Behind Law should be seen as beginning in 1999 or 2003, they don’t seem aware of a more likely conclusion. When debating the effects of stakes attached to NCLB, shouldn’t we ask when stakes started to be attached to its tests?  Schools started to receive additional NCLB funding in 2002, but for many or most of them, the law’s punitive measures did not take effect for another three years.  There is no definitive answer, but I would argue that the 2005 NAEP is likely to be the single best indicator of the first effects of NCLB accountability provisions.

I would also argue that 8th grade NAEP reading scores should be seen as the most valuable metric when evaluating an education policy. A variety of reforms have been shown to improve math scores, without improving reading but, I would argue, being able to read for comprehension is the much more important skill required for schooling and life. Also, boosts in 4th grade test scores that wash out by 8th grade are not as likely to change the educational and life trajectories of students.

But, here I go again. I’m once again trying to parse the details of quantitative research and why some academics seem allergic to asking how their statistical models relate to these pesky real world issues. Instead, I should ask why non-educators who study education policy are so willing to cherry-pick evidence in support of the bizarre spin of reformers who seem to believe in Teleology.

What do you think? Why are economists and think tanks so insensitive to the realities in school and yet so sensitive to the spin of corporate reformers?

Author

Anthony Cody

Anthony Cody worked in the high poverty schools of Oakland, California, for 24 years, 18 of them as a middle school science teacher. He was one of the organizers of the Save Our Schools March in Washington, DC in 2011 and he is a founding member of The Network for Public Education. A graduate of UC Berkeley and San Jose State University, he now lives in Mendocino County, California.

Comments

  1. Elizabeth Hanson    

    I would say that it has a lot to do with money. They ed-deformers see the public schools as a way to make a boatload of money. No matter that the economy is in the toilet for most Americans and that the kids have less chance than just about ever to get a living wage job. We are caught up in a web of marketing slogan whilst too few wish to talk about the real problem- not enough living wage jobs to go around. In fact, 66% of job openings are going to be in areas that don’t require much or any college… like nursing assistant, retail clerk… food server. How about we deal with getting things for our kids to get “college and career ready” for… (I can barely utter that phrase without making a “bleh” face.

  2. howardat58    

    Not for nothing was it once said (by whom I cannot remember) – There are lies, damned lies, and statistics.

  3. Kimberly Reeves    

    I don’t agree with any of this. Having seeing it first hand, accountability can be the only line of defense to force a school district to address a failing school. Unfortunately, when it began in Texas, we set the bar as low as the superintendents wanted it. Which was very very low. So we’ve had TABS, TEAMS, TAAS, TAKS and STAAR. We’ve only come to useful data with STAAR, which goes beyond a minimum skills test. That’s been my general idea for a number of years… we needed to capture achievement instead of minimum skills. You want to compare tests to NAEP? They aren’t even on the same planet. Now, as we slow on NAEP scores, it seems clear to me that outcomes based education cannot work alone. You can’t demand teachers and districts go DO IT, without arming them with the best chances for success. The inputs we need to fund — teacher training, teacher support, community buy in for standards — are getting no traction. And Elizabeth Hanson’s assumptions of employment are incorrect. It’s just the opposite. A total of 66 percent of jobs in 2020 will require some form of post-secondary education, be it a 2-year degree, a certificate or a 4-year degree. Beyond that, if we’re going to foolish enough to stop kids at high school graduation — rather than focus on achievement — then it might be nice if they understand statistics, mortgages, government functions and broad literature references that come in advanced courses. What we find here in Texas is that those superintendents who THINK kids can achieve advanced courses will attempt it. And those superintendents who DON’T, won’t. That’s where we’re going… back to the time where we determine kids are stupid, we blame the parents and people carry the kind of closed-minded opinions expressed in some of the comments here. On behalf of the kids who grew up in my neighborhood, I would ask you to stop it. You think you aren’t racists, but your outcomes will be absolutely divide our schools into haves and have nots. I doubt Elizabeth Hanson would be happy with her own child failing to attempt any type of post-secondary education. But she’s happy to doom others to failure. Thank God for career and college readiness.

  4. rbeckley58    

    Obviously corporations just want to make a buck and convince the public it’s good for them. Pearson uses smoke and mirrors to sell to sell overly-long, imperfectly designed, poorly phrased, glorified bubble tests that they mislabel as rigorous. Take the sample PARCC and Smarter Balanced tests for yourself and then decide if we now expect more from high school students than we do from college graduates.

    On the other hand, the stakeholders, students, parents, teachers and administrators, want nothing but the best for children. But politicians and philanthropists have the gall to make decisions ignoring these stakeholders capable of teasing out the real issues – poverty, lack of public investment and widening gaps between socio-economic groups. Instead, the overlords turn schools over to monied interests with poor track records or who focus solely on testing the top third of students for college readiness, ignoring the needs of the other 2/3.

    Realities facing high school grads are quite different from propaganda. “There are far more STEM-degreed applicants than there are jobs available for them, especially in information technology. Google gets over a million applications each year. Microsoft and Verizon are in the process of sweeping layoffs. At the end of the day, (only) 10 percent to 20 percent of us are employed in professions requiring advanced STEM degrees.” (Register Guard, April 5, 2015)

    The contractor who fixed my plumbing isn’t using his A’s and university degree to make a generous $40/hour. Several families I know who run successful bed-and-breakfast inns aren’t using their university degrees. A PhD holder couldn’t find a job in his field, so he spends summers picking cherries. A colleague left teaching to make far more money that’s at a job requiring neither STEM nor her degree. High test scores and degrees don’t guarantee a job awaits; our economy is far more complex than that.

    Besides, research shows that the greatest predictors of high school students’ future success are social skills, diversity of experiences both in and out of school, passing (!) grades, and a track record of sticking with a single subject over several years.

    Meanwhile, under the current business model for education, gaps between the scores of socio-economic groups are widening, not narrowing. No surprise, disabled and ELL students, who take the same tests, experience plummeting scores and demoralization. If we want to compare our country’s scores with others’, we ought to use the same parameters; other countries only test students functioning at grade level. We’re comparing apples with oranges.

    Finally, pre-NCLB standardized tests cost $335 million, yet in 2012 alone they surpassed 8 billion! That was when PARCC and Smarter Balanced were used by small pilot groups. Imagine how high the costs are now that everyone is expected to comply. These upward spiraling costs have severe consequences: disappearing electives, extra-curricular activities and non-core subjects like Social Studies and Health. Even the US Department of Education’s Civil Rights Office admits this trend is most detrimental for minorities and the poor. The current system is widening the gaps between the have and have-nots.

    I appreciate this chance to ignore the opinions of profiteers and those light-years away from the classroom. Let’s keep teasing out the real issues.

Leave a Reply to Kimberly Reeves Cancel reply