shadow

By Anthony Cody.

This week leaders in the California Department of Education released a new system for “grading schools” that proposes to assign schools a set of colored indicators based on a variety of measurements. This system is intended to replace the old system of performance indicators which gave schools a single numeric score, (the Academic Performance Index, or API) based only on test scores. While this new system brings us elements of that long sought goal of “multiple measures,” I fear that we have yet to escape the box of the “measure to manage” paradigm.

Over the past eight years the state of California has resisted some of the worst aspects of Federal reform. The state resisted federal pressure to tie teacher evaluations to student test scores, and Jerry Brown was a vocal critic of Arne Duncan’s Race to the Top, as indicated by his statement in 2009. In more recent years, Brown has restructured school funding, to direct resources to school districts based on student needs, as influenced by factors such as poverty and the proportion of language learners enrolled.

But the state – and our state teacher union, the California Teachers Association, have also embraced the Common Core (renamed “California Standards” last year.) With cooperation from the state university system, the California Teachers Association and funding from the Gates Foundation, teachers have been treated to days devoted to celebrating the Common Core. This year teachers will get yet another day-long Gates-funded Common Core festival, though once again they are the national standards that dare not speak their name.

California teachers and students have been told that a new and much improved accountability system is in the works, and now the draft has finally arrived. The mandate for this system is in regulations created by the federal Department of Education, which asks states to create a system to show “Measures of School Quality,” described this way:

Increased state flexibility to take a more holistic view of school performance based on multiple measures including achievement in reading and math; academic progress in elementary and middle schools; graduation rates in high schools; rates of progress for English learners achieving language proficiency; and a state-determined indicator of school quality or student success.

The regulations further ask that:

States create a multi-level rating system that clearly communicates to parents and communicates how their schools are doing, taking into account all of the measures of school performance.

Information displayed in a timely manner on annual report cards, designed with input from parents.

The new system being proposed for California schools was designed by technicians at West Ed, and it creates a matrix of color-coded squares that indicate both the absolute status and the direction of change for ten different categories of data. Thus we get a system with ten categories of information, and seventeen color coded boxes. The categories are:

  • ELA assessment (K-8) (scores on Common Core aligned SBAC tests)
  • Math assessment (K-8) (scores on Common Core aligned SBAC tests)
  • English learner proficiency (scores on CELDT tests)
  • Graduation rate (9-12)
  • Chronic absenteeism (K-8)
  • Suspension Rate & Local Climate Survey
  • College & Career Readiness (scores on 11th grade Common Core aligned SBAC tests, plus other indicators)
  • Basics (Teachers, Instructional Materials, Facilities)
  • Implementation of Academic Standards
  • Parent Engagement

Here is what a sample report would look like:

CaModel5

 

 

 

 

This presentation from the July 13 State Board of Education meeting provides greater detail — you can download here:. CaliforniaModelpresentation

In thinking about this proposal, it is important to recall what it is going to replace, which was a single number that was assigned to each school, derived entirely from standardized test scores. We have long argued that education is far more complex, and here we have a system that attempts to grapple with some of that complexity. There are indicators for local climate – derived in part from surveys which measure student engagement – this should be a major focus for every school.

The category “Basics” is the one thing on the list that might be considered an input. How well resourced is the school? What is the level of education and experience of the faculty? These are critically important variables. If the new funding formula is effective at redirecting resources towards schools with the highest needs, we should see improvements in some aspects of this.

I wonder what we might want to include that is not here. What about an indicator of school stability? What is the level of staff and administrator turnover from year to year? Student success correlates positively with stability, so this would be a useful indicator.

I want to back up a bit though, and reflect about what was so problematic about the prior system we had in place. First of all it was only based on test scores, and performance on those scores was largely determined by the income and parental education level of the students that attended the school. Thus the API score was more an indicator of affluence than of school quality. In this proposed system, this will remain true for all the indicators associated with test scores.

There is a second huge problem with these sorts of indicators, and it is called Campbell’s Law. This law states:

The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.

Corruption pressure means that those under pressure to improve these outcomes will find ways to “game” the system. So let’s look at these indicators and see how they are likely to be gamed.

All of the indicators associated with test scores are already being gamed by teaching to the test. The SBAC tests have been designed so that only about a third of students will score proficient or better, so there will be intense pressure to figure out how to improve those scores. This is by design, and it is a basic corruption of our mission as educators.

When English Language Proficiency is scored, there is pressure to reclassify students up and out of their status as English learners. This may result in students being reclassified before they are ready. California has particular challenges with English learners, who make up about 30% of public school students. I asked a teacher in the Los Angeles area, Jeanne Berrong, for her thoughts, and she said:

School systems want English learners to be reclassified ASAP with little or no regard to the realities of language acquisition. This is likely to be intensified if this is used as a factor indicating school quality.

Similar issues affect the high school graduation ratings, which score schools on their ability to graduate students “on time” in four years.* I asked Carol Burris for her thoughts on this, because she recently retired as a principal in New York state. She said,

When you use graduation rates to rate schools, it is very tempting and easy to game the system. It is one of the few measures you can control. Schools create credit recovery systems, and teachers are pressured into giving course credit to students for doing a poster or worksheets.

With English language learners, some come in well prepared and have no trouble graduating in four years. However, you also get immigrants who may not have had a solid education in their native country, and with this system you have to push them out in four years even if they are not really ready to graduate.

There is no sound educational reason to insist that students graduate in four years. It is far better to have well prepared responsible graduates than to have students pushed through to raise a graduation rate so that a school does not get in trouble.

I also want to point out a basic flaw in the “multiple measures” mantra. We have heard this language before, in relation to teacher evaluations. Supposedly, if test scores are only a part of a set of “multiple measures” then this will prevent those scores from causing harm. But just as in teacher evaluations, if thirty or forty percent of your score comes from test scores, this will be a major focus for your work. And just as with teacher evaluations, if your school has large number of students living in poverty, large numbers of English learners, large numbers of special education students, all these factors will drive your scores downward.

So while it is good to see an attempt to recognize the complexity of the school environment, we remain hamstrung by the limits of measurable outcomes. We must remind ourselves that there are many dimensions of learning not measured by tests, or indicated by graduation rates, or other quantitative outcomes.

Richard Rothstein wrote a book several years ago entitled “Grading Education, Getting Accountability Right.” In this book, he pointed out that society actually wants the following things from public schools:

  • Basic academic skills and knowledge
  • Critical thinking and problem solving
  • Appreciation of the arts and literature
  • Preparation for skilled employment
  • Social skills and work ethic
  • Citizenship and community responsibility
  • Physical health
  • Emotional health

Of this list, standardized tests measure only the first. The makers of the current generation of tests also claim that they measure “college and career readiness,” but the PARCC test was recently found to fall far short, and there is no evidence that the SBAC test is any better. None of the other desired outcomes is addressed by the ten categories on the West Ed matrix.

Above all, we have to ask ourselves what purpose these school grades will serve. I asked Dr. Stephen Krashen for his thoughts, and he cut to the heart of the matter.

The WestEd “California Model” for rating public schools was created by people with too much time on their hands, and who have no idea about what is important in education.  Common sense, as well as a great deal of research, tells us that the major problem in education is poverty.

Children living in high poverty have poor diets, inadequate medical care, and little access to books; the best teaching in the world will has little impact when children are hungry, ill and have nothing to read.

If we substantially reduce poverty or at least protect children from the effects of poverty, all the boxes in the California Model table will be blue (or at least green) for all students.  Any evaluation system will give similar results.  Evidence for this includes studies showing that American middle class students attending well-funded schools do very well on international tests.  They would be colored green or blue in all the WestEd categories.

Forget fancy evaluation schemes. Focus on providing better food programs, more school nurses, and invest in school libraries and librarians. Work on the cause, not the effect; feed the animal, don’t spend time and money on more precisely measuring what we already know.

We continue to labor under the illusion that some system of accountability — some carefully designed combination of carrots, sticks and measurable outcomes — will drive our schools to constantly improve. As teachers, we can always use timely information about what and how our students are learning, but this never comes from standardized tests. In our communities, we need meaningful information about how our schools are doing on a variety of dimensions — again, these are not reflected in test scores, But more than that, we need attention to the underlying conditions described so well by Dr. Krashen. Let’s feed the children, make sure they are not afraid of violence in their neighborhoods, allow them to learn about their cultures through Ethnic Studies, stabilize and support their teachers, and watch them learn and grow. As Sir Ken Robinson pointed out, teaching is like growing a garden. You just have to create the conditions for growth. Let’s focus on those conditions and stop obsessing over test scores.

What do you think?

* Note: California hopes to use a 5 year model, but the federal regulations state that you need to include 4 year graduation rates as well:

“For all high schools in the State, section 1111(c)(4)(B)(iii) requires an indicator, based on the long-term goals established under section 1111(c)(4)(A), that measures the four-year adjusted cohort graduation rate, and, at the State’s discretion, the extended-year adjusted cohort graduation rate.”

Author

Anthony Cody

Anthony Cody worked in the high poverty schools of Oakland, California, for 24 years, 18 of them as a middle school science teacher. He was one of the organizers of the Save Our Schools March in Washington, DC in 2011 and he is a founding member of The Network for Public Education. A graduate of UC Berkeley and San Jose State University, he now lives in Mendocino County, California.

Comments

  1. Kathy Markarian    

    I agree with the skeptics — if data on graduation rates and reclassification of English learners become components of a school’s rating there WILL BE kids moved along prematurely just for the sake of that rating. BIG mistake.

  2. Leonie Haimson    

    One element called Basics- involving teachers, facilities and instructional materials- how is that measured? This seems to me to include the most important factors that parents are looking for, and yet it’s unclear what it means and why they are all conflated together. What about class size? This is a top priority for parents and teachers and a major contributor to the opportunity for student learning.

  3. Vladimir G. Ivanovic    

    As you point out, these multiple measures don’t measure what parents really want from schools for their children.

    Also, arguably, none of these multiple measures measure outcomes. In a certain sense, I don’t care how well my child does in ELA or Math; what I’m really interested in is, “Will they lead lives that they think are authentic and fulfilled, i.e. successful on their own terms?” I agree that it’s likely that being fluent in reading and writing and math are important, but I can imagine many lives where those skills are merely helpful (artists, social workers, construction, parent, politician).

    On the other hand, it may be that multiple measures are like charters school (at least in California): they are a vaccine against vouchers. Perhaps multiple measures are a vaccine against a completely test-driven educational system like that in China.

    Finally, it strikes me that we will do anything rather than confront the reality of urban schools children: “poor diets, inadequate medical care, and little access to books,” not to mention violence and trauma at home and in the streets, grinding poverty, lead in the drinking water, incarcerated parents, rampant drug (ab)use, and inequality and injustice everywhere.

  4. Kimberly Kunst Domangue    

    For ESSA, “multiple measures” can equal 40% of the total “measuring” of a school. There are research-supported resource indicators that CAN be placed in state/district accountability plans. Many education organizations anticipated the NCLB/ESEA appropriations reauthorization and began collecting research/data in 2006.

    Links for your disposal/dissemination:
    http://getessaright.org/indicators/opportunity-dashboard-indicator/

    At issue: GETTING MANDATED “MEANINGFUL EDUCATION STAKEHOLDER CONTRIBUTION”.

    THAT, dear friends and co-laborers, is where OUR work begins. Plans need to be submitted to USDOE around spring of 2017, which means community organizing needs to take place yesterday.

    Like, seriously.

    Get the ESSA federal doc, load it in iBooks, read initial table of contents for your specific area of concern, then enter it (case-sensitive) into the search engine for your digital device. Read it… print out or highlight the parts you feel/think most passionately/decidedly you could not abide being “done half”, and just write to USDOE, your state and local LEA/administration. Request specifically when you want to hear a reply from state and local. Ensure marginalized, “invisible” members of our community get the mic.

    This is our responsibility, folks.

  5. ciedie aech    

    When NCLB/R2T mandates hit our reform-greedy district, over the years teachers were told to implement many different “grading” rubrics, none of which changed any long-term outcomes and many of which simply dumbed down the general grading of our students to such a degree that at times there might be no distinction between a student who had actually been in class completing assignments and a student who had been absent for four weeks.

  6. Chris    

    Society seeks a spectrum of educational outcomes including quantitative elements like academic knowledge and amorphous objectives such as citizenship. California’s efforts to rate the effectiveness of Common Core (despite rebranding) using a multi-faceted approach to capture elements like parent engagement and absenteeism should be applauded – with reservations. Quantifying subjective components opens the door to temptation to manipulate data described by Mr. Cody as Campbell’s Law. These elements of Common Core implementation and evaluation are rightfully debated. However, Mr. Cody’s mention of a graduation rate benchmarks seems uninspiring yet worthy of consideration. In accord with his argument, I support diminishing the emphasis on four year graduation rate to acknowledge those who may need an additional year to reach high school. In my dropout prevention classroom, students perform at elementary reading levels with no disability infringement. My school is collectively pressured to graduate students “on time.” This model inherently reinforces the archaic factory mode of education without acknowledging individual development. Optimistically, future education reforms will remove the four year graduation burden so educators can ensure that when students rightfully earn diplomas with the skills to justify their credentials – even it it takes some five years as opposed to four.

Leave a Reply to Chris Cancel reply