By Anthony Cody.
This week leaders in the California Department of Education released a new system for “grading schools” that proposes to assign schools a set of colored indicators based on a variety of measurements. This system is intended to replace the old system of performance indicators which gave schools a single numeric score, (the Academic Performance Index, or API) based only on test scores. While this new system brings us elements of that long sought goal of “multiple measures,” I fear that we have yet to escape the box of the “measure to manage” paradigm.
Over the past eight years the state of California has resisted some of the worst aspects of Federal reform. The state resisted federal pressure to tie teacher evaluations to student test scores, and Jerry Brown was a vocal critic of Arne Duncan’s Race to the Top, as indicated by his statement in 2009. In more recent years, Brown has restructured school funding, to direct resources to school districts based on student needs, as influenced by factors such as poverty and the proportion of language learners enrolled.
But the state – and our state teacher union, the California Teachers Association, have also embraced the Common Core (renamed “California Standards” last year.) With cooperation from the state university system, the California Teachers Association and funding from the Gates Foundation, teachers have been treated to days devoted to celebrating the Common Core. This year teachers will get yet another day-long Gates-funded Common Core festival, though once again they are the national standards that dare not speak their name.
California teachers and students have been told that a new and much improved accountability system is in the works, and now the draft has finally arrived. The mandate for this system is in regulations created by the federal Department of Education, which asks states to create a system to show “Measures of School Quality,” described this way:
Increased state flexibility to take a more holistic view of school performance based on multiple measures including achievement in reading and math; academic progress in elementary and middle schools; graduation rates in high schools; rates of progress for English learners achieving language proficiency; and a state-determined indicator of school quality or student success.
The regulations further ask that:
States create a multi-level rating system that clearly communicates to parents and communicates how their schools are doing, taking into account all of the measures of school performance.
Information displayed in a timely manner on annual report cards, designed with input from parents.
The new system being proposed for California schools was designed by technicians at West Ed, and it creates a matrix of color-coded squares that indicate both the absolute status and the direction of change for ten different categories of data. Thus we get a system with ten categories of information, and seventeen color coded boxes. The categories are:
- ELA assessment (K-8) (scores on Common Core aligned SBAC tests)
- Math assessment (K-8) (scores on Common Core aligned SBAC tests)
- English learner proficiency (scores on CELDT tests)
- Graduation rate (9-12)
- Chronic absenteeism (K-8)
- Suspension Rate & Local Climate Survey
- College & Career Readiness (scores on 11th grade Common Core aligned SBAC tests, plus other indicators)
- Basics (Teachers, Instructional Materials, Facilities)
- Implementation of Academic Standards
- Parent Engagement
Here is what a sample report would look like:
This presentation from the July 13 State Board of Education meeting provides greater detail — you can download here:. CaliforniaModelpresentation
In thinking about this proposal, it is important to recall what it is going to replace, which was a single number that was assigned to each school, derived entirely from standardized test scores. We have long argued that education is far more complex, and here we have a system that attempts to grapple with some of that complexity. There are indicators for local climate – derived in part from surveys which measure student engagement – this should be a major focus for every school.
The category “Basics” is the one thing on the list that might be considered an input. How well resourced is the school? What is the level of education and experience of the faculty? These are critically important variables. If the new funding formula is effective at redirecting resources towards schools with the highest needs, we should see improvements in some aspects of this.
I wonder what we might want to include that is not here. What about an indicator of school stability? What is the level of staff and administrator turnover from year to year? Student success correlates positively with stability, so this would be a useful indicator.
I want to back up a bit though, and reflect about what was so problematic about the prior system we had in place. First of all it was only based on test scores, and performance on those scores was largely determined by the income and parental education level of the students that attended the school. Thus the API score was more an indicator of affluence than of school quality. In this proposed system, this will remain true for all the indicators associated with test scores.
There is a second huge problem with these sorts of indicators, and it is called Campbell’s Law. This law states:
The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.
Corruption pressure means that those under pressure to improve these outcomes will find ways to “game” the system. So let’s look at these indicators and see how they are likely to be gamed.
All of the indicators associated with test scores are already being gamed by teaching to the test. The SBAC tests have been designed so that only about a third of students will score proficient or better, so there will be intense pressure to figure out how to improve those scores. This is by design, and it is a basic corruption of our mission as educators.
When English Language Proficiency is scored, there is pressure to reclassify students up and out of their status as English learners. This may result in students being reclassified before they are ready. California has particular challenges with English learners, who make up about 30% of public school students. I asked a teacher in the Los Angeles area, Jeanne Berrong, for her thoughts, and she said:
School systems want English learners to be reclassified ASAP with little or no regard to the realities of language acquisition. This is likely to be intensified if this is used as a factor indicating school quality.
Similar issues affect the high school graduation ratings, which score schools on their ability to graduate students “on time” in four years.* I asked Carol Burris for her thoughts on this, because she recently retired as a principal in New York state. She said,
When you use graduation rates to rate schools, it is very tempting and easy to game the system. It is one of the few measures you can control. Schools create credit recovery systems, and teachers are pressured into giving course credit to students for doing a poster or worksheets.
With English language learners, some come in well prepared and have no trouble graduating in four years. However, you also get immigrants who may not have had a solid education in their native country, and with this system you have to push them out in four years even if they are not really ready to graduate.
There is no sound educational reason to insist that students graduate in four years. It is far better to have well prepared responsible graduates than to have students pushed through to raise a graduation rate so that a school does not get in trouble.
I also want to point out a basic flaw in the “multiple measures” mantra. We have heard this language before, in relation to teacher evaluations. Supposedly, if test scores are only a part of a set of “multiple measures” then this will prevent those scores from causing harm. But just as in teacher evaluations, if thirty or forty percent of your score comes from test scores, this will be a major focus for your work. And just as with teacher evaluations, if your school has large number of students living in poverty, large numbers of English learners, large numbers of special education students, all these factors will drive your scores downward.
So while it is good to see an attempt to recognize the complexity of the school environment, we remain hamstrung by the limits of measurable outcomes. We must remind ourselves that there are many dimensions of learning not measured by tests, or indicated by graduation rates, or other quantitative outcomes.
Richard Rothstein wrote a book several years ago entitled “Grading Education, Getting Accountability Right.” In this book, he pointed out that society actually wants the following things from public schools:
- Basic academic skills and knowledge
- Critical thinking and problem solving
- Appreciation of the arts and literature
- Preparation for skilled employment
- Social skills and work ethic
- Citizenship and community responsibility
- Physical health
- Emotional health
Of this list, standardized tests measure only the first. The makers of the current generation of tests also claim that they measure “college and career readiness,” but the PARCC test was recently found to fall far short, and there is no evidence that the SBAC test is any better. None of the other desired outcomes is addressed by the ten categories on the West Ed matrix.
Above all, we have to ask ourselves what purpose these school grades will serve. I asked Dr. Stephen Krashen for his thoughts, and he cut to the heart of the matter.
The WestEd “California Model” for rating public schools was created by people with too much time on their hands, and who have no idea about what is important in education. Common sense, as well as a great deal of research, tells us that the major problem in education is poverty.
Children living in high poverty have poor diets, inadequate medical care, and little access to books; the best teaching in the world will has little impact when children are hungry, ill and have nothing to read.
If we substantially reduce poverty or at least protect children from the effects of poverty, all the boxes in the California Model table will be blue (or at least green) for all students. Any evaluation system will give similar results. Evidence for this includes studies showing that American middle class students attending well-funded schools do very well on international tests. They would be colored green or blue in all the WestEd categories.
Forget fancy evaluation schemes. Focus on providing better food programs, more school nurses, and invest in school libraries and librarians. Work on the cause, not the effect; feed the animal, don’t spend time and money on more precisely measuring what we already know.
We continue to labor under the illusion that some system of accountability — some carefully designed combination of carrots, sticks and measurable outcomes — will drive our schools to constantly improve. As teachers, we can always use timely information about what and how our students are learning, but this never comes from standardized tests. In our communities, we need meaningful information about how our schools are doing on a variety of dimensions — again, these are not reflected in test scores, But more than that, we need attention to the underlying conditions described so well by Dr. Krashen. Let’s feed the children, make sure they are not afraid of violence in their neighborhoods, allow them to learn about their cultures through Ethnic Studies, stabilize and support their teachers, and watch them learn and grow. As Sir Ken Robinson pointed out, teaching is like growing a garden. You just have to create the conditions for growth. Let’s focus on those conditions and stop obsessing over test scores.
What do you think?
* Note: California hopes to use a 5 year model, but the federal regulations state that you need to include 4 year graduation rates as well: