shadow

By Anthony Cody.

When I visited the Gates Foundation a few years ago, one phrase stuck with me. “You can only manage what you can measure,” I was told. This is the underlying framework of our 21st century accountability paradigm.

On the 2008 presidential primary campaign trail, candidates Obama and Clinton both garnered the loudest applause when they condemned No Child Left Behind. We started the Obama years hoping that test-centered accountability was on the ropes. But instead of a new paradigm, we got the Common Core and promises of “better tests.

Now, as the Common Core loses support even as these supposedly superior tests are just being rolled out, we are already seeing proposals for “new accountability systems,” offering a “new paradigm.” But before we celebrate the demise of test-centered accountability, we must cast a skeptical eye over these proposals. The consistent reappearance of a hard core of high stakes tests is a reminder of just how tough it is to escape a paradigm once it has become dominant. The very terms in which we view the problem have been defined by the paradigm, and to escape, they must be overthrown by new thinking. This revolution in thinking is not yet realized by these reports.

But there is a crisis afflicting the dominant paradigm that demands a response – even if the response is less than revolutionary.

Last month I reviewed a proposal authored by Marc Tucker, and noted that while he calls for reducing to three the number of high stakes tests, this number would still be sufficient to distort instruction and cause great harm. After all, the Chinese and Korean school systems likewise rely on only a few high stakes tests. But that does not prevent their system from being completely focused on these test scores. Yong Zhao added his perspective as well.

Another proposal was released last week entitled “Accountability for College and Career Readiness: Developing a New Paradigm.” It comes from Linda Darling-Hammond, Gene Wilhoit and Linda Pittenger. Linda Darling Hammond is an eminent scholar, who has written extensively about the need for teacher collaboration, and has done excellent work in the past in association with the Broader Bolder Approach, which has challenged test-centered accountability for the past six years. Wilhoit was on the ground floor of the Common Core process, as executive director of the Chief Council of State School Officers, and now works closely with Student Achievement Partners. Pittenger is the CEO of a human resources consulting firm called people3, Inc.

I know from her work that Darling-Hammond has a deep understanding of the need for teachers to collaborate in trusting environments. She has done outstanding research on the destructive effects of high stakes tests and VAM-based evaluation systems.

While there are some positive elements in this proposal, I think it falls short of the new paradigm the title promises in several ways.

On page 6, the report quotes the CCSSO Accountability Advisory Committee which states:

Each state should establish rigorous statewide measures of CCR (such as through Common Core-aligned assessments), but should also provide latitude for district innovation to expand on those measures to include additional indicators of CCR skills or dispositions deemed important by the local community.

So Common Core-aligned tests remain the foundation of state accountability systems, which can then be supplemented by local assessments.

Here is a fuller explanation:

In a new system of assessments, it should be possible to move from an overemphasis on external summative tests, even as they become better representations of what students should know and be able to do, to a greater emphasis on assessment that can shape and inform learning. This strategy will reduce the “overtesting” burden, shifting time and energy from external summative events to formative assessments that can be used in more efficient and effective ways. (See Figure 3 below.) To achieve these benefits, we will need to rely more on adjudication at the local level where learning occurs. This implies more trust of professionals who are highly trained and supported with judgment tools and processes, such as common rubrics along with moderation and auditing processes for evaluating student work consistently.

The frequency of tests is addressed here:

The system is premised on multiple measures, which include, as one component, robust local assessments that can evaluate deeper learning skills, as well as state standardized validations of student performance to verify the results of local assessments. Such state validation could occur every year for every child, or at points in the grade spans that represent critical developmental junctures (for example, grades 3 or 4, 7 or 8, and 11 or 12), or differentially, depending on local needs. State assessments employ matrix sampling so that judgments can be made about a broader and deeper set of skills without over-testing children. Disaggregation of results is part of the reporting system for assessments.

One clear benefit of moving away from annual tests is that would eliminate the use of discredited yet federally encouraged VAM systems for purposes of teacher evaluation. But tests can be less frequent and still remain intrusive, if high stakes are attached. The other positive element is that there is a suggestion that accountability be shifted to the local level. This echoes the model shared several years ago by Doug Christensen here.

There are two other aspects of the report that are worth mentioning. In the “51st state” the report offers as a hypothetical model, resources are allocated more equitably through “a weighted student funding formula that allocates funds based on pupil needs, allocating a greater weight
 to students living in poverty, English learners, and students in foster care.” California is now in the process of implementing such a funding system, and it moves us in a good direction, in that it recognizes that English learners and children living in poverty require more resources to educate, and provides additional resources based on these factors.

The other component of the system is the establishment of a school quality review system, where teams of “distinguished educators” perform inspections every five years, and offer guidance based on what they observe. This could also be a step forward, in that it could allow for a far more qualitative, less punitive approach to accountability.

One of my concerns about this system centers on the continued reliance on Common Core tests. While it is a marginal improvement to escape from annual tests, this alone will not free students and teachers from intense pressure to “shape and inform learning” in alignment with those tests. The report repeats the promises we have heard for the past four years about the new Common Core tests: “These promise to include more open-ended questions and tasks that 
can better evaluate higher-order thinking and performance skills than many state tests included in the past.” What we have seen thus far of these tests does not leave me inspired. Quite the opposite. When tests are designed to be “more rigorous,” the outcome seems to be to drastically lower the number of students rated as proficient.

Beyond the continued reliance on tests, there are indications that the “new paradigm” on offer will be quite onerous for teachers. Here is what the report offers as an example of this new paradigm in action:

Massachusetts’s new teacher evaluation process is tightly tied to these learning opportunities. One of the more sophisticated in the nation, it draws on evidence of teaching practice from observations, staff, and student feedback; teachers’ professional contributions; and multiple sources of evidence about student learning in a judgment system that is tied to goal-setting and professional learning.

We have learned in recent years that systems such as these that sound wonderful have a way of becoming nightmarish in practice. I asked some Massachusetts teachers for their experiences with this new system, and got an earful.

One teacher wrote me:

Only insights that “embrace” our purchased district-wide professional development may be communicated.  Only teacher-leaders pursuing assigned leadership roles are even considered for exemplary ratings.

Another explained:

On the new evaluation systems.

1. We have to establish DESE outlined S.M.A.R.T goals. SMART= Specific and strategic, Measurable, Action oriented, Rigorous, Realistic and Results-focused (the 3 R’s), Timed and tracked.

2. Our goals have to be tied to our students’ progress on the state
tests. Even the Health and Art teachers have to tie their goals to
the MCAS (and soon to be PARCC).

3. We build a timeline of actions (weekly to monthly) that we are taking to ensure we meet our goals.

4. At the end of the school year, we have to submit a portfolio demonstrating what we achieved.

5. This portfolio is reviewed by a building administrator (my portfolio from last spring was looked at in early September). An administrator makes the call whether we need improvement, are proficient, or exemplary.

All in all there are 12 different documents that need to be submitted.

After a long back and forth with my principal last year, my final accepted goal was: By June 2014 common and formal assessments will have been identified, implemented and analyzed collaboratively providing evidence of student growth of at least 3-5% in 6th grade English language arts.

Nonsense.

We are so short staffed and cut to the bone that I had one visit from
an administrator all year and that was because he had a question about a student. I have no idea how any observational reporting by admin was done.

The truly sad part about this whole process is, my administrator
clicks on a button in the program (MyLearningPlan) that says
“proficient” and that is all that goes to the State. We’ve taken to
calling documenting as filing in Warehouse 13.

As a professional, I would value feedback from families and students. I appreciate the kind notes from parents or students who return for a hug. We haven’t reached that point in the evaluation process, but I don’t trust the format for gathering right kind of “data.”

Yet another teacher responded:

Massachusetts’ evaluation system is pretty much straight out of the recommendations from the Gates Foundation’s MET project.

One of the big problems we have is that a lot of the Primary Evaluators have never had training or done the jobs of the people they are evaluating.  We have a person in our school who has never been a classroom teacher responsible for evaluating teachers.  We have a person who has never worked in a library or received any training in librarianship evaluating the librarians.  We also have administrators with one or two years experience in the classroom, or with no degrees in the subjects they are evaluating the teachers of, serving as Primary Evaluators.  The only qualification to be a Primary Evaluator is to hold an administrative degree.

The system is incredibly complex and stressful.  We have to write goals and submit them by a certain date, oftentimes falling around the time grades or progress reports are due.  I had an art teacher tell me the other day that she was so stressed out trying to figure out when she was going to get the paperwork done that she thought maybe she could do it during class and let the kids just work alone for a while.  She never does that, preferring to circulate and help them grow as artists.

We are also supposed to document student growth with DDM’s (district determined measures).  Every teacher is supposed to have at least two.  They need to be data-oriented and measurable.  Teachers of standardized tested subjects must use that test as one of theirs and also have another one.  Everyone else has to come up with two.   This poses problems for people like nurses and librarians who don’t have classes assigned to them.  We find we are having to make up new things to do, on top of what we already do, to meet the DDM mandates.

Another controversial aspect is the student and faculty surveys.  We have not been provided clear “guidance” on this from the state, but they do recommend this company (they’re hiring and they have great perks and benefits!) to help with administering the surveys.   I found the recommendation along with a potential cost to districts for administering the student and faculty surveys on the MA Dept of Ed website.

I work in what’s known as a “Gateway City” in MA.  These are primarily old industrial towns that have high immigrant populations and poverty rates.  We are laying off staff, our facilities are falling apart, but it seems the data collection and evaluations are generating a whole new industry.  A lot of MA systems are paying for software to manage the evaluations.  This one is popular; I think our district will be spending $10,000 on it this year.  Meanwhile, kids are rubber-banding and taping their English books together.

We are also spending more money on administrators to conduct these complex evaluations.  Teachers are focusing more on collecting “artifacts” and “evidence” and worrying about their evaluations than they should be (which in my mind should be almost never).

Furthermore:

…our state now wants to “transform teacher licensure” and tie it to this wonderful new evaluation system.  Mitchell Chester, our Commissioner of Ed (who also sits on the board of PARCC and was just appointed to the board of NEASC) was on a CCSSO task force about reforming licensure. I think if passed, their recommendations will pave a quick entry for TFA and other fly-by-night education operators.  The title of their task force report is “Our Responsibility, Our Promise.”  Tom Luna headed the task force and they are working to implement recommendations from it now in Idaho.

These evaluation systems are adding many hours of work to the lives of already overburdened teachers, with little benefit to anyone. It is a telling indicator of the state of the teaching profession that not one of these teachers felt secure enough to allow her name to be used in this piece.

On October 23, Diane Ravitch reported on this policy initiative and shared a message from the Massachusetts Teachers Association, which is organizing to resist. The MTA sent out a message to members, which reads in part:

 How would you feel about the prospect of losing your license to teach – not just your job – based on your evaluation and/or your students’ test scores? Several versions of just such a proposal were outlined in a document called “Design Principles and Policy Options” released by DESE on October 20, one day before the first of several DESE-controlled “town halls.” These town halls are part of DESE’s plan to propose a “performance-based” licensure system in the spring and implement it by October 2015.

To return to the proposal from Darling-Hammond, Wilhoit and Pittenger, I have to say that this does not appear to fulfill the promise made of a “new paradigm.” To understand why, we have to question the premises on which this new system is based. When we look, we see some of the same assumptions and solutions as are present in the current paradigm. In a manner similar to Marc Tucker’s recent proposal, we have the suggestion that we might have fewer tests (though even this is left up to the states to decide.) And on top of these tests we have elaborate evaluation systems that require teachers to set “measurable goals,” and provide extensive evidence for every aspect of their work. The models offered could possibly be implemented with lower stakes for tests, but there is not an explicit statement of this principle.

What this suggests is how difficult it is to escape the paradigm in which we are seeing education. We are stuck in a model that says learning must be measured to be managed, and management is the overriding systemic imperative. This necessitates top-down systems, even as those systems are incapable of delivering the sort of change advocates insist upon.

The report states:

The emerging paradigm for accountability must be anchored in this new vision for learning and should be coherently aligned to systemic changes implied by that goal. It should foster a culture of inquiry and continuous improvement at all levels of the system. This new accountability model must foster collaborative change that can transform schools from the industrial model of the past to innovative learning systems for the future. Accountability will need to build school capacity and enable thoughtful risk-taking informed by continuous evaluation to inform improvement.

There are hints of a new paradigm in this statement. We do need a culture of inquiry, collaboration and risk-taking. But the measure-to-manage top-down ethos will not get us there.

To get to a truly new paradigm we need to think beyond this top-down model. As Yong Zhao said in a recent interview,

…any high-stakes testing must come from an authoritative body that has the self-claimed authority and ability to prescribe what children should learn or should know, the tests by definition force students to comply with the answers or the way of thinking that the authority wants. Then you hold the students, the teachers and, to a lesser extent, the parents accountable for being able to get the answers that the authority wants and to show that they have mastered the skills and the knowledge and possibly even the beliefs that the authority wants. That’s how I think authoritarianism and testing are related.

The highest form of education is the pursuit of open-ended inquiry into original and creative solutions to problems we face. This is the most powerful form of professional development in which teachers can engage. We have living examples of where this sort of work has become the norm, and students are the real winners. But so long as the answers have been pre-determined, this approach is not available to teachers and students.

A truly new paradigm would invest confidence in students and teachers, rather than constantly require them to demonstrate their adherence to standards and predetermined curricula and assessments. A new paradigm would refocus our schools on the needs of local communities, and require educators to work closely with parents and community leaders to set goals and share evidence of student progress. Accountability invested in centralized authority is inherently top-down. New paradigm? Not there yet.

Author

Anthony Cody

Anthony Cody worked in the high poverty schools of Oakland, California, for 24 years, 18 of them as a middle school science teacher. He was one of the organizers of the Save Our Schools March in Washington, DC in 2011 and he is a founding member of The Network for Public Education. A graduate of UC Berkeley and San Jose State University, he now lives in Mendocino County, California.

Comments

  1. Lucianna Zephyr Sanson    

    Excellent article, as usual Anthony. I appreciate your ability to present all angles of a scenario without casting judgment. That is a sticky wicket in journalism. I also applaud your inclusion of teacher voices in this article. Thank you for providing a platform for teacher voices to be a heard- as we are the true experts in the field- not the “elite” that have the time to make the “paradigm.” Bravo. Kudos. Thank you my friend.

  2. 2old2tch    

    “A new paradigm would refocus our schools on the needs of local communities, and require educators to work closely with parents and community leaders to set goals and share evidence of student progress. ”

    Successful schools are part of a community. Schools and their communities share the commitment to a quality education through shared community-school governance. This paradigm is nothing new although it has not always functioned as structured. We are still being forced to submit to the worst of micromanagement structures in an attempt to control what cannot be measured well. We also must be honest about what our schools can accomplish. Teachers have been the scapegoats for problems beyond the scope of the classroom. No amount of professional development, no mission statement, no measure of student progress is going to eliminate poverty and the problems associated with despair.

  3. Sergio Flores    

    New paradigms come from qualified consensus and general acceptance. In this time and place, these draconian conditions do not appear in enough quantity to provide a nurturing momentum. For almost two decades, corporate reformers have kept teachers in public schools working in a stressful environment where they claim educational crisis and social blame find their causes in “bad teachers.” The logical outcome from working in such a toxic environment shows constant confusion, chronic frustration, and episodes of fear. Dissenters, critics, and independent thinkers find themselves with no serious opportunity to voice concerns, question, or challenge the status quo. At this point, the CCSS has reached a level of dogma from which nobody escapes unscathed. Administration and teachers association leaders embrace, promote, and defend CCSS –the corporate reformers’ final foundation for their new paradigm. Unless teachers brake lose from the chains of dogmatism, and educate on the issues rather than keep quietly obeying, complying, and trying to conform to the mandates, the possibility of a true new paradigm that includes concepts of fairness, equality, justice, inclusiveness, professional respect, and quality, will be impossible.

    1. Sergio Flores    

      In the second line, I meant to write: In this time and place, draconian policies do not allow for original ideas and consensus to appear in enough quantities to provide a nurturing momentum.

  4. Monty Neill    

    Thanks, Anthony. Indeed a new paradigm is needed. At a minimum we need Congress to allow the space for a new paradigm to exist, which means reducing testing and replacing ‘accountability’ with ways to improve that are locally shaped – then the battle can shift to states.

    Meanwhile, the drive for teacher eval assumes it will lead to improved learning (being kind: clearly in most cases proponents mean higher test scores). But Murphy says there is a lack of evidence that teacher evaluation leads to improved student outcomes (even test scores). There has been good information on positive benefits of Montgomery County MD’s PAR process, which focuses on teachers collaborating to improve – really, not the same thing as the ‘evaluations’.

    Murphy, J., Hallinger, P, and Heck, R.H. 2013. “Leading via Teacher Evaluation: The Case of the Missing
    Clothes?”Educational Reseracher, V.42, N.6.

  5. anthonycody2013    

    Monty,
    Thanks. What I would argue is that what is really needed is a robust culture of professional inquiry, strongly tied to community needs and participation. When this culture is alive and vibrant in a school, the need for intrusive evaluation systems will be greatly reduced. Teachers should be looking at how students are learning — but not in the context of an onerous evaluation system that requires them to justify their existence through measurable goals — which translate into test scores. There are far more revealing ways to look at how students are learning, and improve instruction. Processes like Lesson Study and Teacher Inquiry do this by making teachers active investigators into their practice. When this sort of process is in place, good and great teachers are encouraged to stay, and the culture of the school improves dramatically.

  6. Monty Neill    

    Such things as you describe, Kevin, should be at the core. I do think schools and districts need some transparency, and that if a school is ill-serving its students (based on multiple kinds of evidence gathered over time – I won’t elaborate here, but see http://fairtest.org/fact-sheet-better-way-evaluate-schools-pdf), has been given assistance (not sanctions, not one-size fits all menus a la ‘waivers’ and NCLB), and still cannot do a good job, then state intervention is warranted, indeed should be required.

  7. anncberlak    

    Anthony Cody writes: “Linda Darling Hammond is an eminent scholar, who has … done excellent work in the past in association with the Broader Bolder Approach, which has challenged test-centered accountability for the past six years. …
    I know from her work that Darling-Hammond has a deep understanding of the need for teachers to collaborate in trusting environments. She has done outstanding research on the destructive effects of high stakes tests and VAM-based evaluation systems.”

    I am continually puzzled at portrayals of Linda Darling Hammond as one who challenges the dominant paradigm. It makes so much more sense to see her as, when all is said and done, a devote of the view that “you can only manage what you can measure, the underlying framework of our 21st century accountability paradigm.” She was, after all, the progenitor of PACT, the edTPA and the TPA. (See Rethinking Schools, Summer 2010, “Coming soon to your favorite credential program: Teacher performance assessment”).

  8. Sarah    

    I refer you to my standards. I think this might be somewhat of a paradigm shift and certainly is the paradigm for the wealthy. http://sarahpuglisi.blogspot.com/2013/01/best-of.html?q=mrs+puglisi%27s+100+standards

  9. Randal Hendee    

    “Accountability will need to build school capacity and enable thoughtful risk-taking informed by continuous evaluation to inform improvement.”

    This sentence is gibberish. William Zinsser would have a field day with it. It’s a mess of buzzwords lacking any human actions or actors. And it’s at war with itself. Accountability as they conceive it is completely at odds with risk-taking, thoughtful or otherwise. And using the word “inform” as they do–twice–has the effect of tying the sentence into a knot that can’t be untied. No. “Risk-taking” will never escape that straitjacket.

    Read between the buzzwords, though, and you end up with exactly the kind of suffocating clutter and pointless exercises in paperwork reported by the teachers you interviewed. I still don’t know who Marc Tucker is, and I don’t think I want to know. Now that I’ve read your article, I at least know this about Linda Darling-Hammond, Gene Wilhoit, and Linda Pittenger: They don’t have the first clue about the realities of teaching today. Nor do they have any apparent idea of what genuine learning is nor how it might occur if kids and teachers were unshackled from the wrongheaded systems these writers are hoping to tweak. Hints of a new paradigm? You’re giving them way too much credit.

  10. Richard Davidson    

    Amen Amen Amen! “A new paradigm would refocus our schools on the needs of local communities and require educators to work closely with parents and local community leaders to set goals and share evidence of student growth.” Efficiency and value would return to the funding process. Teacher/student relationship is fundamental. Get the Gates and the Waltons and the Duncans out of the middle. Take out the racism and elitism. The Top Down, “you can only manage what you can measure” model only exists because those who hoard resources can’t stand to allocate to the less fortunate, the less privileged. They resent giving to much to children who are not as easy to teach as their own. They insist on retaining a full measure of control over–and credit for–what they do give. They want to hog the limelight away from the deserving: the children whose own sweat & tears earn them their birthright: an educated mind and an illuminated soul.

Leave a Reply