shadow

By Anthony Cody.

When the history of modern education reform is written one of the most shameful chapters will be the continued embrace of various forms of “Value Added Models” for purposes of measuring the effectiveness of teachers in raising test scores. This month, the Department of Education is asking for comments on its intention to bring this pseudoscience to bear on the field of teacher education.

The proposal states that the new regulations will evaluate teacher education programs based on the following criteria:

  • Employment outcomes: New teacher placement and three-year retention rates in high-need schools and in all schools.

  • New teacher and employer feedback: Surveys on the effectiveness of preparation.

  • Student learning outcomes: Impact of new teachers as measured by student growth, teacher evaluation, or both.

  • Assurance of specialized accreditation or evidence that a program produces high-quality candidates.

This means that just as states have been required to develop systems that incorporate test scores into teacher evaluations, they will be required to likewise measure the “effectiveness” of teacher education programs based on the test scores of the teachers they produce.

Let’s take a look at the arguments and research presented, and the research that has been ignored. First we have this graphic: HQT1

I went to the link provided, and found a reference to a 2013 study by Goldhaber, Liddle and Theobald. The full study costs $19.95, so I did not purchase it. [Note: See update below] However, the summary of the study states this:

This paper presents the results of research investigating the relationship between teachers who graduate from different training programs and student achievement on state reading and math tests. Using a novel methodology that allows teacher training effects to decay, we find that training institution indicators explain a statistically significant portion of the variation in student achievement in reading, but not in math. (emphasis added)

How does this support the assertion in the graphic that the impact of learning gains in math was greater than the effect of poverty? What am I missing here?

The other sidebar graphic is this one, which cites a non-peer-reviewed report from TNTP, and makes the following assertion: “Teachers in the top 20% of performance generate 5-6 more months of student learning each year than low-performing teachers.”TeachersTop20

Matthew DiCarlo at the Shanker blog took a look at the methods of this “study,” and found the following:

…most of their estimates (three out of four districts) are based on only one year of data, and while these scores are hardly useless, it’s not quite appropriate to draw any strong conclusions about teachers’ effectiveness with such small samples, and that includes grand labels such as “irreplaceable.” For instance, a decent-sized proportion of these teachers will not make the “irreplaceable” cut the following year, due mostly to error rather than “real” change in performance.

 

This brings us to one of the most egregious problems with VAM systems. Research shows that student test scores are affected by a large number of factors, and teachers only account for about ten percent of the variation. The lion’s share of effect are out of school factors related to poverty – which reformers have decided we are incapable of addressing in any meaningful way.

Given that students are not assigned or distributed to teachers randomly, any given year a teacher might get a “tough class,” and their scores might plummet. So teachers who are fortunate to be designated “irreplaceable” one year may find themselves on the hit list the next year.

This report from Linda Darling-Hammond, Audrey Amrein-Beardsley, Edward Haertel, and Jesse Rothstein found:

 A study examining data from five school districts found, for example, that of teachers who scored in the bottom 20% of rankings in one year, only 20% to 30% had similar ratings the next year, while 25% to 45% of these teachers moved to the top part of the distribution, scoring well above average. (See Figure 1.) The same was true for those who scored at the top of the distribution in one year: A small minority stayed in the same rating band the following year, while most scores moved to other parts of the distribution.

The other problem is that certain sorts of students have a harder time improving their test scores than others. Research – detailed here, has shown that special education students and English learners are especially hard to move upwards.

Overall, the study found that, in this system:

  • Teachers of grades in which English language learners (ELLs) are transitioned into mainstreamed classrooms are the least likely to show “added value.”

  • Teachers of large numbers of special education students in mainstreamed classrooms are also found to have lower “value-added” scores, on average.

  • Teachers of gifted students show little value-added because their students are already near the top of the test score range.

  • Ratings change considerably when teachers change grade levels, often from “ineffective” to “effective” and vice versa.

We are already seeing the effects of using VAM scores to evaluate individual teachers. Teachers who are recognized as being resourceful and creative all of a sudden are thrown out for their low test scores.

What will happen when these systems are applied to teacher education?

First of all, programs will be obliged to recognize effective test preparation as a necessary element in their institutional survival. That is the clear intention of this “reform.” Programs that do not enthusiastically support implementation of the Common Core and other test-centered reforms are likely to lose access to funds and support – both state and federal. Programs which serve low income populations will be caught in a bind. While they may get some points for serving in high needs schools, the lower test scores these students tend to get, and the lower growth rates are likely to bring their “effectiveness ratings” downward.

A bit of history is in order. Where did this terrible idea come from?

There has been a longstanding frustration on the part of reformers with parts of the education system that are out of their direct control. Schools of Education are housed within universities, and are not directly funded by the Department of Education. Thus neither the bribes of Race to the Top billions nor the threat of NCLB waivers have been enough to completely overthrow many years of education scholarship. An organization called the National Council on Teacher Quality (NCTQ) was created to overcome this institutional resistance. Diane Ravitch was there at its birth, and has explained its genesis. According to Ravitch, NCTQ was created by the Thomas B. Fordham (TBF) Foundation because

We thought (schools of education) were too touchy-feely, too concerned about self-esteem and social justice and not concerned enough with basic skills and academics. In 1997, we had commissioned a Public Agenda study called “Different Drummers“; this study chided professors of education because they didn’t care much about discipline and safety and were more concerned with how children learn rather than what they learned. TBF established NCTQ as a new entity to promote alternative certification and to break the power of the hated ed schools.

Kate Walsh, the head of NCTQ, expressed this perspective when discussing a report the project issued in 2012:

A lot of schools of education continue to become quite oppositional to the notion of standardized tests, even though they have very much become a reality in K-12 schools. The ideological resistance is critical.

So NCTQ recommended what the Department of Education now plans to enact – that schools of education should be ranked according to their VAM scores, and be punished or rewarded according to these indicators of “effectiveness.”

This is the blunt force of federal dollars being used to overcome “ideological resistance” to the centrality of test scores.

Secretary Duncan occasionally makes speeches in which he says we should not teach to the test. He even wrote recently,

…testing should never be the main focus of our schools. Educators work all day to inspire, to intrigue, to know their students – not just in a few subjects, and not just in “academic” areas. There’s a whole world of skills that tests can never touch that are vital to students’ success. No test will ever measure what a student is, or can be. It’s simply one measure of one kind of progress. Yet in too many places, testing itself has become a distraction from the work it is meant to support.

We have heard this sort of talk before. But when this is followed by policies that reinforce the centrality of test scores as a means of measuring teacher effectiveness, these words are not just hollow, they are downright hypocritical.

There is a window of opportunity to comment on this proposal. Everyone associated with teacher education ought to comment. Professors – this is a perfect opportunity to acquaint your students with the policies that will impact their careers in the years to come. Student teachers, challenge your professors to take a stand. Comments should be sent to this address: OIRA_DOCKET@omb.eop.gov by January 2, 2015.

Update, Dec. 11, 2014: A reader provided me with a copy of the Goldhaber report cited in the graphic which claims that “the impact of teacher preparation in Math was considerably greater than the effect of poverty.” Here is the passage from that study that this assertion is based upon:

…there are a small number of programs that can be distinguished from teachers trained out-of-state, and the magnitudes of these differences are educationally meaningful. The point estimates, for example, suggest that the regression-adjusted difference between teachers who received a credential from a program with the lowest performing teachers and those who received a credential from the program with the highest performing teachers is about 12% of a standard deviation in math and 19% in reading. In math, this difference is 1.5 times larger than the regression-adjusted difference in performance between students eligible for free or reduced-price lunches and those who are not; in reading the difference is 2.3 times larger. So, while the bulk of our findings contribute to the growing literature demonstrating that observable teacher characteristics are only weakly correlated with teacher effectiveness, the striking differences in the effectiveness of teachers from programs at the tails of the distribution in Washington State hint at the potential of teacher training to influence student achievement.

This finding is then blown up and turned into the assertion that teacher preparation can overcome the effects of poverty! The authors of the study state in their summary that there is NO significant difference in math between different teacher training programs, but the Department of Ed has cherry-picked one small finding and blown it up out of all proportion. Is it ny wonder that people are losing trust in this “data-driven” operation?

Author

Anthony Cody
Anthony Cody

Anthony Cody worked in the high poverty schools of Oakland, California, for 24 years, 18 of them as a middle school science teacher. He was one of the organizers of the Save Our Schools March in Washington, DC in 2011 and he is a founding member of The Network for Public Education. A graduate of UC Berkeley and San Jose State University, he now lives in Mendocino County, California.

Comments

  1. jonathanlovell    

    Hi Anthony,

    I believe that this window will be open until Feb 5. At least that’s what I was told by my Dean here at San Jose Satte and have been telling those I’ve been emailing. My own comment, as well as my Dean’s comment, will be appearing on successive days on Diane Ravitch’s blog. So I hope I’m right!

  2. jonathanlovell    

    Hi Again Anthony,
    The date when the comment window will close down appears to be February 2, 2015. The link we’ve been providing for those intersested is https://www.federalregister.gov/articles/2014/12/03/2014-28218/teacher-preparation-issues

  3. jonathanlovell    

    One final time, Anthony,

    Here’s the email chain with my dean and Diane from last Monday. Sorry it’s so long!
    My best,
    Jonathan

    On Dec 8, 2014, at 10:34 AM Jonathan Lovell wrote:
    Dear Diane,

    Thanks very much. It would be quite effective, I think, to have thousands of well informed negative responses posted to the USDOE website re this proposal between now and Feb 5.

    With affection and regard,
    Jonathan

    On Dec 8, 2014, at 9:54 AM, Diane Ravitch wrote:
    thanks, will post both, on different days
    Diane

    On Mon, Dec 8, 2014 at 11:47 AM, Jonathan Lovell wrote:
    Hi Diane,

    At the urging of our Dean of the College of Arts and Humanities, Lisa Vollendorf, I just posted my response to this proposal to change how teacher training programs are evaluated. The USDOE’s website for posting responses is surprisingly easy to use (see https://www.federalregister.gov/articles/2014/12/03/2014-28218/teacher-preparation-issues ).

    The deadline for posting responses is Feb 2, and I would urge EVERYONE to do so. I’ll copy and paste my own response, as well as Dean Vollendorf’s, below. Thanks so much for your continuing great work!

    With affection and regard.
    Jonathan

    Dear Secretary Duncan,

    As a teacher educator for the past 35 years in the field of English Education–having spent the past 27 of those years in my present institution of San Jose State University–and as someone who has observed upwards of over 2500 middle and high school English classes over the course of my career, I can say without qualification that the proposed new regulations for assessing the quality of teacher preparation programs would be an unmitigated disaster.

    Others on this site have spoken eloquently about the extremely serious effects, especially to public institutions like ours, of the costs of implementing these regulations, as well as the wrong-headedness of linking the assessment of teacher preparation programs to a VAM-like measure of student performance in the classes of recent graduates.

    I’d like to address a related issue, but one that has been strangely left out of the public discussion to date. It’s the effect of these new regulations on what might be called the “climate” in which teaching as a profession is perceived.

    As I’m sure you are aware, there has already been a 50% decline over the past five years in the number of applicants to teacher education programs in the state of California. While we’ve countered this trend in the English Education program here at San Jose State, we’ve only been able to do so by focusing relentlessly of what helps beginning teachers improve not only their instructional practices, but their sense of personal agency in their chosen profession. The major player in this effort has been the San Jose Area Writing Project, which routinely selects exemplary K-12 teachers for an intensive four and one half week summer institute, then positions these teachers in significant roles in the preparation of new teachers.

    Your proposed new regulations will be perceived as yet another iteration of all-too-familiar Bush-Obama refrain that “the whipping will continue until morale improves.” I believe it’s high time to turn from this thoroughly discredited approach to the improvement of teaching and teacher training, and to start looking more sensibly and honestly at practices that work.

    The practices of the National Writing Project would be an excellent place to start.

    Yours truly,
    Jonathan Lovell
    Professor of English and Director of the San Jose Area Writing Project
    San Jose State University

    And here’s what the dean posted:

    As a committed educator who has devoted her life to public higher education, I am dismayed by the onerous requirements put forth by this proposal. At San Jose State University, which is part of the 23-campus California State University system, we will find it fiscally impossible to comply with so many requirements. In particular, it will cost us much more than we can afford to track our graduates. Moreover, we are deeply troubled by the connection between accreditation for teacher credentialing programs and the test scores of those teachers’ students. The CSU is the largest four-year public higher education system in the nation, and we are committed to affordability and access. That commitment translates into recruiting and training students who are in turn committed to working throughout the community, including in low-income and under-served areas of our K-12 system. By tying the test scores of those children to our accreditation standing, the federal government is sending the message that the only students we should be serving are those who are lucky enough to live in privileged areas with a strong tradition of good schools. I am proud to educate diverse students from all walks of life, and proud when they go out into the diverse communities from which they hail to give back and make society better. These new regulations will disincentive programs and teachers from serving those communities. Please reconsider this overall plan and think again about the adverse effects on those who most need improved schools and those who prepare teachers to work in those under-served communities. Public institutions will be so hard hit by these regulations that we are concerned that we will no longer be able to afford credentialing programs.
    Lisa Vollendorf, PhD

  4. Karl Wheatley    

    Ah yes, now teacher educators will be required make all teachers above average, just as we now expect teachers to make all their pupils above average. And we will judge above average teachers based on scores on high-stakes standardized tests, despite voluminous evidence that if we wanted to truly improve education, we would be getting rid of high-stakes standardized tests.

    What the DOE is recommending is that we base educational policy on bad science, a misreading of science, and pure ideology.

    The National Academy of Sciences has scolded educational policymakers for basing these test-centric accountability policies on ideology not evidence. It is not the teacher educators but rather the policymakers who are implementing policies that are based on the ideologies and financial interests of the large campaign donors.

    The American taxpayers and American children deserve much better than that the DOE perpetuate the failed status quo of test-driven schooling.

  5. jonathanlovell    

    Dear Anthony,
    If you can send your email address to Jonathan.Lovell@sjsu.edu (I’m certain I had it at one time, but perhaps not), I’ll avoid sending these excessively long posts to your blog and FaceBook page. Thanks so much, and looking forward to reading and recommending The Educator And The Oligarch: A Teacher Challenges The Gates Foundation! My best, Jonathan

  6. Anthony Cody

    Anthony Cody    

    Jonathan,
    Thanks for your comments.

    Here is what the site says down at the bottom (http://www.ed.gov/teacherprep), which is a bit confusing. But it seems to indicate that comments received by Jan 2, 2015, will “be considered.”

    Proposed Rule: Teacher Preparation Issues in the Federal Register (Comments due 2/2/2015) The proposed Institutional and State Report Cards may also be viewed on regulations.gov. However, comments on the proposed Report Cards should be emailed to OIRA_DOCKET@omb.eop.gov. OMB is required to make a decision concerning the collections of information contained in the proposed regulations between 30 and 60 days after publication of this proposed rule. Therefore, to ensure that OMB gives your comments full consideration, it is important that OMB receives your comments by January 2, 2015.

    So perhaps I am misinterpreting this — but it looks like comments will be accepted into February, but the ones received by Jan. 2 will actually be considered. I want to maximize the impact our comments will have.

  7. Paul "Matt" Chonka    

    Anthony,

    Great peice on the absurdity of using a process that only gives accurate results if the variables remain the same in a profession where the students are always changing.

  8. Ray Brown, M.A. Bilingual Resource Specialist (Just Retired)    

    I think it is horrendous that credentialing programs, in our universities, will be analyzed under a microscope over scores of teachers in the credential programs. It is also very fatuous to give a failing grade to a university’s credential program or to a teacher who has one bad year. What about the teacher that works in a bilingual school, in a high poverty area and teaches special education students?Teachers in special education get children two, three and more years behind their regular education counterparts. Teachers everyday work with children who come to school very hungry and are late. As a teacher, I always paid out of my own pocket to buy nutritional foods for the children that were late for the school breakfast. One can call the parent and they are in a constant rush to get to their low paying job and never have the skills or organization to get their children to school on time. Who do we always blame? The Teacher!
    Children in special education, and some in regular education can have a variety of disabilities.
    One can have some of the following: 1.Attention disabilities 2. Organizational Deficit 3.Auditory Processing Deficits 4. Visual and Visual-Motor Deficits. 5.Language Impaired 6.Verbal Task Deficits 7.Memory Deficits (I can go on-and-on with mitigating circumstances and there are clear strategies to work with these children, but many regular education teachers have so many children now that it is hard for them to deal with all the needs they will face. Couple this with all the children learning English, from bilingual countries and it seems that the Federal Government wants us to have all children reading at the par of the regular education child when it takes approximately seven years to really learn a language. Now the Common Core tests in my state do not permit accommodations or modifications on the test for the child who has disabilities or is just learning the language. The Federal Government needs to stay out of education and let each state, without pulling funds from our poorest children, decide what is best for the child. The teachers working with these wonderful children know better, and have experience working with these children, not an Arnie Duncan who has never been a teacher in his life. Don’t destroy our wonderful universities and there credential programs nor destroy education for the benefit of creating more unproven schools. Let the professors and teachers do what they know is best: Let the teachers teach! Ray Brown, M.A. Bilingual Resource Specialist. (Just retired)

Leave a Reply