“Value Added” is wrong, not funny
The UFT took a good ad with the wrong graphic yesterday. We were right to push back, that was necessary. But not with this:
1. It is not NYC’s Value Added formula. It is a generic Value Added formula. Not sure if I’m right? The UFT calls it a “sample”.
2. This formula, with even more terms and symbols and subscripts and Greek letters, this formula on steroids – is what the UFT and NYSUT are helping New York State develop.
3. I can read this. I can teach you to, if you’d like. It’s hard, but not that hard. But it’s probably not worth it.
4. We are teachers, dammit. We don’t make fun of learning or knowledge. Or at least most of us don’t.
5. This great big formula is designed to see what difference each teacher makes “on the test.” That is wrong. Completely wrong. There is much more to teaching than “the test.”
6. Even if we cared more about tests, not THESE tests. I’m not going to start.
7. The text of the ad was good! But the graphic was awful.
8. I wish the UFT really thought “That’s no way to evaluate a teacher!” Our work at the State level speaks otherwise.
9. I have no illusion that what comes out of Albany will be any better. Perhaps State Ed will describe their formula in words so that no one can make posters out of the scary equation. That way we can focus on what’s wrong with the entire concept, and stop making fun of math (though honestly, that number-free formula is not from a math course. Stats for sociology or psychology majors maybe?)
JD2718 
They will never be a formula that can predict outcomes of human behavior. We might have routines; we’re creatures of habits. But, to predict an accurate, discrete data of a person, let alone a child, whose propensities will constantly change as he/she grows, is not realistic. It should not be used. Using it will achieve only more confusion, distorted facts, and devalue the relationship that teachers “naturally” have with their students.
Great post.
Putting aside all the stupidity of using a straight linear model (at least use a logistic-style model to estimate the probability of hitting a particular grade) in an inherently stochastic situation (I guess they don’t cover Wilkie, etc., in education doctorate programs), the accuracy of this type model is often choatic — seems to work most of the time (“work” loosely defined here) but can really screw up the borderline and corner cases. Don’t get me started on the attenuation and stability problems with applying linear regression to situations where there are meaningful errors in the measurements of the independent variables (“What do you mean the pre-test doesn’t exactly and perfectly measure what I say it’s measuring?”). And then there are the error term assumption issues…
Obviously the problem is the core belief that everything can be reduced to a useful number (or at least an {A/B/C/D/F} order statistic). Even in the business world metrics are generally understood to be indicators not conclusions. The biggest business disasters are often tied to unmoderated reliance on “objective” measurements that workers have figured out how to end run or to bad assumptions about the robustness of the formal model generating the expected values.
Bottom line: Tell your employees that their performance will be determined by a rigid model, they immediately begin experimenting to determine the fastest and easiest ways to maximize performance without regard to the best interests of the organization or its clients. If the model is flawed, the results will be what is desired only by accident.
This is not the NYC Teacher Data Report model, as might be evident from the inclusion of a district effect. You can see the model in http://schools.nyc.gov/NR/rdonlyres/A62750A4-B5F5-43C7-B9A3-F2B55CDF8949/87046/TDINYCTechnicalReportFinal072010.pdf. No sociologists were involved in the development or estimation of the model — it’s almost exclusively economists. There’s a lot to criticize about the modeling, but I encourage doing so only after reading the technical report.
They are looking for effects on test scores – the entire premise is wrong. The quality of the model touches on the technical details – of a project that is wrong in its fundamental conception.
(And, for the record, those are fairly lousy correlations they are touting… talk about low standards!)
Although I agree that one should not be making fun of mathematics, the application of such formulas to rate teachers is an appropriate subject for derision. And, given the general public that we have, including the ones who read the WSJ, there may have been no better way to spotlight the absurdity.
I say this as a person who spent much of his life dealing with much more complex mathematical operations, but never under the illusion that these could be appropriate or in any way complete if applied outside the quantitative sciences. I also agree that the union is screaming foul too late and that it should never have agreed to enter into any sort of collaboration in this direction.
I am aware that all of us are evaluated by actuarial formulas that determine such things as pension payments and insurance premiums. There too, one may take issue. But one is fully aware that the reasoning there is probabilistic. In this case, we are assessing individuals based on faulty statistical data on their (alleged) students and using this to rate job performance and hence to decide on employment. What’s worse, we are now making this publicly available.
The whole business of evaluating teachers needs to be revisited. It was flawed and highly subjective when we relied on supervisors’ observations. It is going to get even more flawed now. The purpose of required qualifications, hiring interviews and (reasonable) probationary periods is to select workers of some merit and capability. At some point, they need to be left free to do their job as they best think fit. While we can never do away with some sort of periodic evaluations and have to be cognizant to the fact that there will always be a few who are derelict who need to be checked and, if need be, let go, this will never be a worthwhile profession if we are subject to this kind of pressure.
I wrote a reply to Andrew Cuomo on receiving on of his cock-crowing e-mails. I suggest that all who receive such letters do the same, taking care to remain civil and checking spelling and grammar before sending their replies, so as not to get splashed on the pages of Murdoch’s NY Post or other tawdry rag.
One must not forget the bigger picture — that this is part of the nationwide (and global) assault on what’s left of the unions and, more generally, part of the War on Workers (to lift some phraseology from the warmongers) that has been proceeding apace from the 1980’s in this country, effectively pushing back against the gains made by workers earlier. Outsourcing, information technology, the fall of the Soviet Bloc and of Chinese Communism — all of these have accelerated this trend.
It is up to us to collectively push back. One has to be thoughtful, but one has to either take the offense here or perish. Nuanced defenses and alliances of convenience — be it with what remains of the old Democrats or with people such as Bill Gates — will no longer work by themselves.