It goes “When a measure becomes a target, it ceases to be a good measure.” How does that work in practice?
In: Other
Most of the top responses here only get this half right. As many others have said, the problem is that people alter their behavior in response to quantified targets, which can produce unintended consequences. One of the most important consequences is that, because they are now targets, the metrics that those targets are based on *no longer actually measure what we think they measure.*
A classic example (which fans of The Wire will be familiar with) is the use of test scores to evaluate teacher and student performance. Let’s say you want to improve schools by rewarding good teachers and firing bad ones. How do you identify good teachers? One way is by looking at their students’ standardized test scores. Presumably, students of good teachers will on average perform better on standardized tests than students of bad teachers. At the outset, student test scores are a plausible measure of teacher quality.*
What happens when you tell teachers and schools that they will be rewarded or fired based on student test scores? They will do whatever they can to improve those test scores. In many cases, they will start ‘teaching to the test’ – that is, sacrificing other goals in order to produce high test scores, rather than being good teachers. If this becomes widespread, students’ test scores will no longer measure overall quality of teaching, they will measure teachers’ single-minded focus on teaching to the test. *The metric no longer measures what we think it measures*.
* Many people would debate this premise, but for the sake of the example I am stipulating that it is at least plausible.
Latest Answers