It goes “When a measure becomes a target, it ceases to be a good measure.” How does that work in practice?
In: Other
Nail factory rewards employee for number of nails made = employee makes a lot of uselessly small nails
Nail factory rewards employee for total weight of nails made = employee makes very heavy, uselessly large nails.
“Measure becomes target” basically means “gaming the system”
I can give you a good example: Food labeling, specifically protein.
It’s actually pretty difficult to measure components in food. The standard method for measuring protein is to basically blast a sample into atoms and count how much nitrogen is there, because most of the nitrogen present in any living being is in the protein.
Like I said, it’s the standard method. If you’re selling something that’s supposed to have a certain amount of protein in it, you want to hit that target. The problem is that the target is no longer really the amount of protein. It’s the amount of nitrogen. There are much cheaper, nitrogen-rich molecules you could add to the food that will “pump the numbers” and show up an protein in the tests. That’s why Chinese manufacturers started adding melamine to pet food and baby formula.
It was kind of an open secret for years that this was going on. If that sounds bad to you, it gets worse. Melamine is relatively harmless, so at least they weren’t hurting anyone (apart from the risk of protein deficiency). Then some manufacturers realized they could save even more money if they changed from the industrial melamine to using the batches that had been rejected for industrial use due to contamination. It was the contaminants that started killing cats, dogs and babies.
You need to regulate and control things, but you have to be careful about the metrics you use as the target, because if there’s a way that someone could game the system, maliciously comply, or overdue compliance on that one metric to an extreme degree, they probably will.
The modern US educational system is this in perfect practice right now. They came up with standardized tests to determine how well the students were doing. This was ok, but then they added funding to the results. The better the results, the better the funding. So schools started teaching to the test. Now the only thing the students are really learning is how to navigate a particular test. This can really be seen in practice in how most schools more or less shut down as soon as the test is over near the end of the semester. It’s basically two weeks of glorified baby sitting.
A real life example: I work in healthcare. “Central lines” are a type of IV that goes in a “central vein,” frequently the jugular vein in the neck. Germs can get on these devices and cause bad infections. Because they’re bad and largely preventable with good hand hygiene and aseptic technique, rates of central line associated infections are nationally reported and tracked metrics for hospitals. Medicare, Medicaid, and most (probably all) private insurers will not reimburse hospitals for *any* of the costs of a patient’s care if they develop a central line associated infection.
The way to diagnose a central line associated infection is to remove the culprit line and send it to the lab, where they cut the tip off of it and see what, if any, germs grow from it in culture. I have personally removed central lines from patients, handed them to the nurse assisting me, and said “please send this to the lab for culture.” They throw it in the trash and tell me they can’t do that or they’ll be fired.
There’s literally an institution-wide policy in every hospital I’ve ever worked in that says you are not allowed to culture central line tips.You are banned, under threat of firing, from making that diagnosis because having a central line associated infection rate higher than zero affects their bottom line. Therefore every hospital “officially” has no central line associated infections. By making a target (zero infections), they made the metric worthless.
Ever heard about the bounty on snake heads in india during the britsh colonial period?
The brits didn’t like snakes (still butthurt after that adam and eve thingy) while indians didn’t care much about them (naga are even positive figures)
Smart brits put a bounty on each snake head brought to them to reduce the number of snakes on thier land
Smarter indians started breeding snakes in their home, chopping their heads off. The number of snakes in india increased significantly, while the indians started profiting
The brits had to stop the bounty thing because it was actually hindering their intended plan
Feel free to get to the moral of the story on your own
In a lot of business situations there was this rush to come up with KPIs (Key Performance Indicators). Managers would realize they need to determine if their teams are being efficient or not and where things need to improve.
They would create scorecards and leaderboards based on these great KPIs they came up with.
The thing they missed, which is the core of Godhart’s law is that if you create a very specific metric that you are going to define as “success” then people will naturally find ways to make that metric look better, not caring about the original intent behind the metric.
If they can manipulate the process to make that metric look better they will. This immediately makes the metric turn into a negative because the manager was probably trying to solve an overall issue, but by creating this metric the people doing work are no longer working on the overall process, they want to make the metric move and be considered more successful.
Companies that try to use KPIs exclusively will generally find they end up underperforming overall because it is difficult to come up with a set of KPIs that work together to be measurable and produce the desired overall result.
This was a big problem in the 80s through 2000s (and still exists in different companies). Some refer to it as attempting to manage “by spreadsheet” or other negative views of it.
Imagine that you want someone to learn how to do math. To see if they are learning, you give them a test. The test decides if they can go to the next grade, so it is super important.
Now, I could teach someone math OR I could teach them how to take this math test. That second situation is what we’re talking about here. My test no longer checks if you learned math, it checks if you learned how to take the test.
The most familiar outcome of this is when people talk about “teaching to the test”
The best example was years ago, in software. Some company decided to give bonuses based on the number of bugs fixed.
So what happened? Engineers started to write bug filled, awful code.
They were measured by the number of bugs they fixed, so they just created more bugs to fix.
Essentially, you’re changing the motivation from “do a good job at X” to “do a good job at this very specific number we are using to try to see if you are good at X.”
Current situation at a certain aircraft manufacturer is a great example, their goal the last couple decades has been share price and nothing else. So they took their previously high emphasis and budgets in Quality and compliance and used these funds for stock buybacks. Great for stockholders, shitty for airplanes. And in the end when planes start falling apart mid flight, stock prices end up falling even though the whole goal was to get these up.
Often people want one thing but can’t measure if they are making progress so they measure something similar. This will lead to other people gaming the system to seem like progress is being made even if it isn’t.
There is a story from the historical time period when Brittain ruled India. In India, there are snakes and the Brits were having none of that. They wanted to get rid of the snakes, but you can’t really measure how many snakes exist in India. As a close enough thing, the British decided to reward anyone who brought them dead snakes, thinking that the Indian natives would kill the snakes for these rewards and initially a few probably did. After a while however, the Indians started growing their own snakes to kill, so they could then butcher them and sell them to the British. This went on for a while, but for obvious reasons, the amount of snakes wasn’t going anywhere. After the British caught wind of what was going on, they did what felt logical can discontinued the bounty program, which simply made the Indians release their snakes to the wild, increasing the amount of snakes.
Latest Answers