What is Survivor Bias?
In: 1017
Say you judge three popsicle stick bridges at a science fair. However, only one is of them in tact. You give that one the prize for best bridge. And you conclude that this kid has what it takes to get it done. He gets an engineering scholarship for making the best bridge.
What you don’t take into account is the first one was built by a math and engineering whiz who built something meant to withstand an enormous amount of weight, but on the way to school the bully took a hammer to it. And the second one caught fire because the student next to her had a display about a magnifying glass and the bridge caught fire under the concentrated sunlight beam.
By only looking at the results of what survived, you miss out on important data points from those that didn’t, which can ultimately give you erroneous results. In this example, the winner might not have been the best bridge, but is awarded prizes for appearing to be such.
We can sometimes think of things like this when making our own conclusions in life. We listen to the stories of people who succeeded and try to live by their examples. And that’s not wrong. But it’s not everything. Sometimes they’ll say things like, “work real hard. It’s about being good. You have to want it more than anyone.” And that’s great advice, but it’s not true for every example. The Kardashians/Jenners are the most popular people on Instagram and have a billion dollar empire. And while they work hard, they were able to build their businesses by having reliable financial investments, access to the wealthy, status, good looks, media before the time of social media, publicity, lawyers, a well-connected and famous family, etc. They could afford to fail in multiple large ventures in life and learn from them, plus afford trustworthy consultation, while most of us could only maybe do that once or even twice if we’re lucky. Working hard is only part of their equation to the level of success they achieved. Actors know this well. The most talented people I know will never be famous, but the greats may tell you that you have to just be a great actor. It’s not just being a good actor. It’s being a good actor, having the right connections, right timing, right financial abilities/time to pursue the career, looks, branding, marketing, clout, charisma, timing, visibility, desired skillset, and an insane amount of luck. Getting a job is one thing, but being able to get enough big things ahead of a very good talent pool to build a career is a whole other thing. Like, you can see all of the amazing and unbelievable talent on America’s Got Talent, but a week later you’ll forget about them and that standing ovation appearance doesn’t mean a whole career is inevitable. Doesn’t mean the advice isn’t good, but it can also be incomplete and you may have to tailor your strategies to your personal situation. So if you’re say looking up to a musician to be like them some day, “practice every day and get good” is great advice. But it’s not the entire picture and it won’t exactly lead you to the result you want.
We can look look at this with older people, too. Some people will be like, “I lived to be a hundred drinking milk everyday. I smoked until I was 80.” Things like that. And you might be like, “smoking doesn’t kill everyone. Milk is good for me if I drink it every day. I know what I’m doing.” On the whole, no. Not really. Smoking is extremely unhealthy. Milk isn’t very healthy past a certain age because it’s generally not really balanced in a diet for most people. The reason some people live long lives is sometimes good genes. Some people’s bodies can withstand things that are generally bad for the population. One person could get cancer after a few years of smoking. Many have long term health issues because they smoke. Risk of cancers, emphasyma, lung issues, throat damage, mouth cancers, dental damage, risk to children when smoking while pregnant, etc. But if you go by the one person who didn’t get wrecked by it and live by their example, you miss the fact that a good chunk of people will get wrecked by it and likely you are someone who will be in the middle result and deal with the bad things that come from it.
Plane comes back with bullet holes near wings.
People think hey we need to strengthen the wings.
No.
The planes that came back with bullet holes in wings, CAME BACK.
The ones that didn’t come back were hit in the cockpit.
Strengthen the cockpit.
Survivor bias.
Survivorship bias is when you say:
“Hey I’ve noticed that a decent number of people describe having encountered grizzly bears, but almost nobody ever describes having been attacked by one…… clearly grizzly bears are really peaceful and safe to be around!”
The survivorship effect in this case is that, while grizzlies don’t always attack, when they do it is so vicious and overpowering that the victim does not survive, and hence cannot recount the experience.
You could put it this way: the fallacy is in assuming that the lack of data on *failed* attacks represents any kind of useful insight into attacks in general.
There’s a famous story from WW2. They were losing planes and saw that planes were coming back with lots of bullet holes in the wings. They added extra armor to the wings, but were losing about the same number of planes. The armor wasn’t making a difference. So they started adding extra armor to the cockpit and engine and they started having more planes return. This ]]kpkis due to survivorship bias.
The planes that were making it back, were only shot in places that a plane can get shot and make it back. So if we add armor to where those returning planes weren’t shot, the future plane would have more armor in the vital places.
If you only look at subjects that survived all the potential sources of failure, you have no idea how you could fail.
“Look at these amazing old buildings. They used to be constructed so much better back then!”
False conclusion. The majority of older building are gone because they are inferior. The few examples that survived are the very few buildings that are exceptionally well-built.
If we only use those buildings as a point of comparison, we are drawing a bad conclusion based on a sample set that doesn’t fully represent all older buildings.
A group of five people are traveling.
A: “My opinion is that this trip is excellent! I’m having a lot of fun!”
B: “My opinion is that this trip is excellent! I’m having a lot of fun!”
C: “My life is in danger. I need to get out of here immediately.”
D: “A&B are crazy and they keep creating dangerous situations.”
E: “I have no opinion about this trip. I’m here to get paid.”
**Due to the incredibly irresponsible actions of A&B, C, D, and E die.** If you do a poll in order to find out the overall opinions about the trip, the only people available are A&B, therefore, the trip they took seems like it was wonderful.
When you hear war stories that say things like “I’m the man I am now today because of my time in the military! Going to war helped separate the men from the boys and showed me what I was capable of!”, you don’t get to hear the opinions about war from the people who bled out.
You get the opinions of people who were seriously wounded and the people who were not seriously wounded, and the number of soldiers who get wounded in the service is much smaller than the number of non-wounded individuals.
It’s actually a huge problem, because the people who survived tend to end up becoming leaders in the military, and they’re biased towards combat.
This contributed to how terrible World War I was. It was the first industrial war, and the people fighting it were completely unprepared to experience a meat grinder because their leaders thought of war as a romantic adventure, where a couple of people died, there were some men maimed but survived, an army was routed, and it was all over in a season.
My favourite example is popular music.
“Music today is almost all garbage. Music from a few decades ago was so much better!”
Common refrain, but it’s just another example of survivorship bias. Most of the music back then was garbage as well. We just haven’t bothered to keep listening to it. “Sugar Sugar” by the Archies was one of the top songs of its time, but no one points at it today as an example of great music.
When a successful actor says, “Follow your dream”, you don’t hear about the millions of aspiring actors who followed *their* dream, and wasted years of their life trying and failing to get their “big break”, while working menial jobs in Hollywood for little money, only to eventually give up and move on.
That’s an example of survivor bias: the condition where we only see evidence of a subset of the total data available, because the rest of the data is filtered by a selection process of some kind.
Incident: Sailors tell stories about how dolphins helped push them towards shore, saving their lives.
False conclusion: Dolphins like to help humans
Survivorship bias: Only the people pushed towards shore were able to tell their stories of being saved by dolphins.
Probable reality: Dolphins like to play with humans in the water, pushing them around. Some get pushed to shore, some get pushed further out to sea to drown.
Ok, so let’s pretend that you’re a WWII aeroplane designer. Your job is to make sure planes return with their pilots in tact. This means you gotta find out where to put armour.
In an ideal world, you could armour the whole plane. Problem is, both fuel & metal are expensive, so you only wanna put armour where it’s needed.
So, you make a rule. “Whenever a plane gets back to base, a group of people will check it top to bottom for bullet holes, and record where they are.
You wait about 5 months, then look at the data. Most bullet holes are found along the body of the planes. The only places where bullets are absent, are the cockpit, the engines, the rudders, and thin parts of the tail.
Where do you put armour?
A novice would recommend that you put armour where the bullet holes are, because that’s obviously where planes are getting shot. But this is flawed.
Think about it: What happens when a plane gets shot in the engine? It crashes. What if it’s shot in the cockpit? Well, the pilot dies, so it crashes. Rudders? Plane can’t keep steady, so it crashes.
Whenever a plan is shot in those places, that plane never gets back to base. That means your men never find a plane that’s damaged there, and you never hear about it.
That is the survivorship bias in play.
Sometimes, the way you gather information will skew the results you get. You have to account for this in order to give an accurate assessment..
Latest Answers