What is Survivor Bias?
In: 1017
Example: Old Buildings are much better made than new buildings. There is a beautiful 500 year old church in the middle of my town and the 70 year old house next to mine is a dump.
This is survivor bias, because you see none of the houses that were built when the Church was built. So, you see only the survivor, the church, and so it’s “typical” of buildings of the 1500s. If you had seen all the other buildings from the era fade you’d appreciate that the Church was much, much better built than typical buildings of the era, a more unbiased assessment.
Its the notion that basing statistics on only the observable results from samples that passed some screening or “success” criteria will lead to skewed results and incorrect interpretations.
For example, a study that examines corporate performance of companies who have been in business longer than 10 years – but only examining those companies who are still in business. You should also examine the companies who _were_ in business longer than 10 years but currently are NOT in business. If they were so successful why are they not _still_ in business?
Or, the classic example – in determining where to put extra armor on airplanes, designers looked at where the holes from enemy shells were on the planes that came back to base. But obviously, the ones that made it back to base with bullet holes clearly survived – therefore the bullet hole locations in THOSE planes are not where they needed additional protection – the survivors survived anyway. Its where there were NO bullet holes in the survivors, that’s where the additional protection was needed – clearly planes that got hit in those locations are the ones that didn’t make it.
In most situation, we are observing what survived. If we only look at those that survived, we are ignoring what didn’t survive and this can lead to false conclusion.
Someone already presented an example about old building. [Here](https://www.youtube.com/watch?v=P9WFpVsRtQg) an example about planes in WW2. At first people were looking at surviving aircraft and where on them we found bullet holes. Obviously those are the places where planes get shot at and we should reinforce those places to improve the survival of planes.
But that’s is a false conclusion. In reality, those planes were able to survived because they were not shot in critical area. Instead we should reinforce where no bullet holes are found in survivors. Because the planes that were shot there, didn’t survived.
The people originally based their conclusion on the survivors only and this was a mistake.
I just had a kid so my example…
Experts recommend putting baby on their back to sleep. No blankets, no pillows, no stuffed animals, etc.
My mom and grandparents tell me “we put you to sleep on your stomach with blankets and you survived”.
Boom. Survivors bias. What about all the babies that did not survive.
any time you see an awards show and an actor/musician wins an award and says “you just need to follow your dreams and never give up!” Millions of young people see this and think “That could be me!” But they’re not hearing from the hundreds of thousands of people that pursued a career as an actor or musician and crapped out. they’re hearing from the ones who made it. the one’s who ‘survived’
Your grandma still has her avocado green 1970’s fridge, but you had to replace the 10 year old fridge in your kitchen… so you think, “man, they don’t build appliances like they used to” even though 99% of the avocado green fridges are long gone to the landfill by now.
The bias is the bias towards considering survivors *and only* the survivors. You are ignoring the losses.
100 planes go out.
10 planes come back.
“These 10 surviving planes sure have a lot of bullet holes in the tail. We had better increase the armour in the tail area.”
But if you examine the planes that were lost, what will you find? Perhaps that they were all shot through the cockpit. The planes don’t need more tail armour, they need more cockpit armour.
This bias typically happens when something is selected in some way and you only look at one group of the two. Practical examples:
20% of planes return from a fight with the enemy -> you only look at those 20% instead thinking about the 80% who did not make it back (the classical example used with the red dots for bullet damage and asking what to improve)
27% of women work in STEM -> you only ask the ones who already work in STEM but not your accountant why she does not
Another good one is from Wikipedia: 6/10 students on an elite class are from one school. You think the school is really good, but it could just be way larger than the schools where the other students are from.
You will see this happening everywhere all the time once you understand the bias.
“it never did me any harm” ignores all the people it did harm, basically. People who didn’t live to tell the tale aren’t represented.
As everyone else is saying, it’s when you reach a conclusion based on data without considering what factors may be filtering your results. If you’d gone to a restaurant last year and asked patrons how worried they were about COVID-19, your results would be skewed by survivorship bias – the people who were most concerned about COVID-19 would not have been at a restaurant.
Latest Answers