What is the Bayes Theorem and why is it useful?
In: Mathematics
Bayes theorem basically says that the probability of something, without any evidence, should not be ignored.
Like, imagine this.
99% of all cars are red.
Eye witness testimony is 90% accurate.
A witness says the car was blue.
What color do you think the car was? It may seem like the car was blue since witnesses are pretty reliable, but you can’t ignore the first fact. Bayes theorem tells you the correct probability.
It’s about correlation, and reverse correlation.
Suppose you notice that 90% of people wearing hats are men. Is it therefore reasonable to assume that 90% of men wear hats?
Well of course not – that example seems obvious, but there are lots of examples that aren’t that clear.
Let’s say that 90% of cancer patients got a positive result in some blood test. If you take the test and get a positive result, does that mean there’s a 90% chance you have cancer?
Let’s say that 90% of terrorist acts are committed by people from a certain country. If you see someone from that country, is there a 90% chance they’re a terrorist?
You can’t just reverse the probability like that – it’s flat-out incorrect.
In maths terms, you write P(A|B) for the probability that A, given that B is true.
From my first example, we can say that P(is-man|wearing-hat) = 90%, or 0.9
Now we want to flip that around and get P(wearing-hat|is-man).
You can do this, but you need two more bits of information: the probability that *anybody, regardless of gender* is wearing a hat, and the probability that *anybody, regardless of hat-wearing*, is a man.
Let’s say that hat-wearers are 1% of the population, and men are 50% of the population.
Now we have enough information to work with.
* P(A) = P(is-man) = 0.5
* P(B) = P(wears-hat) = 0.01
* P(A|B) = P(is-man|wears-hat) = 0.9
Now, Bayes’ theorem says that P(B|A) = ( P(A|B) * P(B) ) / P(A)
* P(B|A) = (0.9 * 0.01) / (0.5) = 0.018
So there’s a 1.8% chance that someone will be wearing a hat if they’re a man. That’s one hell of a long way from the 90% that a naive approach might suggest.
Bayes’ theorem is useful in all *kinds* of fields, but I’ve found it’s super-good for yelling at racists with.
If you want to know *why* Bayes’ theorem works, take a look at https://oscarbonilla.com/2009/05/visualizing-bayes-theorem/
Bayes Theorem is a way to weight evidence by the strength of the evidence. Say you have a coin and you think it’s not a perfect 50/50 chance when you flip it. If you test it comes up heads twice, do you assume it has a 100% chance to land heads? No, but your data is saying it’s 100% heads. So you use Bayes Theorem to adjust the observed data by your prior assumption. Assume any random coin is 50/50 and adjusting your expectation as you collect more data from additional coin flips. The more data you collect the more confident you can be that your data represents a true deviation from an average coin.
It’s very useful for making statistical predictions because it makes a distinction between an Observed Probability and the true underlying probability of something happening by factoring in the likelihood that the observed data could have just occurred randomly.
Can I throw in the reverse question, and ask, what is an ELI5 of bayes theorem limitations?
Latest Answers