What is Bayes’s Theorem?
In: 48
Baye’s Theorem provides a simple model for reasoning about conditional probability using algebra. Conditional probability is a way to incorporate prior knowledge into a measure of probability.
Conditional Probability ===
Say 40% of a population has an illness. I can estimate that I have a 40% chance of having that illness as a member of that population.
Say 66% of elderly folks in that population have that illness. I can more accurately estimate that I have a 66% chance of having that illness *given* that I am elderly.
Young & healthy – .5
Young & ill – .2
Old & healthy – .1
Old & ill – .2
Sum = 100%
edit:
Back to Baye’s ===
To tie this back to Baye’s without getting into math, Baye’s formula allows you to translate between the “elderly *given* ill” and “ill *given* elderly” scenarios by plugging and chugging into a nifty lil formula.
Thanks toferino 🙂
Bayes’ Theorem is P(A|B) = P(B|A)P(A) / P(B).
What it’s talking about is the scenario where you get up in the morning thinking that A (whatever that might be) has a certain chance of being true, and then you get some new information, called B. The theorem tells you *how much* you should adjust your belief, now that you’ve heard B.
For instance, you might wake up with a 99.997% confidence that your girlfriend is faithful (her being faithful is A), and then B is the new information: Lying Joe tells you that she’s cheating on you. Should you instantly start believing that your girlfriend’s cheating?
In general terms, if you think Lying Joe is completely full of shit, your opinion of your girlfriend shouldn’t change one iota. But if you know he tells the truth occasionally, your 99.997% might turn into a 98.7% or something, and if Lying Joe is the most reliable person in the universe (?!?) then your confidence should fall substantially. The formula lets you decide that, based on P(B|A) and P(B) which are indicators of Lying Joe’s reliability.
Bayesian reasoning is good because when you get new information, it makes you slow down and think! It asks you, basically, how confident were you in your belief before this (they call that your ‘prior’), how believable is this new claim, and how “well connected” is this new claim to reality.
One of the standard examples is, imagine the doctor’s office calls a week after you get some blood tests, and tells you that you have kuru, an ultra-rare disease associated with New Guinea that you usually get from eating brains. (Imagine for this example that you live very far from New Guinea, have never been there, and have never knowingly eaten brains.) Some work with Bayes’ Theorem can tell you that “mistake in the lab” or “office called the wrong person” are both more likely than “you suddenly got this ultra-improbable disease”.
I’ll give you an example where Bayes theorem would come into play:
Say there’s a disease that affects 1% of the population. There’s a test for this disease. It always picks up positive cases, but it also sometimes gives a false positive. If you run the test on a person who doesn’t have the disease, there’s a 10% chance that the test will come back positive.
The disease is very serious, so doctors run the test on everyone to make sure they don’t miss any cases.
Your test comes back positive. What are the chances that you actually have the disease?
You might be tempted to assume that there’s a 90% chance you have the disease—after all, there’s only a 10% chance of a false positive.
Fortunately, this is wrong.
If you ran the test on 100 people, you would expect to see one true positive result because of the 1% of the population with the disease but you would expect to see about 10 false positive results from the 10% false positive rate. That’s 11 total tests that come back positive and 10/11 are false positives—91% of positive tests are false positives.
Therefore, if your test comes back positive, there’s about a 91% chance that you don’t have the disease and only a 9% chance that you do have it.
Bayes theorem is the mathematical equation that lets you figure out this kind of probability problem.
Let’s call having the disease A and getting a positive test result B.
Bayes theorem says that the probability of A given B (the probability that you actually have the disease, given a positive test result) is equal to the probability of A (1%) times the probability of B given A (the chances of someone with the disease getting a positive test result, 100%) divided by the probability of B (~11% because it’s the 1% true positive plus the ~10% false positives).
P(A|B)=P(A)*P(B|A)/P(B)
Without getting too deep into the maths, Bayes’ theorem basically tells us how to update our beliefs based on the new evidence we see. When you dive into Bayes’ theorem, you’re going to come across this well-known equation: p(A|B) = p(A)*p(B|A)/p(B), but I prefer to write it like this: p(H|E) = p(H)*p(E|H)/p(E). Here, H stands for “hypothesis” and E is for “evidence”. Let’s break it down.
The p(H|E) part is what we call the posterior probability. It’s all about our belief **after** we take a look at the evidence (that’s why the “|E” part is there). This is the number we’re trying to figure out.
Then, there’s p(H), which is the prior probability. This one tells us about our belief **before** we’ve seen any evidence.
So, we start with the prior and we want to get to the posterior, right? To do this, we need a kind of “adjuster”, and that’s where p(E|H)/p(E) comes in. This is the likelihood ratio that helps us modify our belief. Basically, it’s about how likely we are to see this evidence if we assume the hypothesis is totally true (which, to be clear, it can never be 100%, but we assume it for the sake of calculation). Say the evidence would occur just as often whether the hypothesis is true or not, meaning p(E|H) = p(E). In this case, the likelihood ratio would be 1, so your prior belief doesn’t change and your posterior probability stays the same. But if the evidence is twice as likely to show up when the hypothesis is true, meaning p(E|H)/p(E) = 2, then you double your prior belief.
Here’s where it gets a bit tricky though: doubling your prior doesn’t always lead to a major shift in the posterior. Like, if your prior belief was 1%, even with a likelihood ratio of 2, your posterior only bumps up to 2%. So, it’s not always as significant as it might seem.
Let’s look at a practical example: You’re a doctor and you’re testing for a blood-borne disease. From past experience, you know that 2 out of every 100 people have the disease in your population. All positive are accurately detected. However, the test also returns a positive result for 5 out of 100 people who don’t actually have the disease. Now, let’s say a random person tests positive. What’s the probability that this person actually has the disease?
Sure, you could just crunch the numbers and get the answer, but let’s try to understand the intuition behind it. You start with your prior belief before the test, which is 2%. This means, without any additional information, there’s a 2% chance that this person has the disease.
Then, you run the test and it comes back positive. Given what you know about the test, you can expect about 7% of tests to come back positive anyway (2% true positives and 5% false positives).
But what you’re really interested in is how much you should adjust your initial belief based on this new piece of evidence. So, you calculate the likelihood ratio. Assuming the person really does have the disease, then they’re 100% going to test positive. This is 14.29 times more likely than getting a positive test result by chance, because p(E|H)/p(E) = 1/.07 = 14.29.
But we’re not done yet. Even though a positive test is 14.29 times more likely for a person with the disease, it doesn’t mean they’re guaranteed to have it. You’ve got to remember that this disease is pretty rare to start with. So, even with a positive test result, the posterior probability only jumps to 2%*14.29 = 28.6%. It’s a lot higher than 2%, but still less than a coin flip.
In practice, we’d likely run more tests, and each new posterior probability would become the prior probability for the next test.
I hope this explanation and example show that Bayes’ theorem is just all about adjusting our beliefs in light of new evidence.
Nate Silver has a book on this – The Signal and the Noise. He uses presidential election polls in the US as an example.
Is the most accurate indicator the quinnipiac poll , or the Washington Post, or a different source?
Bayesian theory says the most accurate guess will be an aggregate all of them, and to weigh each one proportionally to to how accurate those polls are. (It provides specific math).
Using this method, Silver was famously able to accurately predict not just the 2008 election, but he was actually able to predict the outcome of 49 of the 50 states correctly.
Most importantly – when Bayes says the prediction is 80% certain – it really does predict the correct outcome 80% of the time. This is useful for all sorts of predictions – elections, weather, sports, stocks, etc…
According to google, 87% of billionaires are male.
Gosh! That means 87% of males must be billionaires, right?
Wrong.
Bayes’ theorem tells you precisely *how* wrong.
The probability of being male *given that* you’re a billionaire is 87%. You can write that as P(male|billionaire) = 0.87
What you want to know is the reverse of that: P(billionaire|male), the probablility you’re a billionaire *given that* you’re male.
And believe me, that number is not 0.87, or anywhere close to it.
But if we have two more bits of information, we can work out what that number is.
We need to know P(male) and P(billionaire), independent of each other. What percentage of all people are male, and what percentage of all people are billionaires.
Off to google:
* Total number of humans = 8,036,677,056
* Total number of billionaires = 3,311
* Percentage of humans that are male = 50.42%
* Percentage of billionaires that are male = 87%
Bayes’ theorem says that
P(A|B) = P(A) * P(B|A)
—————
P(B)
Translating that into our terms, that’s
P(billionaire | male) = P(male | billionaire) * P(billionaire)
—————————————-
P(male)
P(male | billionaire) = 87% / 100 = 0.87
P(billionaire) = 3,311 / 8,036,677,056 ≈ 0.000000412
P(male) = 50.42% / 100 ≈ 0.5042
Now we can plug all those numbers in:
P(billionaire | male) = (0.87 * 0.000000412) / 0.5042 ≈ 0.000000711
The chance of being a billionaire if you’re male is thus 0.0000711%.
That’s one helluva long way from 90% – and an awful lot of racism and bigotry is based on not understanding the difference.
If you want to know *why* Bayes’ theorem works, someone’s explained it a helluva lot better than I can, using Venn diagrams.
[Take a look](https://oscarbonilla.com/2009/05/visualizing-bayes-theorem/)
Latest Answers