Bayesian probability

In: 2

Bayesian statistics is a theory in probability that expresses a degree of belief in an event. The degree of belief can be affected by various factors, including prior knowledge about the event, such as previous experimental results, or on personal beliefs.

Bayesian probability is a likelihood of an event A happening given event B happens where event B is an independent event. (Throwing 6 on dice given it is thursday is an example of independent. Getting wet given it is raining would be dependent.)

Since it is very often used for this purposes, think about a test (cancer test for example)

You get a positive cancer test (event B). What is the probability, given positive test, you have cancer (event A)? This question can be expressed as probability of A being true given event B or P(A|B)

We know that approximately 2% of the people have cancer that is a general probability of event A happening or P(A) = 0.02

We also know that if you have cancer the likelihood of getting positive test is about 95% this is expressed as a probability of event B happening given A being true P(B|A) = 0.95.

The last number we sould use is P(B) or a general likelyhood of geting a positive cancer test. Both correct and incorrect positive test and out historical data claim 10% are positive Or P(B)=0.1

We can now answer the original question

What is the probability, given positive test, you have cancer?

The bayesian theorem supplies that P(A|B) = P(B|A) * P(A) / P(B)

And it will come down to about 20%

It is a very elegant way to show, that just because something is 95% accurate, it doesn’t mean it is necessarily very likely.

It may seem a very contrary to intuition so let me add some numbers to show where this discrepancy comes from.

You are one of 100 000 people. Only 2000 P(A) are expected to have cancer.

You test all of them

Given the likelihood of 10% for positive test P(B) = 0.1, 10 000 people are marked positive.

Out of 2000 actually ill, 100 are incorrectly misidentified as healthy (due to P(B|A)= 0.95) and 1900 are correctly identified as ill.

That makes 8100 people who are diagnosed positive despite being healthy.

You can find a mostly ELI5 explanation in the opening paragraph of the wiki article here: https://en.wikipedia.org/wiki/Bayesian_probability?wprov=sfla1

“Bayesian probability is an interpretation of the concept of probability, in which, instead of frequency or propensity of some phenomenon, probability is interpreted as reasonable expectation representing a state of knowledge or as quantification of a personal belief.”

Basically, the reason people use the term Bayesian is because schoolbooks teach probability as stuff associated random events, or multiple experiments. That’s a simple explanation, but results in a bunch of contradictions or confusing questions. Instead it’s more robust to think of probability as a quantification of one’s state of (usually incomplete) knowledge.

To take an example which I found helpful: let’s say I have a coin, which I guarantee to you, is perfectly biased (i.e. it will always land on the same side when flipped). However I don’t tell you if it’s biased to land on heads, or on tails (i know it, but i don’t tell you). Now what’s the probability it will show heads when flipped?

If you think about it, it’s still 0.5.

The coin throw, or the coin itself isn’t random. What’s incomplete is your information about the coin.

The term Bayesian comes from the mathematician Bayes, who is also responsible for Baye’s theorem. Bayes theorem is a very important theorem in probability, and life in general, given the increasing use of questionable statistics to promote misinformation in modern media.