How do small percentages continue to work out in the long run?

Question

If something has a 5% chance of success, and I’ve failed 19 consecutive times, the next time independently still has a 5% chance of success, fairly low, rather than a guaranteed 100% chance of success in view of the previous attempts. How does this relate to updating percentages in light of new evidence, or is that something separate?

In: 6

Answer 1

> If something has a 5% chance of success … How does this relate to updating percentages

It doesn’t. If _p_ for a series of independent events truly is 5%, or you are sampling _with replacement_, it’s 5%. That doesn’t “update.”

If you are drawing without replacement, then _p_ would change with each draw; were it 1 winner and 19 losers in a hat, _p_ of the next draw winning would approach 100% till the winner is drawn.

> updating percentages

Let’s presume that _p_ is unknown. You could study the phenomenon and try to estimate _p_. If the event seems to be with replacement or independent each time, then your _most likely_ value of _p_ would be whatever wins-per-draws ratio you observed. Advanced statistics could give you a _confidence interval_ from that, which is another level of chance. For example, let’s say you observe 10,000 times and see exactly 442 successes. The most likely value of _p_ is 4.42%, but a _95% confidence interval_ could figure to be something like 4.00% to 4.84%. This says that if _p_ is a fixed value, there is about a 95% chance that this sample is the result of _p_ being in that range, and a 5% chance that _p_ is out of that range but had a strange run of luck.

If we continue to sample, the range will get narrower, but we still don’t _know_ what _p_ is or if our sample is becoming more or less representative of _p_. If we sample to 4,420 in 100,000, the most likely value of _p_ is still 4.42%, and then our range becomes 4.29% to 4.55%, but we’re still only 95% confident.

We _can’t know_ what _p_ is from sampling, but depending on what is most important to us—risk of being wrong, accuracy of our guessed range, or the cost of additional sampling—we usually can get to something satisfactory enough to estimate with, or to decide if our theory is supported or should be reconsidered.