When trying to estimate the tendency of all things (typically humans) from a small selection of them (maybe a few hundred at most) organized into groups for experimentation, you have to make a few assumptions. The big one is:
* Your groups are as identical as you can possibly make them
Obviously that’s really hard to nail down because you can’t possibly get 2+ identical groups. Even if you selected a hundred sets of twins and split them between the groups there are still differences between them. If you’re conducting a medical experiment and by sheer rotten luck everyone in group 1 was a smoker and everyone in group 2 doesn’t smoke…. well, your results are going to be inaccurate as a result. If you don’t catch that mistake, your results are not going to be trustworthy.
Another big assumption that might be wrong:
* Your small group(s) statistically represents the entire population
Obviously this can also be very wrong without you realizing it, in many of the same ways. Again, a group of 100% smokers (by rotten luck) doesn’t represent all humans.
So the 95% confidence says how sure you are the results are correct and that your results correctly measured whatever it is you’re measuring, vs the 5% chance that you got shitty group assignments and the results are invalid as a result.
Latest Answers