P-Value and hypothesis testing theory
In: 2
When comparing 2 different groups of people together – that is, the one getting the real medicine vs getting the placebo – you can never really get identical groups to start with. Even if both groups got placebos, your measurements will still show one group doing better than the other. If your testing medicine was ineffective, it might as well be a placebo and this still applies.
So we use math to try and quantify the probability of it being a coincidence. It’s not enough to say “Group A did 5% better than group B”. If the spread of scores within each group is 30% wide, then a variation of 5% isn’t very convincing. On the other hand if the spread within each group is only 2% wide, then 5% sounds a lot better.
The P value is the estimated probability that you have a 2-placebo scenario and the difference is just luck, calculated from the concepts mentioned above, though much more mathematically rigorous. If the number is very small – preferably less than 0.1 (10%) but ideally even less than 0.01 (1%) – then you may conclude that the medicine was actually effective. If your P value is much larger like 0.3 (30%) then you’re not really convincing anyone the medicine was the deciding factor.
Latest Answers