You have an idea that for whatever reason men take longer to do some cognitive task than women do. You run the experiment on 37 people and find that on average men take 1.2 seconds longer than women do.
Null hypothesis testing is asking “IF there were really no difference between how long it takes men and women to do this task, how hard would it be to get a sample of 37 with a difference of 1.2 seconds or longer?” and you end up with a p-value.
Yes, it’s a weird and backwards way of thinking about it. We do it this way because null hypothesis testing converts a difficult Bayesian problem (How likely is it these results reflect a true difference and not just statistical noise?) into a much simpler sampling problem with dramatically easier math. Computers nowadays are fast enough to just do the Bayesian problem if you’d rather, but the process is still complicated. Just in different ways.
Latest Answers