# stat sig or statistical significance

314 views

I’m hearing this phrase a lot in my meetings and from what I gather, it’s basically the number an analysis needs in order to reach a calculated conclusion. Basically more data = closer to stat sig?

In: Mathematics

If I roll a dice 1 time and it comes up showing a 6, can we say it is an unfair weighted die that shows 6 more than statistically reasonable? No we can not, we don’t have enough information. So we roll it 10 more times, and it comes up showing 6 twice more, out of those 10. Can we say it is an unfair weighted die now? Not really, even though that’s still showing 6 slightly more often than we would expect. There’s simply not enough rolls to feel strongly about the results. Getting 6 3 times in 11 rolls is a lot, but not super improbable. It’s not statistically significant enough.

If we roll it 1,000 times and it rolls a 6 300 times, now can we say it’s weighted? Yes we probably can. That’s enough rolls where it should have distributed more evenly than that. It’s a significant statistical anomaly.

So, there will be a particular deviation level (usually 2 standard deviations, or 95% of probable distribution) where we would consider it to be significant and unlikely to happen by chance except in the most outlandish of instances, to the point where we feel comfortable in the resulting conclusions.

I’m not sure if this is exactly what it means for your situation but with psychology research statistical significance means if the difference between the numbers actually means something or if its just a difference due to chance. So most papers will use “p<0.05” which means the results showing a difference are less than 5% likely to be due to chance. If the p value is over 0.1 then there’s more than 10% likelihood it’s random and you can’t really say something if meaningful or effective if its just chance.