I did a biology undergraduate degree and often did reports where would statistically analyse our results. P value of less than 0.05 shows that the results are statistically significant. How do these tests actually know the data is significant? For example we might look at correlation and get a significant positive correlation between two variables. Given that variables can be literally anything in question, how does doing a few statistical calculations determine it is significant? I always thought there must be more nuance as the actual variables can be so many different things. It might show me a significant relationship for two sociological variables and also for two mathematical, when those variables are so different?
In: Mathematics
> For example we might look at correlation and get a significant positive correlation between two variables. Given that variables can be literally anything in question, how does doing a few statistical calculations determine it is significant?
You’re doing it wrong.
You start with a null hypothesis. This is the thing you want to show is false.
For example, you think that people who eat more apples also eat more pears. So your null hypothesis is that people eat the same number of pears regardless of how many apples they eat (no correlation). Then you go get data and test it.
But people also eat plums, and that might affect whether they eat pears! So you include plum eating in the formula.
If you find a correlation between eating pears and eating plums with a P value of less than 0.05, is that statistically significant?
No, it is not. Why? Because that is only your hypothesis BECAUSE you found a correlation, which biases your results. You might have had 20 different fruit you were correcting for, which would mean odds are at least one would have a correlation – even if there was no correlation at all.
Doing random P testing on things you don’t think to have correlation is simply wrong. You might well do that at a preliminary stage to find out what hypothesis to test in the first place, but the data you use to come up with the hypothesis cannot be the same data you use to test it.
To see why, imagine a man who sees a coin tossed 4 times. Each time it comes up with heads. He thinks, maybe the coin is biased. He then tests that by using his observations of the coin coming up heads 4 times, and, low and behold, the data backs it up – this coin is not a fair coin! That’s crazy, right?
> Given that variables can be literally anything in question, how does doing a few statistical calculations determine it is significant?
Basically, P values tell you, “what are the odds we’d get this if our null hypothesis was true?”. If it’s 0.05, that suggests but does not prove that your null hypothesis is false. Go do more testing.
Latest Answers