As said above, I am trying to understand what statistical significance and confidence interval means. I am a psychology major whose reading a chapter for my Research Methods in Psychology class. The books definition is not helpful. Investopedia’s definition is better, but not enough.
So far, to my understanding, statistical significance refers to a claim that set of data that are not the result of pure chance, but instead are the result of a specific cause. What is a great example of statistical significance?
As for confidence interval (CI), It seems to be a probability that a parameter will fall between a set of values. In my case, this relates to the correlation coefficient (r). What is a great example and explanation of CI?
In: Mathematics
When you are doing an experiment, you get the control results and the treatment results (where you changed something). You look at the results and see if what your testing is an “improvement” over the control, but then you have to ask yourself, what if it was just luck? What if the numbers just happened to land in a lucky way that makes my result look good? Was it a fluke?
Well, what you can do is see what randomness looks like and compare your result to that. We can take all the results we got, throw them in a hat and shuffle them around, and then randomly re-distribute them out and see what the result looks like. That’s one example of what randomness can produce. Then we can do it again and look at that result. And again. 1,000 times again. Now we have 1,000 results of what random looks like.
Now we can look at our original result again and see where it falls in the distribution of randomness. **How extreme is our result? How much of an outlier is it? Is it so extreme that it would be really unlikely for randomness to produce it?** Is it at least more extreme than 95% of the random results? If so, then we can call it **statistically significant,** because it has met the 0.05 (5%) burden that we set for ourselves.
Latest Answers