Trying to understand, in statistics (related to psychology), what does statistical significance and confidence interval mean exactly?

Question

As said above, I am trying to understand what statistical significance and confidence interval means. I am a psychology major whose reading a chapter for my Research Methods in Psychology class. The books definition is not helpful. Investopedia’s definition is better, but not enough.

So far, to my understanding, statistical significance refers to a claim that set of data that are not the result of pure chance, but instead are the result of a specific cause. What is a great example of statistical significance?

As for confidence interval (CI), It seems to be a probability that a parameter will fall between a set of values. In my case, this relates to the correlation coefficient (r). What is a great example and explanation of CI?

In: Mathematics

Answer 1

Statistical significance is related to the chance that a measured value for a treatment will be different than what it is being compared against (the control). A p value below 0.05 means that a difference in results between the treatment and control was observed greater than 95% of the time. The selected p value (commonly 0.05) allows for the study to set their standard for the amount of times a difference is observed compared to the control in which they would consider the difference to be statistically significant. Given a p value of 0.05, if a difference between the treatment and control is only observed 90% of the time then the results would not be considered statistically significant.

If a study produces a result in the form of a specific number the result should be reported including a confidence interval which denotes the range of values that the true value for that result could fall under. Essentially, values reported in this way are generated from repetitions of the same experiment which likely will produce slightly different results each time. Because of this it is impossible to know the exact true value of the result so the number reported includes a confidence interval which is commonly the value determine plus or minus the range of one standard deviation of all of the collected values for that treatment.

I hope this makes sense! This is my understanding of the concepts as a first year masters student. Anyone in the comments feel free to correct me if I have misrepresented the meaning of these terms!