I’ve been reading about how some studies that we assume provide us with information about the world actually don’t teach us anything because of statistical manipulation that makes them look more relevant than they are. How does this work?
In: Other
Basically, p-hacking is the science equivalent of going to a firing range and covering the far wall completely with target papers.
You then proceed to fire every round you have blindly down range.
Once all ammo is expended, you go pull down all of the targets, discard anything which has a hole NOT on the bullseye, and collect all the targets where you did hit the bullseye.
Now you present this collection of well-shot targets to somebody and ask to be recognized as a sharpshooter.
A p-hacked result does show actual collected data. But the data was initially collected among one target population… and then it was narrowed down to be from a completely different target population so that the data does show something statistically relevant, when under the initial plan for the study the results were not conclusive.
Like maybe I was thinking that all high school students exhibit X behavior, but when I took data on 3 million high school students across the nation I found that wasn’t showing up in the data, I then start playing with the demographic data also provided. Did all hispanic students exhibit X? How about all students on free and reduced lunch? How about all females? Maybe all 12-14 year old Inuits from a single parent household? Ooh, I am in luck, there are actually only two such people in my data set, and they happen to both exhibit X! Now I proclaim that my “…study of 3 million high school students shows that, for certain populations, behavior X is present…”
Latest Answers