I’ve been looking at some series that are based on peer-reviewed science like “Missing Links” by Gregg Braden, and I’ve been reading that recent studies use peer-reviewed science, but it’s not clear to me what it really is. If someone could ELI5, I would appreciate it.
In: Other
A scientist comes up with a theory and tests out the theory with experiments in order to see that the experiments are genuine and produced genuine results another scientist in the same scientific field checks the results, this is peer review.
Peer review is the process of having your work reviewed by others in your field before it is published.
The idea is that if you write a scientific paper and it isn’t total bullshit a team of reviewers who are also experts in your field should be able to review it and say “Yeah this is proper science and makes sense” and then they pass it on to the journal and say “We think this is good enough to be published”
Science is very complicated, at the edge where new discoveries are being made. As a result, the editors of a scientific journal can’t possibly know everything about every research topic. So instead, they keep a rich directory of who is working on what topic. When a paper comes in about topic X, they look up who else they know is working on X. Then they send the paper to these scientists, who are called “peers” of the scientists who wrote the paper. The peers read the paper and point out things that they think are missing or incorrect. The authors of the paper get an anonymous copy of the peer’s comments, and they revise the paper to fix it. Then the peers review it again, etc. When the peers agree the paper is “not wrong”, then the journal feels confident to publish it.
This is a peer-reviewed paper, a much more trustworthy source than a scientist’s blog post.
Peer-reviewed means one group of researchers did the experiment and recorded all the information about it (why they did the experiment, how they did it, what the results are, and how they interpret the results), and then *another* group of researchers reviewed all that to determine whether or not it was actually done *right*. Like, if a group of doctors did a study about amputating nipples, other doctors or a panel would review that study to decide whether the study followed all the rules for medical studies that were established by the international association of doctor folk.
It makes the study more legitimate/valid. It is the difference between me doing some random crap in my basement and then saying, “Hey look at my findings here, this will change the world!” and a researcher applying the right methods to do an experiment to ensure everything was done right.
Peer review raised the likelihood that the conclusions drawn in the study are supported by the data in the study. It is also a process that refines the reporting of the study. It does not necessarily mean the conclusions represent Truth in the Universe.
I submit a paper to a medical journal. An editor reviews it, and decides if the subject, methodology, and conclusions are worth potentially publishing. If yes, the manuscript is submitted to (usually) three or more reviewers, and one of them is often a statistician. The reviewers do a (hopefully) detailed review, with comments to the editor regarding bigger picture issues (2 tables need to be refined and some specific methods need to be better described; this is potentially a useful addition to this body of knowledge), and more detailed comments to the author(s) USUALLY designed to make them better writers. Grammar/typos are left to the editing process if accepted.
The manuscript is rejected with a polite (but often not overly helpful) letter, accepted as is (pretty uncommon), or returned with a request for revisions. If the revisions are then accepted, it gets published. NOW you, the reader, can safely assume that the study is worth your time and effort to read critically (also assuming it is in a journal that you care to read).
***HOWEVER,*** depending on why you are reading the study, you still need to do so critically. What confounders are there that may have influenced the conclusions, and in what direction? In medicine, we know that 40-60% of what gets “accepted” as truth this year will be shown to be not so true over the next 5 years or so…the premise that any science is settled is sketchy beyond some physical constants.
Others have explained what peer review is. Although it’s a whole load better than nothing it’s not perfect. Many peer reviewed and published papers describe work that turns out not to be reproducible by others; and perversely unreproducible results get cited more than reproducible ones.
In a way it’s not surprising. Papers making exciting and interesting claims are more likely to cited … but they’re also more likely to be wrong and hence not reproducible.
I have a great new idea. I devise an experiment to demonstrate my idea. It seems to show that my idea is true. I submit the paper for publication, and the publication sends it to other experts to see if there are any glaring errors. They might see faults in my methodology or data. If they don’t see big problems, the publication publishes, and my idea and experimental methodology are shared. The other experts are my “peers” (equals).
Once it has been peer-review published in a reputable journal that doesn’t make my idea accepted fact yet, because other people will try to disprove it / see if they can replicate my results. They are also my peers. If enough people can reproduce my results and people generally agree that my interpretation is correct, it starts to be accepted as an idea, and others start to build upon it.
Look at “Dr” Andrew Wakefield who started off publishing in a very reputable journal, but over time his peers couldn’t find evidence to back up his MMR vaccine claims, and over time disproved them, and later still showed that he was a fraud who faked his results in oreder to increase sales of separate vaccines that he had a financial stake in. It takes a long time to weed out the fraud, but it is rigorous over time, and science rejects things which aren’t demonstrable.
It’s an excuse for not doing science properly.
The scientific method isn’t supposed to involve trusting people are checking they did a good job with their experiments. Don’t need it. All you have to do is have person A say “hey I did X and then Y happened” and then other people can try the same thing and see if it happens for them too.
No need to worry about if that first guy was a liar, a cheat a fool, or just got lucky. If anyone can reproduce the experiment for themselves that’s science.
But it’s often expensive to repeat the same experiment and a bit boring and it doesn’t get you in the newspapers or win science prizes. So more and more people don’t bother. They want to be doing their own cool experiment not reproducing someone else’s and then that guy gets the credit. So they cut corners. Instead of reproducing the experiment they look at the other guy’s notes and try to criticize it theoretically. Pointless. There’s no substitute for trying to perform the experiment for yourself.
Latest Answers