Peer Review and Appeals to Authority

Suppose you write a blog post arguing a point. Some readers will reply (or, worse, think it without saying it): “If that was true, you would get it published in a peer reviewed journal. If you don’t do that, it’s because you’re wrong.” The assumption here is that peer reviewed journals do very accurate gatekeeping about which ideas are good or bad. The assumption here is that almost all the good ideas get published and few of the bad ones.

This is a companion discussion topic for the original entry at

I think this is a relevant quote from Atlas Shrugged:

“Independence is the recognition of the fact that yours is the responsibility of judgment and nothing can help you escape it—that no substitute can do your thinking, as no pinch-hitter can live your life—that the vilest form of self-abasement and self-destruction is the subordination of your mind to the mind of another, the acceptance of an authority over your brain, the acceptance of his assertions as facts, his say-so as truth, his edicts as middle-man between your consciousness and your existence.”

I shared some of your articles about peer review with a friend of mine (he’s a medical doctor). He didn’t seem overly concerned about the lack of transparency in the peer review process. His attitude was mainly that it would create a lot of extra work that nobody would even end up reading.

He said the biggest problem was people falsifying or manipulating data. Another problem he brought up was that the peer reviewing your paper could be working for a rival lab.

He said the blame for the lack of transparency was on the companies doing the publishing. He said that they are too greedy to do something good (add transparency) if people aren’t willing to pay money.

I pointed out that people already have a high level of trust in the process of peer review, even when they’re aware of problems around reproducing results or falsified data. If a “greedy” company was losing customers due to lack of trust then maybe they would consider adding more transparency to the process. Also, if there was a demand for a transparent peer review process then some “greedy” company would rise up to fill that demand. If people don’t want a transparent process, I think it’s more on the people than on the companies. If people want an authority to trust in, they might not like the idea of seeing behind the “magic curtain”.

That being said I assumed the peer review process was more transparent (inadequate society). I’m not sure how transparent any of these companies are regarding their lack of transparency in peer review.

Thanks for sharing these great articles.