Who peer reviewed an academic paper? That’s secret.
More importantly, what criticisms did the peers reviewers come up with? That’s secret.
Reaction/commentary on the article:
Quote from article:
And, despite all the citations, they tend not to cite many counter-arguments to preemptively address potential criticism. They mostly cite things to back up positive claims.
This is in contrast to Popper’s or CF’s method of criticizing flaws in existings ideas or arguments. Cites with positive claims don’t add strength to your claims. I would guess the main purposes that one should cite positive claims are to show which ideas your claims are building upon and where to find more detailed arguments for ideas used in your paper. In addition to referencing those supporting papers, academics should be citing and criticizing their opponents. Researching and addressing all outstanding criticism is how scholars could and should be making progress in their fields.
Quote from article:
When a member of the public thinks of a criticism of a paper, they can’t tell if a peer reviewer already brought up that issue, or if the reviewers just missed it. They don’t know if the issue was analyzed or neglected.
I would guess that very few peer reviewed articles are addressing anything like the state of debate in their field. In the prevailing view of epistemology, the authors of academic papers don’t have to address all their critics to publish what they consider a valuable contribution to the field. That leaves open the possibility that a lot of published papers neglect outstanding issues.
Quote from article:
I think peer review debates often don’t actually reach conclusions, just compromises. Authors and reviewers still disagree, so some wording is changed or some citation added, which no one is fully satisfied with.
Peer reviewed papers that result in compromises are like the compromise political outcomes referred to in BoI.
Quote from BoI:
If a policy is no one’s idea of what will work, then why should it work? But that is not the worst of it. The key defect of compromise policies is that when one of them is implemented and fails, no one learns anything because no one ever agreed with it.
The peer review process could be creating many papers with arguments and explanations that no one want’s to defend in debate because no one believes those arguments. Compromise papers create the potential for criticisms from which no one learns anything.