factual

Reviewers rarely check data or analysis

Most people misunderstand peer review as a checking process, but in fact nobody is usually looking at the data or rerunning the analysis—because doing so requires opening datasets (often not provided) and redoing analyses, which is a major undertaking—so the most important thing review could do is virtually never done; reviewers instead render an aesthetic judgment.

factualpending

Speaker

Adam Mastroianni

Evidence Quote

“It's a big undertaking to actually check the results of a paper, which is why it's virtually never done. Although that is, of course, maybe the single most important thing that this process could do, rather than provide some kind of aesthetic judgment.”

Source

Adam Mastroianni on Peer Review and the Academic Kitchen— EconTalk
Created: 6/13/2026, 12:08:52 AM

Your notes

Loading notes...