Editors vs. peers

Andrew Gelman links to Bruno Frey "suggesting a change in journal review processes, so that the editorial board first decides whether to accept or reject a paper and then referees are brought in solely to suggest changes on accepted papers." As Gelman points out this is vaguely similar to what some journals (I believe the best examples are Nature and Science) already do: have editors screen the articles before sending them out to review. There's a crucial difference though: at Nature, what seems to happen is that the editors screen the article to see if its claims are interesting, and mostly leave it to the referees to figure out if the claims are true. This is sensible because the referees -- who are typically people working on similar research -- are poor judges of broader interest whereas the editors are poor judges of whether the article is true.

The downside to this is that it creates a systematic incentive to make your article seem more interesting than it is and to hide the caveats in places where the editors will miss them. Sometimes the referees will object -- this article does not explain the origin of time -- but recommend publication anyway, and the editors are unlikely to overrule the referees even if the revisions make the article uninteresting enough that it wouldn't have gone to the referees in the first place. To some extent these problems are just inherent in the idea of a prestige journal, but there are ways to get around the puffing-up. One possibility is to do away with prestige journals, publish more or less everything that seems true, and have selections like Physical Review Focus to spotlight articles that were an especial hit with the referees and/or the editors. As referees vary in their willingness to praise, this scheme isn't that great either. In some ways a better plan is to have intelligent physicists blog their favorite new papers in a given area of physics, and to sort of hope that over time blogs with good taste win out. (As physicists are interested in fame, and probably read the relevant journals anyway, this isn't necessarily too taxing to work.) The -- possibly fatal -- objection is that it gives said bloggers an inordinate amount of influence over the career prospects of young physicists.