A few days ago I wrote this as one reason why I would decline to do a manuscript review:
"I've already reviewed the manuscript for another journal, it was rejected, and I can't bear to read it again in its resubmitted form to another journal even if I feel that humanity will suffer if this paper is ever inflicted on the world."
If I get a manuscript to re-review for a journal different from the one in the first go-round and the manuscript looks like it has been substantially revised, then I will review it. BUT, if I see the exact same (or very similar) awful manuscript again, it means the authors have ignored my comments completely and are just going to try again in the hopes that they find an editor who will either not send it to me or who will ignore my comments and recommendation.
It's just not a good use of my time. I used to do it, but some manuscripts work their way down the journal food chain and appear somewhere, in some cases with an acknowledgment section in which I am thanked for my input (but no mention that my input was that the paper should never be published and that I violently disagree with the methods, data, interpretations, figures, title, punctuation, citations..).
I should mention that I almost always sign reviews, but being acknowledged in papers I think should never have been published is one time I wish I'd chosen anonymity. When I read a really awful paper, I always wonder "Who reviewed this thing?" and I check the acknowledgments..
I have a colleague who keeps a list of people whose papers he reviews and who don't take his comments into account when revising the paper. If he sees a published paper that does not show that the author revised according to his major review comments, he refuses to review a paper by that author again. I am not that extreme, and am not so far gone that I think all my review comments are so great that a paper revised without them is fatally flawed. But I can sympathize with the feeling that doing the review was a waste of time, so why waste more time?
From the point of view of an editor, I know that most authors provide a detailed and thorough response to reviewer comments, and seldom are major reviewer comments 'ignored' without a reason.
14 years ago
7 comments:
I think it's great that you sign (most of) your reviews. Usually in my field reviews aren't signed, which is unfortunate at times. I think that usually it is better to sign reviews. Also the major conference in my area is trying double-blind review this year. This doesn't seem to be a great idea to me; it's often not such a mystery who's doing what.
This is great. I believe that peer review system is outdated. Peer Review is a self-regulatory system invented by Professional Doctors to police their professional peers. It is akin to Ethical guidelines devised by legal doctors. What is not surprising is that Doctors defined their professional self-regulation to be science. Reading how peer review works at the fundamental level proves what a sham peer review is.
I don't think peer review is a sham. I think it has flaws, as does anything involving large groups of people, but in general it works. As an author, I greatly appreciate it when I get a thorough and constructive review, and this is the most common case for reviews.
Thanks for that clarification. Peer review is a lot like democracy--it's far from perfect but no one has devised a better system. As an editor I can see the process now from all three sides (author, reviewer editor). In spite of all of our complaints, I think the system largely works well, with reviews based largely on the merit of the work rather than who did it, and the procedure improving papers and rank ordering them as well.
In biomedical sceince (my field), my major beef these days is escalating expectations. The last five papers I have submitted have been returned without a single real complaint about the quality of the work contained therein. Instead, they come back with detailed suggestions of the next experiments that need to be done. I also know what needs to be done next, but think at some point you have done enough and the paper needs to be published. As reviewers, we often feel our "job" is to find something to criticize, and if the work is good we need to invent new experiments to be done. I have been working myself and lobbying my colleagues to realize that sometiems a paper is a nice peice of work, and we need to say that and say that forcefully.
Hooray for Anonymous who brings up the Escalating Expectations problem. It's horrific now.
Reviewers seem to prefer mountains of crappy data over a few elegant, carefully chosen and expertly executed experiments that actually support the claims being made.
LONG gone are the days when a 1-panel figure could stand alone, at least in my field.
I tend to agree that peer review is flawed to the point of being almost useless, at least in my field (where no one signs reviews).
It's a complete crapshoot what 3 reviewers you get. How does that really represent science, especially if you're writing on a project that crosses fields? Then you get one expert from each field, none of whom can evaluate the whole thing?
And just one bad review out of 3 or 4 is enough to sink a paper now. How is that a good thing for science as a whole? Are we to assume that 1 bad review is the 'right' one, or is it more likely the 1 bad review just happens to be an idiot or deliberately trying to sink the paper?
I also think conflicts of interest are a far bigger problem than anyone wants to admit.
So... do you think it's usually better, if you get half-useful and half-idiotic reviews, to revise & go back to the same journal rather than go elsewhere? I have been at the mercy of my advisors' decisions re: which journals and when to fight crappy reviews vs. go elsewhere, so I'm really curious about what you think.
If the editor is doing their job, the idiotic reviews should be ignored. For some journals with astronomical submission rates and rejection rates, the editor might not want to deal with a negative review. In other cases, though, a well-reasoned letter to the editor is worthwhile. I've had to fight for some papers -- in some cases I won, in some I lost.
"I don't think peer review is a sham."
Maybe that was an oversimplification. But what is the point of peer review? Reading your blog I get the impression that you are spending a ridiculous amount of time on administrative chores including reviewing and letter writing, and endless meetings dominated by bureaucratic types. Let's imagine that the government imposed peer review system on technology companies and startups. Since peer review is such a good thing it would help the technology companies to make more advances and add to the scientific knowledge of humankind. But this seems not to be true. I would say that if technology was also burdened by peer review system we would not have the computers or the internet. If you want your paper reviewed just post it in the internet. But as an academic you cannot do this. The system requires that you get academic points for publishing in peer reviwed journals. In order to advance your career you must publish in top tier journals controlled by giant publishing industry. What a waste of human resources!
Post a Comment