Do you:
• Quit because clearly you are too stupid to publish?
• Cry/scream and consider finding a career that makes everyone love you (e.g. proprietor of a lemonade stand)?
• Descend into paranoid bitterness because the reviewers and editors are morons and may be plotting to steal your ideas and/or squelch your manuscript so they can scoop you?
• Ignore the reviews completely and resubmit the same manuscript to another journal?
I personally wouldn’t do any of those things, although maybe I would do a bit of some of them first before calming down and dealing with the situation in a constructive way.
I have been thinking about this lately not because I got Mean Reviews (recently) but because I am working with a few people who have clearly been affected by reviews they have received in the recent past. Writing (different) papers with them now means having to navigate the psychic effects of past mean reviews.
If you get Mean Reviews, after screaming and being bitter for an hour or two or 168, look at the reviews closely. Some bad reviews have nothing of value in them and are just basically mean, but many contain some constructive criticism that can be used to improve the manuscript. If I’m going to resubmit a manuscript somewhere – to a different journal or the same one – I revise it in light of whatever review comments I think are useful. Submitting the exact same manuscript, even if to a different journal, might be a mistake, especially if it goes to the same reviewer(s).
It’s worthwhile reading reviews in a calm, dissociated way, especially if you don’t yet have much experience with reviews. I have written before about how, on several occasions, one of my grad students or postdocs has gotten reviews back and has been devastated by how negative they are. I look at the same review comments and think “This is a great review! These are nice comments!”. You can’t expect reviews to tell you that you are brilliant and that your science is perfect. Reviews are intrinsically critical, and if the system works as it is intended, this criticism will improve your work and your writing and help you publish the best paper possible.
I think that some disappointment stems from an unrealistic expectation that if you work extremely hard on an interesting paper and have excellent data, well-written text, beautiful figures, and all the rest, that the reviews will all be entirely positive. With reviews, you don't get an A for effort, but critical reviews are not the end of the world (or your career). And it's good to keep in mind that not all criticism is negative.
If the reviews are negative (as opposed to critical), in some cases a thorough and calm-but-firm rebuttal letter to the editor can be very helpful, especially if you can point out in a compelling way that the negative review is (1) in error about the perceived flaws of your manuscript/research, or (2) mean but vague (i.e., not specific about why the reviewer hated the manuscript).
In other cases, the negative reviews sink the manuscript. Maybe the reviewers were not objective, careful, or nice, but you still need to deal with the reviews and think about the comments carefully.
Not every paper can or should be published, but if the paper is (or will be) good, any disappointment with critical or mixed reviews should be short-lived as you dive back into the paper to fix it up so you can resubmit it and eventually have the thrill of seeing it published.
And in terms of how mean reviews affect your approach to other papers, I think it's important to keep some perspective and not become an ultra-perfectionist who focuses so much on cosmetic details that might be targets for scathing comments that completion and submission of the manuscript never actually occurs (Fig. 1)
I love your Figure 1 and have lived through collaborations with manuscript perfectionists. I'd just like to highlight the increasingly popular other threshold -- submitting a paper that's only 70-85% done. This forces the reviewer to work harder (reading a not-ready-for-primetime draft), and perhaps to do some of the manuscript writing for the authors (esp when the paper has only 15 references, instead of the 45-60 it actually needs). This could be a brilliant strategy for authors -- someone else is working on your paper for you! And you don't even have to acknowledge them!
ReplyDeleteBut it increasingly irks me that authors feel it's okay to waste my reviewing time with manuscripts that aren't ready for review, and it is getting harder not to write a justifiably 'mean review' in response.
Any scientist who does not develop a thick skin about paper and grant reviews is going to waste a lot of emotional energy through the course of her career.
ReplyDeleteHow about things from the other side? Do you have any advice on how to write a negative review? Or rather, can you suggest some things to keep in mind when writing a negative review so that it is received in the best way possible given the scientific content?
ReplyDeleteI just wrote a kind of negative review of a book. At times I wanted to be as frank as possible with the editor, and at other times I wanted to be very careful about not causing unnecessary harm. I probably didn't do such a great job, which is why I'm asking.
As a reviewer the thing I always find diffcult to square is the question of whether the manuscript reports "competent science" and whether it is interesting to me personally. With journal obsession over impact factors, referees are almost directed to reject papers if they don't consider them "exciting".
ReplyDeleteClearly we should all be able to agree on those mauscipts that are *scientifically* poor, and also see those that are really exciting, novel as well as being scientifically competent pieces of work.
But then, in my opinion, you get into the realms of personal choice for the majority of the middle ground of submissions. They are done well, but may be just don't rock your boat. Should you reject these?
It's probably my lack of experience, but I think I could handle working with someone who was a bit more on the perfectionistic side. What gets me are the people who think it's perfectly fine to completely ignore what I take to be valid comments by reviewers. Maybe they aren't always stated well (or nicely), but that's not grounds for saying you don't need to worry about them. (Especially if this paper is for a conference, and the reviewer could show up and ask why a point isn't addressed...)
ReplyDeleteI'd like to see the plot that explains why the last 5% of work on a manuscript takes 95x the time (or at least, perceived effort) of the complement.
ReplyDeleteI have a simple policy for dealing with paper responses. I look “above the fold” to find out what the decision is. I then put aside the message for 2-3 days, and read the actual reviews only then. It's important to read the decision immediately, because of gnawing curiosity. After a few days the disappointment wears off, and now you're receptive. The practice works remarkably well for me, and I encourage all my students to do the same.
ReplyDeleteIn general, I'm rarely upset by reviews. In fact, I tend to get more upset at poor reviews of papers (or grants or ...) that get accepted—because they've robbed me of a chance of getting useful feedback—than by any reviews of material that is rejected.
Not, of course, that there is any doubt whatsoever that the reviewers who rejected my work were dead wrong and simply lacked imagination and vision.
Good advice. When I get negative reviews of any sort, after the initial yelling and screaming I put them away and try not to think about them for a few days. It's amazing how some initially-negative sounding reviews become quite positive when considered more objectively.
ReplyDeleteAnd that figure was a great start to my Monday. Thanks!
I am very much a perfectionist when it comes to writing. It pays off. I often get compliments on the clarity and content of decades-old papers of mine which have been read by colleagues or students. I think it is well worth the effort, and I wish more of my colleagues would adopt this attitude so that their papers would be easier to read and understand. I have only thrice been asked to rewrite a manuscript, and have only twice had a paper rejected that did not eventually appear somewhere.
ReplyDeleteI have a question about resubmitting a paper following a first round of 'bad' reviews. After appropriate revision, the author wishes to resubmit the manuscript to a different journal. Should the author:
ReplyDelete1. Not tell the journal that the manuscript was previously considered elsewhere (i.e., pretend that this is the first round).
2. Tell the journal that the manuscript was previously submitted elsewhere, but not include the reviewers' comments or the author's rebuttal.
3. Tell the journal that the manuscript was previously submitted elsewhere, and include the reviewers' comments and the author's rebuttal.
In my academic life, I have seen people do all of these options, but which one is the most likely to lead to publication?
"Any scientist who does not develop a thick skin about paper and grant reviews is going to waste a lot of emotional energy through the course of her career."
ReplyDeleteActually, it's worse than that. They fail.
And, the advice about putting away reviews and reading them later is absolutely necessary for those who don't have naturally thick skins.
janus professor - That's what I am planning on writing about for tomorrow, so it's good to know that someone else is interested in this topic as well. Thanks.
ReplyDeleteI'm a section editor of a journal in my field and for the most part, reviewers keep the nonsense at bay. My two big complaints with reviewers are 1) not being critical enough as in "hey, looks good... accept" and 2) not checking into some of the other work cited to see if it actually backs up their findings or not.
ReplyDeleteUsually rejections come from one reviewer really putting some thought into it and saying flat out "reject" with evidence and citations to back up their reasons for recommending "rejection." If there's one flat out "reject" and one flat out "whoo hoo - accept" but the "accept" review isn't a critical review, then it becomes "major revision - please fix and resubmit" and it gets re-evaluated (sent to another reviewer in addition to the comments about reviews).
The question by Janus -
PLEASE SEND THE REVIEWER COMMENTS along with your submission!!! This helps SOOO much. I have gotten this and it helped me tremendously. It also saves reviewer time (prevents re-review of the same paper, essentially). Do a point by point of how you addressed each previous concern after each reviewer comment (you can provide line numbers rather than "bubble commenting" in the text. It's best to stick the "comments to previous reviews" part in with the cover letter attachment when you first upload your paper to new journal.
For me as author - I get reminded that I am a woman quite often. One of my single author paper reviews amused me - pretty much asked if I could cite a basic tenet of my field "as proof I understood" basic concept (learned in "my field" 101 class, about 15 years ago). My response was that I teach "my field" 101 class and here's a citation from the text I use for lectures.
I have gotten my fair share of mean reviews - one retard going so far as to "reply to" my accepted article (which royally ticked him off that his mean reviewer comments were shot down in flames). My dream responsive action to mean reviews: blow them to hell. After I get done dreaming, I write out a nasty rebuttal and put it away for a week - then I shred it into confetti and delete from computer. After my little "cleansing of my mind" exercise to calm down and get rational, I then write a civil direct point by point blow to each mean reviewer comment. It makes all the difference in using a filet knife in a calm manner rather than a butcher knife to whack away!
As with so much else in writing, learning to make use of a review (mean or otherwise) is a skill. It is learned with no small amount of effort. And there is nothing sadder than experienced writers who have never acquired the skill. It forever limits their growth.
ReplyDeleteOften after you calm down, it is a good thing to talk with the editorial staff. Don't blame them, don't get off on the reviewer but walk through why you think they should accept the paper
ReplyDeleteI thought I had a thick skin but my last review threw me for a loop. In the first place there's maybe eight people who can understand what I did. Second the uber mathematicians protect their turf like crazy, kicking out anyone who dares come near their work. I got a review saying I needed to reprove a theorem a different way. Problem is the referee was deliberately vague/I can't figure out what he's saying. I don't want to ask for more help because the referee also suggested my proof wasn't sophisticated enough and maybe I shouldn't be playing with the big boys. I'm thinking damn my proof is right and first proofs are never so good anyway. So what if it is not sophisticated. How to approach this? Thanks
ReplyDeleteThese are interesting posts. I work for a one-word-name journal and we reject about 90% of the manuscripts we receive.
ReplyDeleteOften, reviewers fall into a few categories:
-those that don't do a thorough review and offer no feedback.
-those that obviously know and dislike the manuscript authors and let their personal feelings show (often the manuscript authors can identify these people just by their tone/comments).
-those that do a thorough review, complete with comments to the editor and author and suggestions for revision. Some even go through the manuscript and highlight the grammatical errors using the track changes feature in Word.
And then there are the reviewers who agree to review and then never turn the reviews in.
I guess my biggest complaint is the tone. I don't see why so many reviewers get so emotional. They can't be critical and write scientifically, nooooo. Instead they are downright nasty and make everything personal.
ReplyDeleteMy guess is that this means the majority of reviewers on my papers either have a conflict of interest with the subject or hate my boss (not that hard to imagine, really) or both.
Unfortunately the editors never reject the REVIEWS or go out of their way to find someone qualified but objective.
This is why I think peer review is bullshit. In some fields, the politics are too strong. All the papers are reviewed by the same handful of hot shot old boys, and they act as gatekeepers for the science. Unfortunately when we're deciding what fields we're interested in studying, nobody tells us these things (ie in grad school or young postdocs).
It seems like you've had a difficult time with reviews, and I am sorry for that, but I hope you don't really believe your statement about editors and reviewers:
ReplyDeleteUnfortunately the editors never reject the REVIEWS or go out of their way to find someone qualified but objective.
I am angry about slow reviews. When reviewing my submission takes eight months, then a rewrite is requested (turned around in three weeks), and then another eight months go by before the whole paper is rejected for being "inappropriate for this journal", something is drastically wrong. How could it possibly, in any universe, take a qualified reviewer eight months to read a paper? In my husband's field, if a reviewer fails to submit a complete review within six weeks (at some journals, two weeks!), then the editor will find another reviewer. I want to change fields just so I can get some feedback before my hair turns gray waiting.
ReplyDelete