It is completely understandable that some people who are new to reviewing manuscripts may lack confidence in their reviews and wonder whether they are commenting on the "right" things in the "right" way and in the "right" amount of detail. It might be comforting for these uncertain reviewers to see the comments of the other reviewers to gauge whether they are on track with their own review.
Consider, though, that the content and styles of reviews vary a lot from reviewer to reviewer because reviews should vary a lot. It isn't necessarily such a good thing, then, if your review comments correspond well with those of other reviewers. In fact, if you commented on completely different things that the other reviewer didn't consider or notice, this could make the reviews even more useful to the author(s) and editors.
As an editor, I try to select reviewers who will examine a manuscript from different points of view. Of course different reviewers should be able to detect any major flaws in a manuscript, but beyond that, I want different reviewers to comment on different aspects of a manuscript. Otherwise, why get more than one review?
Doing a review is not a test that you pass or fail as a reviewer. You are providing a service to your professional community by sharing your expertise.
What should a review contain?
Reviews should provide a thorough and constructive critique of the content of a manuscript, focusing in particular on the data, analysis, and/or ideas that form the basis of the work. You should consider whether you think the interpretations are justified, making it clear whether something is objectively "wrong" or whether you just don't like it.
I am not impressed when a reviewer says "This is wrong" about something in a manuscript but doesn't explain why and doesn't provide any suggestions about better alternatives or ways to get at a not-wrong result. If you want your review comments to be considered by the editor and the author, you need to back up any criticisms as much as possible. It also helps to be polite. Your comments will be taken more seriously if you write "I do not agree with the interpretation that A follows B, and suggest instead that..." as opposed "These people are seriously stupid if they think that A follows B."
Note: Some reviewers write their reviews in the active voice, directly addressing the authors as "you", as in "You should consider deleting that entire section that starts on page 17"; others refer indirectly to "the authors", as if the comments are addressed primarily to the editor, although of course the authors will read these comments. Perhaps the custom varies in different disciplines or different journals, or maybe it is just a reviewer personality thing.
In your review, if you are going to suggest the addition of a citation of one or more of your own papers, you should explain why this should be done. As a reviewer, I typically comment on citations of my own papers, or lack thereof, only if (1) a paper is mis-cited (i.e., my paper on purple kangaroos who live in lunar craters is cited after a statement noting that green rabbits live on Neptune); or (2) if there is an egregious omission (i.e., a statement that purple kangaroos live in lunar craters and then no citation of my work whatsoever, despite the fact that I have published extensively on this topic and no one else has).
Reviewers should, to some extent, consider the manuscript as a completed thing and only suggest the addition of new items (particularly those involving new research activities that may be costly in terms of money and time) if those items are absolutely essential to the manuscript; i.e., without them, the paper is not publishable. If you truly believe that a paper is not publishable based on its current content, of course you should state this in your review and back up this opinion with reasons, but if you just think the paper would be better with more data, make it clear that you are making a suggestion, not pointing out a fatal flaw.
This is where an editor can play an important role. In many cases when I see this type of do-more suggestions in reviews, I tell the author that I think this is an unreasonable suggestion and they do not need to address it in their revision. In other cases, I concur with a reviewer and tell the author that their paper is not publishable without this additional work; fortunately these cases are quite rare.
It is also very useful if a reviewer comments on technical aspects of a manuscript, particularly those issues related to clarity. Does the organization of the text make sense or does it interfere with your understanding of the major points of the paper? Is the paper too long/too short? If not, what is a better way to present the information?
I understand if a reviewer does not want to take the time to fix the writing problems in a manuscript, especially if there are a lot of problems. I don't like doing this as a reviewer and I find it annoying as an editor, especially if at least one of the co-authors is a native English speaker. I am not paid for my work as an editor, and co-authors who are able to fix their own writing problems should take the time to do so before submitting a manuscript and expecting others to clean it up. If none of the authors are native English speakers, of course I can and do help with writing issues, but I am always grateful when a reviewer takes the time to do this as well.
To those who lack confidence in their reviews: don't worry. Just do a careful job of reading and commenting on the manuscript, and there is a very good chance that your review will be helpful to the author(s) and editor.
Keep in mind that your main job is not to point out new and different research activities that the authors should have done or could do, and focus on what is in the manuscript. Provide both general and specific comments (including noting what you liked about the manuscript, if anything, not just what you didn't like), note any problems you find, make constructive suggestions, and don't take too long with your review. Also remember that by reviewing a manuscript you are doing a great service to all involved, and perhaps learning something interesting, so don't stress out, just dive in and write out your comments.
14 years ago
33 comments:
I am not impressed when a reviewer says "This is wrong" about something in a manuscript but doesn't explain why and doesn't provide any suggestions about better alternatives or ways to get at a not-wrong result. If you want your review comments to be considered by the editor and the author, you need to back up any criticisms as much as possible.
This is my single biggest pet peeve as an author in the peer-review system. Every review I write, if I make a criticism, I do my homework to back up my criticism to make absolutely sure it is correct (and communicate the results of the homework to the authors for their benefit). Nothing enrages me more about reviews than a blanket statement that something is wrong, unrealistic, unjustified, etc., with no arguments to back up the criticism.
Let me backtrack on that a bit: nothing enrages me more than that, except editors who allow such nonsense reviews to pass uncritically. It reminds me of modern 24-hour cable news. Instead of trying to figure out who is correct and incorrect based on the strength of their arguments, instead some editors put authors and reviewers in a cage and let them bloody each other verbally. Whoever yells loudest wins.
If I am an author, and I get a review that states, "This is wrong" with no justification, I blame the reviewer but I also blame the editor, even more so given their elevated level of responsibility. True, it may be their job to farm out the time-consuming reviewing to referees. But if a referee comes back with crap, it is the job of the editor to recognize it as crap, and to either demand something better from the referee or to find a new referee. To pass it along to the author unchallenged means that the editor may as well be replaced by software.
Most reviewers I've had seem to make it their mission to play a game of "gotcha!"
The professional value of reviewing?
I was hoping for comments on how important being a reviewer is, especially with respect to newly minted PhDs, when it comes to personal career advancement. Does being a good reviewer help with your own publications, does the goodwill it generates go a long way?
How do you feel, as an editor, when reviewers present incompatible views?
I recently had reviews back on a paper where reviewer 1 said "I'm not surprised you haven't found an association between A and B, your sample is different to the main studies that have found this, condense this section and move on". Reviewer 2 said "How VERY surprising you haven't found an association between A and B, ALL the other studies have found this, this must represent a major flaw in your study, you need to explain this more".
I am inclined to agree with Reviewer 1 (though I also feel our methods are rather different to most previous studies) and this isn't an R&R - it was rejected - so I suppose the editor didn't need to give me instructions on what to do about the clash. But I've had editors before leave such discrepancies with no clear indication of what they think, even where a revision was invited.
(I am working on this paper for submission elsewhere, and I'll probably condense the discussion of A/B relationships while emphasising more the sample difference. But if I was left to respond to both of these reviewers I'd have a hard time!)
Some reviewers write their reviews in the active voice, directly addressing the authors as "you", as in "You should consider deleting that entire section that starts on page 17"; others refer indirectly to "the authors", as if the comments are addressed primarily to the editor, although of course the authors will read these comments.
I have had reviewers refer to me directly by name, like "I know that PhysioProf thinks that blah, blah, blah, and he is always beating this horse to death, but really PhysioProf is full of shit."
BTW, there is an omission in your instructions to reviewers. While not all journals do this, many journals ask their reviewers to opine on the suitability of the work for that particular journal and in light of its stated editorial standards.
This is usually couched in terms of "importance" of the work or its interest to a "broad audience", and reviewers are invited to opine on whether the work is "more suitable for a more specialized journal". This judgment has nothing to do with whether the work is carefully performed, well written, and contains all of the appropriate controls, etc. Rather, this is an explicit judgment that goes beyond the metes and bounds of the work itself and to that extent can be contrary to your admonition to "[k]eep in mind that your main job is not to point out new and different research activities that the authors should have done or could do, and focus on what is in the manuscript".
In Cell Biology it seems the reviewers almost always ask for at least a few additional experiments. Sometimes they are 'beyond the scope of this paper' but other times a couple months more work must happen before revising the paper and resubmitting. At least that's been the experience of myself and most people I know in that field.
I think these are great suggestions. As an editor, I agree with all of these points -- especially about being polite, trying to say something nice, being clear on suggestions (and how to fix them), and not automatically telling authors they need to cite you. Related to being polite -- us, you are anonymous to the author, but not to the editor, who may not forget (ditto about late reviews).
As an editor, another issue I've encountered is a review that is mostly positive, and then the (not sent to authors) recommendation to reject. If you think it's not publishable, make sure that's clear in your review. Otherwise, I (who am not anonymous) come off looking like the bad guy.
Thank you for addressing editorial work. I was one of the people who requested some posts on these issues, but have been too busy (with backed up editorial duties, among other things) to read your posts thoroughly.
Some of the journals I work with no longer have copy editors. I never make detailed corrections on the writing as a reviewer, because it is not a good use of my time, and this seems to be a common attitude now. So who checks the writing of mss now? Is it the editor's responsibility to proofread it? (Please say no, please say no, please say no). Should the editor read through it then send it back if she detects any typos?
As an editor, I have to laugh at the idea that I am some sort of secretarial intermediary between the important people, you know, the reviewers and the authors. I suppose this could work in an ideal world. But frankly, as an editor, I see a lot of bullshit behavior with both authors and editors.
I'm so tired of seeing lists of recommended reviewer that are all colleagues of the authors, old friends, known easy reviewers. And the list of opposed reviewers includes anyone qualified to evaluate the paper. Come on.
And reviewers? How hard is it to decline to review something in a timely way? How would you like someone to sit on your manuscript for a couple of (extra) months? And when an editor contacts you personally to ask if you really think you'll be able to get it done in the next week? Answer the email. Even if you have to say no. Come on.
And the reasons for not reviewing papers. Please. You're too busy? Fine. Not qualified? Ok. Because your last manuscript to X journal was rejected? You should take it as a compliment (or at least a vote of confidence) that we value your opinion (and it might help you get your next one accepted). Because your last manuscript was rejected and you didn't find out about it for a really long time? Then be part of the solution. And this, my all time least favorite half-assed excuse: You don't get paid to review papers and you don't get credit for it, so why should you do it? I will blow a fuse the next time I hear this. Anyone who says this, or thinks it, should not be allowed to submit their manuscripts for publication.
The issue of suitability of a manuscript for a given journal is really tricky. Oh, maybe not, for those of you living in a lofty physioproffian world of C/N/S papers. But for the rest of us, yes.
I publish, review, edit primarily in middling specialty journals. Hey, it's my field. The problem is that even middling specialty journals state that they only want submissions that are of interest to a broad audience, representing original, innovative, blah blah blah, fancy shit research. Obviously there's a major disconnect between such statements and what actually gets published in those journals. But even middling specialty journals vary in quality, citations, readership, respect, etc. At some point, the editors (and/or reviewers) have to decide if a paper is "good enough" for a given journal without any objective guidelines. I did not appreciate this until I became an editor of JMSS (Journal of Middling Specialty Stuff).
I got a one-sentence comment like "Your work is well-done but not of general interest". Is it enough for a fair comment even if the work is not too good?
thanks for responding to my question from yesterday. one small follow up. When I do make an effort to accept 90% of the papers I am asked to review (or suggest someone else) and to get my comments back on time, is there any chance the editors will remember that when I submit my own paper? I hope they do.
If someone is a very diligent reviewer, I try to handle their own manuscripts as efficiently as possible. That's the only way I can (ethically) 'reward' someone for their reviews in terms of the editorial process, although the esteem and goodwill you develop as a diligent reviewer may be useful in other aspects of your career. Conversely, I don't 'punish' people who refuse to do reviews or who do lousy reviews, but I don't make any extra effort to move the process along rapidly when they submit a manuscript.
When I review, I diligently check for plagiarism, and it seems that other reviewers neglect to do so. I find plagiarism of other folks' work - in addition to self plagiarism - all to often.
- in addition to self plagiarism -
JP, what is self-plagiarism in your book? A copied intro paragraph, substantial reuse of the same data, republication of results?
I consider myself a careful reviewer, but I have never thought about checking for plagiarism. That is too depressing, that we should have to worry about it.
Great post. I wish all editors shared your philosophy.
I wish it were the case in my field that editors eschewed "this is wrong" comments without justification or constructive suggestion attached. In most cases that I've seen, the editor isn't capable of acting as a filter. They're overworked, lack the expertise, and they don't have time to go find a better reviewer. They take what they can get, no matter how poor the quality. Maybe the editor even acknowledges this reviewer's performance makes them a poor choice in the future; but it won't help me with my manuscript right now.
I also heartily wish it were the case in my field that manuscripts would be considered as essentially "complete". It is quite common to ask for additional experiments that would easily constitute an entire additional manuscript worth of work (year or more) if not an entire career's worth (if only I had a lab army of my own!).
And then people tell me my CV looks weak (!).
I think the only thing I might take issue with in this post is the idea that people don't or shouldn't want to improve the writing. If we're going to be lazy about that, we might as well just make video presentations.
Dear Janus Professor:
You can not, by definition, plagiarize yourself.
If by chance an author is repeating a large portion of their own writing in a manuscript it could be rejected based upon it being not a new body of work. However, it is not plagiarism.
Regards,
Prof. L-Siz
You can not, by definition, plagiarize yourself.
I feel like it's not that clear-cut. If authors A, B, and C co-author paper P, and then authors A, B, and D co-author paper Q by copying some of P, has plagiarism occurred? If so, who has been plagiarized by whom? Would the answer be different if the author order was C, A, B on P and D, A, B on Q?
Ms. Ph.D., I draw the line at editing the writing. Why? Because my time is precious. I can spend a couple of hours rewriting someone's manuscript for them (circling all the typos...), or I can spend a couple hours thinking about and evaluating the science. Which is more important?
Don't get me wrong. I have very high standards for the writing, but it's the authors' responsibility. I'm convinced that a lot of people barely even proofread their writing any more - they are the lazy ones, relying on reviewers to do the work for them.
Thank you for this! I wouldn't be surprised if this is a common concern ("am I doing it right?"), especially among junior faculty. I know it was for me.
I've only been first author on 5 papers, but I gotta say--I've never had a review that wasn't either:
a) helpful in a number of ways, even if annoying or
b) low content but positive overall.
Dear L-Siz:
You can, absolutely, plagarize yourself. Because what you are plagarizing is a previously published article, which is usually copyrighted or at least has been through some formal license to publish which does not allow ridiculously lazy people to just copy and paste their previous text into the next manuscript without using their brains a tiny bit.
Sorry for the aggressive response, but it's not acceptable that you don't know that.
I've never had a review that wasn't either:
a) helpful in a number of ways, even if annoying or
b) low content but positive overall.
That had been my experience until recently, when we broke some new ground creating a new area. The reviews of this work, whether positive or negative, were almost all universally worthless. Getting the message across of "look, you really need to think this anew before you can pass judgment on it" was a very difficult process, and I believe we succeeded more by having the same reviewers revisit the paper in resubmissions to alternate venues than by anything we wrote.
In my opinion this is due to the fact that as reviewers we spend most of our life reviewing incremental work, and when faced with a breakthrough whose value is not self-evident (as ours was), we don't even know where to begin the process of evaluation.
I'll correct writing if the errors are infrequent. However, if every single sentence needs at least one correction due to either bad grammar or highly non-standard usage then I stop trying because the constant corrections interrupt my train of thought while reading and trying to understand the science. I'm not talking about a couple of corrections per paragraph, I'm talking about every single sentence.
I've worked hard to try to give the authors the benefit of the doubt in those cases, but the science in those sorts of papers is almost always really bad. My theory is that it's about sloppiness: People who are conscientious about their work will know that they are bad at English, and they'll care enough to get a more skilled colleague to correct it. Or they'll hire a freelance copy editor. Or something. But people who do not care enough to do a good job will, well, not care enough to do a good job. And it will show in the science.
Or maybe the science is good but the barrier of unreadability is preventing from realizing that. Either way, the paper needs to be rejected and completely redone.
The professional value of reviewing?
Reasons for reviewing, especially for new authors:
1. You learn a lot about writing (e.g., in terms of technical proficiency and personal “style” or “voice”) and what it takes to get published. You learn as much from reviewing poor papers (what not to do) as from excellent papers (what distinguishes them from the pack).
2. You learn what the standards are (in general and for specific journals) and what specific criteria are used by reviewers and editors.
3. The more journals you review for, the better you understand what subject matter is being sought (and you are less likely to commit the error of submitting inappropriate material to a journal).
4. You become known to editors, especially if you are a consistently good reviewer. Editors then tend to go out of their way to handle future submissions of your own work personally and ensure that they are evaluated fairly and quickly.
5. As you become known as a fair and thorough reviewer to editors, you are in a better position to challenge a negative review of your own work
Do the manuscripts that you review have the authors' names on them, or are they double-blind reviews? I always thought that to be fairer and more objective, it really should be double blind. You shouldn't know whose work it is you are reviewing because it could influence you (gee, you think???)
But every manuscript I've had to review did have the author names.
I once was asked to review a manuscript for a relatively high-impact specialty journal in my field. As usual the manuscript had the author names and instituions on it. The work was very sub-par (not thorough at all, and full of gaping holes and unbacked-up claims) and the actual writing itself was bad. Certainly it was not ready to be published in its present form. I was shocked to see that this crappy manuscript came from the lab of a very well known and highly influential Big Shot in our field as he was the last author on it (which in my field, the last author represents whose lab the work came from).
I wrote a very detailed, itemized, 10-page review and recommended "extensive major revisions required." I was wondering if I should have jsut said "do not publish" but I figured that as long as those major revisions I had recommended were done, then the work would be publishable so why not give the authors the chance.
Well...I completely forgot about this paper until a couple years later when I happened upon it. Guess what, it got published in that good specialty journal, but WITHOUT most of my required revisions. Why did I even bother?? Why did they even bother having it reviewed if they were gonna chuck aside the reviewers' comments, I'm guessing cos this came from a Big Shot lab where the PI "owns" the field. I was shocked that this particular journal went ahead and published that article in that form. No it was not Nature or Science, but in our specialty field it was still one of the more highly ranked journals.
"You can, absolutely, plagarize yourself."
Sorry, Anonymous, you are wrong here. What you were talking about was copyright violation, which is a different offense than plagiarism, and much less sinful in academia. Plagiarism is claiming someone else's work (specifically their words) as your own. Copying your own words is not plagiarism, though it may very well violate copyright if you have given some publisher the exclusive right to publish those sentences.
The self-plagiarism I've caught - and this is the worst case - was where an author copied an introduction word for word. Then, they republished data as if it were new. The previously published manuscript had not been sited once. Call it what you wish, but it shouldn't be double-published on my watch.
Thanks Kevin. Appreciate the clarification to angry anonymous. Copyright infringement is completely different than plagiarism. One can not self-plagiarize.
And also, comparing manuscripts to grant proposals, they are not copyrighted and as such you can completely copy and paste as much of your previous written work in a proposal as you want.
there must be something wrong with me because I almost NEVER find reviewing manuscripts to be personally fulfilling in any way. I would prefer to just read the final published version of a paper that I chose to read. The only reason I do it is out of obligation - since reviewing is a service, you have to participate in it since you are the recipient of it at other times.
Hi,
I totally agree with you. Reviews can be very important and it’s not an easy thing to do. It’s always a real challenge to stay objective and not get our personal opinions come into the balance. We always have the responsibility of being fair and take into consideration the work that has been done to write the manuscript.
Hi,
I totally agree with you. Reviews can be very important and it’s not an easy thing to do. It’s always a real challenge to stay objective and not get our personal opinions come into the balance. We always have the responsibility of being fair and take into consideration the work that has been done to write the manuscript.
Post a Comment