Monday, April 21, 2008

Vex Ed

Last week I chatted with the editor of a science journal -- a different journal from the one for which I do some editing -- and we compared notes on the more vexing aspects of the job.

One thing we talked about was: Who are the most disgruntled people we deal with as editors?

The answer is not the most obvious one: the authors of rejected papers. No, the crankiest people -- or perhaps the people who hesitate least about sharing their anger -- are those who did a review, recommended rejection or major revisions, and then are angry when they see the paper published, or published without all of their suggested revisions accomplished.

This makes sense. Although some authors of rejected papers do indeed indicate their displeasure in a less than polite way, most do not, perhaps because they hope to publish in that same journal at some later date, or perhaps because they recognize the awesome wisdom of the Editor.

Reviewers, however, have performed a free service for a journal, and may have devoted quite a lot of time to a review. To have that review seemingly ignored can be infuriating. I have felt that way as a reviewer, though I can't imagine angrily confronting an editor about it.

My editor-colleague has experienced this angry jilted-reviewer situation a lot more than I have, and I was curious about that. He has been an editor longer than I have, and that may be the explanation. Nevertheless, I couldn't help wondering whether he ignores or disagrees with reviewer recommendations more often than I do. To go against a substantive review and recommendation to reject by a reviewer, I need to be very sure that the reviewer criticisms are unfounded or excessively negative.

It is seldom the case that highly negative reviews are without basis, although it does occur. Some reviewers hate every manuscript they review (but nevertheless provide valuable comments, so editors continue to solicit their reviews); some reviewers are less objective than one ideally hopes a reviewer will be about particular topics; and in some cases a reviewer misunderstands a manuscript (e.g. owing to poor writing by the author or careless reading by the reviewer).

In some cases, one reviewer hates a manuscript but another thinks it is excellent and fascinating. In those situations, at least one of the reviewers is going to be annoyed no matter what the final decision is. That's not what matters, though. An editor has to take every substantive review seriously, read the ms and reviews carefully, and make a decision, possibly after seeking additional reviews.

I just looked at my editorial statistics for 2007, and my acceptance rate is in line with that of a peer group of editors. There may be some vexed reviewers (and authors) out there, but for the most part I think my little corner of the peer review ecosystem is functioning in a fair and efficient way.

10 comments:

The Mad Chemist said...

Most people who review a certain paper are chosen as referees because they are familiar with work in that area (usually because they have done actual research in the same area), correct?

There were some rumblings a few years back about a reviewer (in chemistry) who gave negative reviews to a paper because if published, the paper would scoop his research before he could publish it. The paper was rejected and he published his work in the same journal a few weeks later.

I never heard what happened to that researcher but I was wondering what safeguards are in place to prevent such acts?

Anonymous said...

Huh. I would've guessed the most difficult group would be the (accepted?) authors. Every time I get a review, my first response is outrage - how dare they impugn my brilliant work! :-)

I calm down within a few hours and deal with it professionally. After the fact, my reaction can be seen as quite funny.

Ms.PhD said...

I don't mind so much when I've been asked to review a paper. I got to say my piece, and then a decision was made, presumably taking my comments into account.

Today I was looking over an old paper on which I'm a co-author, where my opinion was not solicited prior to submission or publication. I saw a very different version of the paper than the one that was accepted.

If I had been the reviewer at the journal, I would have rejected it. Maybe this is why they didn't give me a copy of the version they submitted!

I'm much more disgruntled about that.

re: the mad chemist's comment, I'm not aware of ANY safeguards for things like this. My understanding is that it happens all the time, at least in my field. I think it might have happened to me relatively recently.

Hard to know for sure. Someone who was requested as a reviewer (at my advisor's suggestion) appeared as an author on a related paper that came out shortly after mine was rejected.

Hard to know if this person was actually the reviewer and was actually objective (or even trying to be).

One can only wonder. I've never met this person or had the opportunity to ask anyone who would know (or be willing to tell me!).

Unbalanced Reaction said...

Hmm...I'd never thought of the manuscript acceptance/rejection funness from THAT perspective before. Makes me feel (very teeny-tiny, itsy-bitsy) empathetic for the gnarly editor that rejected my paper based on two awesome reviews and two requests for minor reviews. Or not. ;)

Anonymous said...

@The mad chemist...

The most useful topic we discussed in the required ethics training for my pHd program was issues such as this.

What we were told was that if your research is "too close," such that you are working on research that would be scooped by the paper you are reviewing, you must recuse yourself from review. Other reasons to recuse yourself are if you are a friend or collaborator with the person whose paper you are reviewing, or if you are a known RIVAL in the field.

Most journals have channels for appealing a rejection decision but a lot of it is left up to the editor of the journal. I think the only "safeguard" is the reputation of the reviewer and the editor. I could imagine lawsuits might be involved??

Anonymous said...

My husband and I were just talking about this very subject the other day. We were wondering whether it would be inappropriate to complain to a journal editor about how our reviews are handled. I take it from your post, FSP, that the answer is "yes".

Still, it's hard not to feel as though my opinions are thought of as less-than-worthy, and my comments thought of as insignificant. If you don't want my opinions, don't waste my time asking for them!

It is extremely irritating to have my reviews essentially ignored. At the very least, an explanation of why they were ignored would be nice. At least that would force the editor to prove that my review was actually considered.

Female Science Professor said...

I certainly don't mind if a reviewer has a polite conversation with me about a manuscript that they reviewed.

Notorious Ph.D. said...

I've been dealing with this situation from the other side: trying to get an article published in Journal of Excellent Studies, and going through revise-and-resubmit umpteen times. I have one reader (out of three) who keeps coming back with new things to dislike about my paper. The editorial board seems to want it, and have even offered me (in light of my suffering, perhaps) an in-house review this time around. But I'm now worried that getting this paper published over this one reader's strenuous objections might be as bad or worse than not getting it published at all.

My question: will the jilted reviewer be madder at the journal editors, or at me?

Anonymous said...

Reading your blog comments related to the editorial process, prompted a question. How does an associate professor with plenty of reviews under her belt maximize her chances of an invitation to become an associate editor. How does this process work?

Female Science Professor said...

If you know any editors you can ask or if you find an opportunity to talk to one or more at a conference, you could express interest in playing a more active role in the review process. If you don't know the editors well, you might want to get a sense for the situation in an exploratory way first. I say this in part because my fellow editors and I recently received an e-mail from someone expressing interest in being on our editorial board at some point in the future. I thought "Great, let's keep this person in mind", but one of my co-editors thought the person was being pushy and obnoxious. You can't worry about that too much, but maybe it's best just to explore the options first.