Wednesday, May 30, 2007

Pay-Per-Review

Now that the academic year is over, it seems that reviewers and editors (including FSP) are getting caught up with their tasks. Yesterday I wrote about some reviews I received. Soon after, I got an email from someone who was reviewing another manuscript of mine.

I hate it when this person reviews my manuscripts because he always writes to tell me that he is doing the review, has some possible problems with my interpretations, suggests that we 'discuss' it, and then he asks me for a favor. Once he asked me to pay for his travel to the U.S. This time he asked me to do some analyses for him. Sometimes I request that this person not review my manuscripts, and I give a vague reason like "conflict of interest", though I didn't do that with this manuscript.

This particular unethical reviewer I've just described is the only one I've encountered who does this. I think it is a rare situation, but it raises the general and more relevant issue of how and whether to suggest that someone not review your manuscript or proposal.

Most journals (and NSF) give authors an option of listing "non-preferred reviewers". I don't like to use this option if at all possible. In an ideal world, we are all objective and will focus only on the Science. But back in the real world, there are people who should not review certain (or any) papers/proposals, and sometimes editors and program directors don't know who these people are without some information. If a reason must be given, "conflict of interest" is vague but professional. Somehow I have never felt comfortable writing things like "He harassed my postdoc" or "He asked me for money the last time he reviewed one of my papers".

If I don't have direct experience with someone's giving me an unfair review, I don't list them as a non-preferred reviewer. I would never use a suspicion that someone might be unfair as a reason to request that they not review something of mine. Delusional though it may be, it's best to give someone the benefit of the doubt unless there is evidence to the contrary.

As an editor, when I see non-preferred reviewers listed, sometimes I know the situation and can evaluate whether the list is valid, but in other cases I wonder why. I typically respect the wishes of the authors unless their list is long and includes everyone with any expertise in their subject. In these cases, the list of preferred reviewers is typically loaded with the names of the authors' friends and former co-authors. [memo to authors: Don't do this.]

I am sometimes asked whether it "looks bad" if you list non-preferred reviewers, as if you are afraid of criticism and not confident about the excellence of your paper. The advice I give is basically what I described above: if you have a concrete reason for requesting that someone be excluded from reviewing your work, you should do it. If you just have a suspicion, don't do it.

15 comments:

  1. Why not forward the inappropriate e-mail request to the journal's editor? Problem solved, right? You can't be the only one with this problem.

    ReplyDelete
  2. Hmm...

    I think it is legitimate to ask that certain people not review your papers. Usually these folks are known in the community for being difficult and those clued in will not wonder why you have done this.

    I've had two problem children in the past. One is now dead, so that takes care of that. The other is still out there. It was usually pretty easy to tell when one of these folks was doing the review, because there were always requests for numerous citations to their work. Usually I would - if possible - use their own work in support of my arguments and to counter their criticism.

    But to put the squeeze on you while someone has a paper of yours under review - Ick! Maybe you should blow the whistle to the editor?

    Mr. B.

    ReplyDelete
  3. Wow, that sounds really awful. How about forwarding the email unethical reviewer sent you to the editor? This person has no business reviewing any manuscript at all. It is quite confrontational, but I bet you are not the only one who is being targeted by him. I wonder if he is writing these kinds of emails to the male professors whose manuscripts he gets to review as well, or if he dares to contact only women professors.

    On the other hand, I can understand why you would not want to tell the editor about it - it is not a pleasant situation. You mentioned you are an editor yourself and I am really curious - if one of the reviewers for your journal was doing similar things, as an editor, would you want to know about it? Or would you prefer the authors deal with such a reviewer by themselves and never mentioned anything to you?

    ReplyDelete
  4. The first time this reviewer wrote informing me of his imminent review, the results of which were apparently open for discussion, and asked for money, I forwarded the email to the editor. The editor did not reply, the review was not particularly positive, and the paper was published.

    As an editor, I would want to know if a reviewer were doing this to an author.

    ReplyDelete
  5. I have a protective streak so if this happened to me, I would dob the reviewer in. My rationale is that although I might personally be able to refuse, there are others who might be less assertive, less experienced and less well-supported than me. If these people get the same requests as you do, then they might become victims.

    This has two effects:
    1. The victim is out of pocket and perhaps disillusioned by the peer review process.

    2. The reviewer says, 'Fantastic, this gig works!' and he continues onto the next person.

    What if someone has already made a complaint against him and the journal editors won't act on it because it is an isolated complaint?

    ReplyDelete
  6. This review has to be in violation of some ethics code somewhere. He should be kicked out of whatever professional society he belongs to. I hope you report him again.

    ReplyDelete
  7. Yuck. I'm well past the belief that life is fair, but situations like this still leave a bad taste in my mouth. I think he deserves a kick in the shins with your pointiest shoes.

    ReplyDelete
  8. I definitely think you should report this to the editor - you don't want this person in the referee pool. This person ought to know better, too - asking for an implied quid pro quo is waaaay over the line ethically.

    ReplyDelete
  9. See, I think people like this should be blacklisted because otherwise they will abuse people who don't have enough power to do anything about it.

    And the editor who ignored you? Should be made to see reason. Long boring ethics lectures/ mandatory training sessions might do the trick.

    ReplyDelete
  10. Having served as a reviewing/associate editor as well as co-editor-in-chief for a few journals, I think it is imperative to point out this kind of behavior to the editor handling the paper. The behavior you described clearly shows that this reviewer is not exactly being objective (among other things).

    ReplyDelete
  11. You really ought to pass this along to the journal's editors--particularly since it sounds as if this is a habit for this reviewer.

    This is unvarnished corruption, and overlooking it does a disservice to other reviewers, authors and science in general. Peer review is one of the strongest things that science has going for it, and this sort of behavior will undermine it very quickly.

    ReplyDelete
  12. Is the review process in your discipline double-blind to the peer reviewers (I know that the editors have to know who submitted the paper)? And, if it is, was the person able to guess that you are the author because the work is so closely related to yours? I am sorry if these are dumb questions.

    ReplyDelete
  13. All manuscripts are signed, but it varies from journal to journal whether reviews and editorial comments are signed. This person could have chosen to be anonymous, but for his own reasons he chose to email me 'informally' to tell me he was reviewing the paper. This is not typical in general, but is a habit for this particular person.

    ReplyDelete
  14. Wow! I have not run across this in my field (atmospheric sciences) yet. Absolutely I would forward the email to the editor. He is in violation of the entire peer review process and I think it's our responsibility (as scientists beholden to the peer review process) to keep that process as true as possible. I would be stunned to have this kind of request sent to me. Shows I'm living in some kind of naive world, I guess. :-)

    BTW, I'm very much enjoying your blog. I'm a woman scientist (at a government agency), trying to balance research and motherhood. It's a fantastic mix, in my opinion...

    I enjoy reading the differences in your world (academia) versus mine (government). Not so very different, but different nontheless.

    Jen

    ReplyDelete
  15. I'm an editor at a Big Journal, and I would absolutely want to know if one of our referees was breaking the process in such a way. We would never use him/her again. That's just wrong.

    [in fact, it's so wrong that our Big Journal has a news staff and I'm sure they'd like to hear about it too. I'm just saying, that's how wrong it is.]

    And, for future reference, absolutely we want people to list up to 4 scientists who should not review their paper. We certainly do not want to send your paper to a competitor any more than you want us to do that.

    It is helpful to us if authors list up to 4 specific people and say "we know they have competing work" or something similar. It is NOT helpful for authors to say things like "please don't send this to anyone in San Francisco Bay area" or "to anyone working on chemical biology." You may be laughing but it does happen.

    ReplyDelete