A manuscript that was submitted earlier this summer was recently returned from review, and lucky for us there were many positive comments about the work. There were also some very negative comments from one particular reviewer.
This reviewer, who indicated his identity in the review and recommended that we cite more ofpapers his , didn't really care about the cosmic implications of our work. He focused intensely on one particular method. He wrote in detail about the assumptions of this method, giving us a little tutorial about the method we used. Apparently his research is an excellent example of the appropriate uses of this method. Apparently, our research is deeply flawed.
We disagreed with every one of his comments, and wrote a detailed response letter to the editor, who accepted the paper.
By complete coincidence, as part of an entirely different project, some colleagues and I found ourselves revisiting the research of this reviewer, including the specific subject of the paper he held up as an awesome example of the appropriate use of the method in question. We had not intended to check into his research so closely, and the timing of his review relative to our delving into his research was unintentional.
Nevertheless, the term 'house of cards' kept coming to mind as we delved. Many of the issues that the reviewer raised as problems with our application of the method were present at similar or more serious levels in his own work.
I think that his work still has a lot of merit. The study he did was important and the resulting publication is interesting. Nevertheless, it was extremely hypocritical of him to take our manuscript apart as he did, and hold us to a higher (impossible) level of purity -- a level he himself did not even approach in his own work.
I don't know this reviewer well, so I have no insight into the psychological aspects of his review. Does he really think that his paper lacks the flaws he sees in ours or does he know this but prefers to pretend otherwise?
Is it ever reasonable for a reviewer to hold others to a higher standard?
In fact, in some cases it is. For example, early studies to explore a new method or a question may be a bit rough; later work should improve on this if possible.
That wasn't the case with the review in question. In this review, the researcher was being pedantic and hypocritical. We were fortunate to have an editor who weighed the reviews, positive and negative, and decided that the negative comments were not reasonable. Hooray for thoughtful editors.
14 years ago
7 comments:
I do not think that in general, if you have done stupid stuff in your own research, you should be obliged to accept it in that of others. Quite the opposite, in fact.
(Not that the reviewer here seemed to think his work was full of mistakes. But simply to answer your question of higher standards in general.)
Hooray for you!
I've run into this with my advisors. When I confronted them about their hypocrisy and how my work was held to a completely different standard from their other recent postdocs' work, they just got angry. When my data showed the house of cards for what it was, a house of cards, they just got angrier.
I had been taught that you could always win if you had enough data. And then I found out that sometimes having more data than anyone has ever seen before just pisses people off even more!
I often wonder about these things, if the PI reviewer always had doubts about their own trainees' work, but would have rather that you, as an outsider publishing different work, could hopefully reinforce the original findings and allay their fears?
Or, as you say, the psychology of this person may be such that the potential flaws, easier for him to see in your work than in his own, are now impossible to ignore?
I mean, what if it is a house of cards? And he's afraid that more people standing on it will just make it collapse?
Or, maybe just pissed off that you didn't cite him often enough.
Who cares! Yesterdays topic was much more interesting than this.
Maybe, the reviewer got exactly the same comments on his papers, and justified his experiments to the editor the same way as you?
There's some old saying like, "The traits that annoy you the most in other people are the ones you yourself possess." I find this sometimes explains hypocritical people.
Glad to hear your editor was thoughtful!
People have a tendency to be blind to their own faults. No new parent thinks their own baby is anything less than the most beautiful thing in the world.
I have had the same experience. I know the person who commented on my work and I know that he strives to be unbiased, yet his bias is very obvious to me. However, I like this particular person, so I'd like to 'write-it off' as human nature..But that doesn't help me feel that the criticism is unfair.
Post a Comment