Wednesday, November 23, 2011

Take Down

For some reason -- or, possibly, for no reason -- the manuscripts I have been sent to review lately have been of a certain type: the type of paper that has an interesting dataset or idea that would make an interesting focus for the paper, but the authors instead choose to spin the paper as an attack on someone else's idea/s. That is, the papers seem to be aimed mostly at criticizing someone else's published work and proposing something different instead, even if that something:
  • isn't all that different from the original idea they are trashing;
  • isn't nearly as interesting as what they could focus on instead (says me); and/or
  • isn't supported by their own data.
These cases are different from those in which there is clear evidence that a published idea or dataset is wrong and that wrong needs righting.

I have disproved the work of others before (including one of my early grad advisors, who hated me for it), but I don't derive any particular pleasure from it -- at least, not on a personal level. As a scientist, I can appreciate the sweeping away of an old, bad idea and replacing it with a beautiful new idea that explains things, and I feel satisfaction and pride if it's my research that does this or helps with this process, but I don't enjoy an attack for the sake of an attack.

For that reason, I find it hard to understand when someone else chooses -- and it is a choice -- to go that route when there really isn't much of a point to doing so. That is, when some researchers try very hard to find something, anything, no matter how unimportant, to tear down, and focus on that so much that the rest of their work is subsumed.

Yes, I know that sometimes there is personal animosity involved, but in the cases I recently encountered, the people involved actually get along quite well, at least as far as I know. The attacks in the manuscripts under review are not vicious or personal; they seem almost formulaic, as if the primary authors were told that this was the best way to write a paper that will be noticed (cited) or that they should be sure to distinguish their work from that of others.

In fact, the primary authors of these manuscripts have all been PhD students or postdocs. Maybe they are trying to make a splash? I think the papers could be really nice contributions if the focus were more on the substance of the research, not on some far-fetched or unfounded undermining of a minor point in some other publication. 

Probably my reviews will sound patronizing to the authors, and of course they and the editors can ignore my advice, but I think it is a mistake to go negative when there is nothing to be gained by doing so.

If you have gotten advice, particularly as an early-career researcher, about the best way to set up a paper, did that advice include anything about this issue? For example: framing a paper as an argument or attack is a good way to write a paper (no matter what), this is a bad way to write a paper, only do this if you think you are totally justified and it is an important issue etc.?


16 comments:

Anonymous said...

Talk about a strangely timely article! I just met with my adviser today after immersing myself in a couple of papers that I thought made bold, (in my head) unfounded conclusions. We both know the author of the papers very well, and it turns out that my adviser holds the author in high regard, even if the conclusions aren't always 100% correct.

In my case, it's hard being an older graduate student. You know that you want to contribute in some way, but don't yet know how to tell which ideas are the most interesting from an outside perspective. It's a lot easier to think, "I can easily refute the conclusions of this attention-grabbing paper" than it is to come up with truly novel ideas of your own.

Kris said...

My partner (a postdoc) had recently completed a paper draft and was told to explicitly rewrite it in the style mentioned in order to get it into a higher impacting journal (even though the scientific argument was the same). The rationale seemed to be something about the paper then being more scintillating and thus more attractive to editors because it looks as though it creates a controversy, and thus more fertile field for citations. He wasn't too happy about this, but then, it's not his call, and apparently is a strategy that works. It's not an approach I was aware of, but perhaps why I don't get papers in such high-impacting journals ...

Comrade PhysioProf said...

Interestingly, a paper was recently published in my field that was couched exactly as you describe in relation to a paper I published about a decade ago: "We have now demonstrated that the model proposed by Comrade PhysioProf et al cannot be correct, and their observations are likely explained not by the cool thing that they claimed, but rather by these other things that are really not very interesting at all."

The basis for the claim in this new paper was the argument that they had performed the exact same experimental manipulation we had in the past, only in a more refined and controlled way, and obtained a different result. The problem with this paper, sadly for its authors, is that the experimental manipulation they had performed was almost certainly *not* the exact same one we had in the past. They failed to make the simple and obvious measurement that would have told them whether it was not, and this near-certain difference in experimental manipulation exactly explains why they obtained a different result than we did.

It is too bad for these authors that someone like you didn't review their manuscript and prevent them from making complete fools of themselves in the scientific literature.

Anonymous said...

When I was a grad student I saw myself much more in competition with other researchers than I do now. My view then was "my ideas are the best and you're all going to know it"; my view now is closer to "let's all sit around the fire and sing kumbaya together". (And no, my politics have not changed at all in the same timespan.)

I certainly never got any advice to write as if I was competing with everyone else. In fact, my advisor probably tried to curb that tendency. I just ignored him.

Anonymous said...

As someone on the other side (work has been the focus of an attack), it is very frustrating. Particularly when no advance notice (by authors OR editor) of said attack is given, and the paper is written in a VERY personal manner.

And I'm probably biased, but, yes, it does mask the science in the paper (somewhat) when the majority of the sentences contain phrases like "As shown in figure 1, XXX was wrong." with little to no discussion of the data in Figure 1 - which was interesting and did not show I was wrong.

As a result, their paper hasn't been cited by anyone except them. Which, again, is a shame, as the science/data is interesting, but the presentation/discussion is completely lacking.

The first author - a senior scientist (who should have known better).

GMP said...

The issue you mentioned -- needless attacks on the work of others, especially over minutiae -- is fairly common among young researchers, perhaps even more so when they give presentations than when they write papers. It's understandable, as they may not have a full grasp of the the big picture and are overcompensating
for the insecurity over the significance of the work.
I have certainly encountered this situation with my students, and that's a good teaching opportunity -- sit them down and tell them about an appropriate way to frame criticism, distinguish critical from superfluous points, and talk about the broader context for the work, what the important open questions are and where exactly your joint contribution falls.

Having said that, I think that it's ultimately the senior author's fault to have allowed papers with needless petty attacks to be submitted. The senior author should know better and is, in my opinion, ultimately responsible for the content of the paper, and that includes cases where something is wrong or inappropriate. Junior researchers need to be trained in the best practices when writing papers; these skills are very hard to develop without the advisor's direct input.

Anonymous said...

I've seen one paper like this recently, and in that case, the approach seemed to be driven by an almost philosophical disagreement, to emphasize that (even though the results are not that different), *in principle* the correct way to do calculation X is to use method Y. Philosophical problems may be more appropriate in pure theory papers like this, though.

The "takedown" papers I've written (I'm a grad student in physics) have been in response to papers that were initially very influential on me, but that I came to realize had major flaws. I'm not sure I'd bother writing a paper like that if I didn't care strongly about the original work.

Alex said...

I can think of one journal (which shall go un-named) that is flashy and will allow in flashy crap. I'm doing something that enables a benchmarking of different techniques for a field where most people are currently using home-brew tools. For our first set of tests we chose 2 widely-cited techniques, one of them published in that flashy place, and a technique that we developed ourselves. Suffice it to say that it will be a challenge to write the paper without saying "This thing from the flashy journal is crap."

Anonymous said...

I don't understand the attack mode either. I recently reviewed one of these from a postdoc I am considering hiring. In it, he criticized some of my previous work, although he seems enthusiastic about working with me on a related project. Is this a red flag? Maybe, but I think I will hire him anyway, after a conversation with him about the situation. Maybe he got some bad advice from his PhD advisor? It wasn't a vicious attack, but it was rather pointless and not about a major issue. It was mostly just strange.

thebuggeek said...

As a grad student, the message I often hear (explicitly or implicitly) is that your PhD work needs to be ground-breaking and earth-shattering, and paradigm-altering. It can't just be really cool work - it has to refute or alter what has been done before, so why not go big and really put the screws to the "old" concept/idea/theory/experiment. I suspect that's why you're seeing this trend with PhD/postdoc work.

Anonymous said...

I did my PhD in Atomic Physics. My grad adviser (a senior member of the field. He had just experienced an unpleasant talk as described by this post.) once told me and a couple of post-docs that one's work should never be "They were wrong, and this work shows you why." Even when a previous work has incorrect statements, one should say "Previous work did this. Here, we present our cool stuff and show you what we add to the topic. This is what we get." Bringing attention to the negative parts of other group's work is usually unnecessary.

Dan said...

My first paper when I was a grad student was somewhat along these lines, though it wasn't too personal in the attack, I hope.

As a new grad student, I read a whole bunch of papers that cited a particular example of one important phenomenon. Digging through the literature, I found that there was actually very little existing evidence that this phenomenon applied to the example, and I provided more data to demonstrate that this example was not a case of the phenomenon.

The response of the field was a giant shrug. I don't see that example cited anymore, but there are lots of other well supported examples of the phenomenon, and people have gone on to cite those.

I may have written the paper the way that I did because, as GMP points out, I didn't have the whole picture of the literature- but I don't think that is all bad. When you are new to a field, you take individual papers very seriously, because they are all you have. When there is a minor error, you want to correct it- because it doesn't seem to minor to you. But I think that this attitude can be important in setting the record strait. It might seem like a superfluous point to an experienced researcher, but if we want the literature to be self-correcting, then junior folks with narrow vision are doing us a favor. (They may just need to get rid of the personal attacks.)

Anonymous said...

I'm in a similar situation as anon 06:17, but on the other side of the table. I recently criticized the work of someone I am hoping will hire me as a postdoc.
From my point of view, the criticism is very mild. However, during the review process I felt some pressure from the referee to make criticism to other papers stronger/more-visible, particularly on this person's work because we use somewhat similar methods. I found this a bit strange, but complied anyway. I really hope the target of my criticism overlooks this minor situation and still hires me.

Anonymous said...

Interesting comments. When I was a grad student, I was often advised to submit very neutral, boring papers. Even when we had data that clearly contradicted big prof X, we would write a bland data paper that didn't attack him. Attack papers always got rejected. Once the data was published, the killer attacks could come in a review article.

Anonymous said...

Interesting timing, because I just encountered this on a semi-personal level recently (the attack was on my advisor's work). The unpleasant part of it was I have met with the attacker, and he does seem to have something personal against my advisor. The attack is not very well justified, and it seems like just a gratuitous addition to a perfectly fine paper (of the attacker), which is just unnecessary unpleasantness.

Anonymous said...

I recently received such a paper to referee. The paper also included a minuscule amount of new analysis. In my review I said that the nitpicky part of the paper was nitpicky, inappropriate, and made the authors look defensive. They then split the paper into two, resubmitting the nitpicky one separately and I received only the expanded new analysis. It'll be interesting to see what happens to the nitpicky papaer now. Such papers are fairly unusual in my field, and I hope it stays that way.