Monday, September 14, 2009

Scientific Overlap

It recently came to my attention that a certain scientist had submitted identical proposals to two different funding sources at the same time. Submitting "overlapping" proposals to the same funding agency is prohibited by some (but not all?) funding agencies.

Scattering the same proposal among different programs in the hopes that one (or more) of them will fund the proposal might seem like an efficient strategy for the PI who invested a lot of time in a proposal, but this practice is prohibited for good reasons. Given the time and effort involved by funding agency employees, panel members, and reviewers to deal with the large number of submitted proposals, a prohibition on overlap makes sense

NIH outlines in detail the consequences of the simultaneous submission of proposals with scientific overlap. I did a rather quick search of the NSF website to try to find a similar document, but didn't find anything. It seems like there must be a policy, though, so maybe I just missed it.

Rejecting proposals for reasons of overlap is straightforward if the proposals are identical or even "essentially identical" (in the words of the NIH document). It might become more difficult to judge overlap when the definition is extended to proposals containing "similar" research, another term used in the NIH document, but I suppose experienced program directors know it (overlap) when they see it.

In the case to which I referred in the first sentence of this post, there was no administrative mechanism by which the "overlap" could have been detected if a reviewer had not been become aware of both proposals. I suppose there is a high probability of a reviewer's discovering overlap in cases in which the research is highly specialized and both funding programs use peer review. The same reviewer(s) are likely to be sent both proposals.

In the first sentence of this post, I used the vague term "funding source" deliberately so that I could consider the ethics and consequences of the following situations:

- Identical proposals are sent to different programs of one funding agency that specifically prohibits submission of overlapping proposals. This is clearly wrong and both proposals would be rejected

- Identical proposals are sent to different programs of a funding agency that does not have specific rules about overlap. Reviewers might balk at this (if the overlap is detected) because it is annoying and seems unethical, but if there are no rules against it, what's to stop a dual proposal submission?

- Identical proposals are sent to two completely different funding agencies, one or both of which may or may not have rules about overlapping proposals within each agency. Again, reviewers might balk at this, but is it wrong?

I have never attempted submission of overlapping proposals and am not really interested in doing so, though I'm not sure why not. There is something appealing about the general concept of sending a proposal out into a broad funding universe to see if anyone would like to give me money for my research. This would reduce the time spent trying to figure out to which one of several possible programs a proposal will be sent, a decision that may involve making likely unfounded assumptions about the scientific preferences and sanity levels of the program officers and hoping that an interdisciplinary proposal won't fall through the cracks between programs.

It does seem wrong, though. And speaking as someone who just reviewed quite a few proposals, I certainly don't want to see multiple versions of any of these, even the ones that were Very Good*.

* In NSF-speak, Very Good is not as good as Excellent but much better than Good, which is not good at all, except when compared to the dreaded Fair and Poor.


Anonymous said...

There is another problem with overlapping proposals. I know of a case where someone submitted 2 overlapping proposals to different agencies (I don't know if that was against the rules). The problem was, both were funded! But then this person couldn't do the same work twice and both grant agencies got very annoyed that she failed to do the work promised under their remit. So overlapping submissions aren't necessarily good for the PI either.

EliRabett said...

You do have to disclose all pending proposals, and at least for some agencies the overlap between proposals.

In the past, for the agencies Eli is familiar with, if both were approved, the program managers would negotiate about which would fund which part

Angry Professor said...

This is interesting - I never knew NIH had such a prohibition. I read the guidelines at the link you provided, and it only covers programs within PHS. I have, several times, submitted nearly identical proposals to NSF and NIH, and there is a little form you fill out at the NSF to indicate that you have done so. I have also told the program directors about this, and they have said simply, "Just tell us how it works out."

My impression is that most program directors are happy to see identical proposals funded at another agency, because then they can throw very small amounts of money (a few thousand dollars for a trip or something) at the same project and then get to claim it in their own portfolio.

Prof-like Substance said...

NSF's policy is that "new" investigators, without previous federal funding as a PI, can submit their proposal to multiple programs. Otherwise, this is not allowed. There is nothing to prevent you from sending the same proposal to different agencies, however, but one would have to include the other agency under the Current & Pending section.

Personally, I try to modify similar grants that go out to different agencies so that if both were funded, the work would be complementary and not the same. In sending out identical proposals, one runs the risk of getting both funded but having to decline one, which seems like a waste of everyone's time.

Princess Fiona said...

Hi. I have a related question - how about if the PI and a postdoc write proposals on the same project (so in effect, essentially similar) - but one in a lot of detail (e.g., an RO1) and a much smaller version for a postdoc funding agency? is that considered unethical/annoying as well?

John Vidale said...

It's a poor idea to submit duplicate proposals, although I suppose it might be hard to resist trying it for more funding.

Between people on panels who see all the proposals, program directors who talk among themselves, and people who keep track of their colleagues, the chance of noticing duplicate proposals is not as small as one might think.

One might get away with the funding, but having it noticed afterwards is another risk. Scientific progress depends on being able to trust other scientists - one cannot verify every result oneself. So when I notice a non-kosher action such as a duplicate proposal, it undermines someone's hard-earned credibility.

Anonymous said...

I'm struggling with this myself - as a new investigator, waiting the 6-8 months for (likely) rejection to then submit a new version of the proposal seems incredibly inefficient. Of course, if you happen to win the lottery and get the first one, then you are stuck telling the 2nd place what you did (which is not appealing either).

And, who defines 'similar'? I know a lot of PI's that you could take their research proposals and just exchange protein A for protein B and it's the same experiments, etc. Or experiment A for experiment B on the same protein. Yet, these senior people are well funded, so how are we newbies supposed to interpret it?

I've asked multiple colleagues and former mentors for their thoughts on this - and there is no consensus.

Kevin said...

Anoynmous waiting the 6-8 months.
You are fortunate if you are waiting so little---I've had to nudge program officers after over a year. One rejection notice took 18 months (these are the same program officers who complain if you are one day late with your annual report).

When the funding probability has dropped below 10%, waiting for one agency to reject a proposal before sending it to another means that your average time to get funding on a proposal is between five and ten years.

Pagan Topologist said...

I have known several people who have submitted identical proposals to, say NSF, and DOD sources. When both were funded, they were obligated to turn one down.

Anonymous said...

Yeah, in my field it seems like common practice to submit to two agencies... especially if only one of them is NSF/NIH and the other is a private foundation or other organization. For example, one should be allowed to submit the same general project to NSF/NIH and also to an "individual" award. Ex. an award for new investigator, or gender/race diversity foundations, etc. After all, for these rewards it is really YOU as the PI that's being funded. The research you propose is more an example of what you can do than anything.

drdrA said...

IT was my understanding that this doesn't apply to new investigators. New investigators may submit overlapping proposals to multiple agencies. Is that correct?

AcademicLurker said...

To my knowledge, there is nothing wring with submitting overlapping proposals to different agencies, you're just not allowed to accept more than one if they all get funded.

Multiple overlapping proposals to the same agency is another matter, and is generally not allowed.

bob said...

If you submit two different grants to two different agencies, then reviewers will spend some amount of time reviewing them. If you submit the same grants to two different agencies then either the same is true, or there is reviewer overlap, in which case it would take less time for the second reading. I don't see how this wastes anyone's time unless the proposal is clearly inappropriate for one of the agencies (it takes someone's time, but it doesn't waste it).

Why is this unethical, provided one grant is turned down in the event of double-funding?

What about postdoc fellowships? Is it unethical to submit the same research proposal as part of your applications to multiple agencies? If not, why is this situation importantly different than the situation with grants?

Anonymous said...

When the funding probability has dropped below 10%, waiting for one agency to reject a proposal before sending it to another means that your average time to get funding on a proposal is between five and ten years.

Exactly. I've been a "new investigator" for a long time now because, due to advice like this blog post that I was given when I started my career, I would submit a proposal and wait months to get rejected before resubmitting elsewhere (only to get rejected again). My time is running out now. My conclusion is that those who have the luxury of not having to worry about their livelihoods coming to an end due to lack of grant funding, can take the high road and get all preachy about the ethics of wasting other people's time on duplicate grant submissions.

John Vidale said...

Some comments with which I disagree:

Angry Prof: "most program directors are happy to see identical proposals funded at another agency, because then they can throw very small amounts of money"

Agencies want to take credit for work, not share it. Sometimes several programs at NSF share funding, but after reviewing a single proposal by multiple panels. Joint funding between agencies would also double the monitoring, paperwork, and reporting requirements.

Anon@12:53: "After all, for these rewards it is really YOU as the PI that's being funded. The research you propose is more an example of what you can do than anything."

I don't think so.

Bob: "I don't see how this wastes anyone's time unless the proposal is clearly inappropriate for one of the agencies."

The problem is if everyone takes all their proposals and sends them everywhere, which they would if this practice were encouraged. The NSF panel I was on a few years ago would have had 200 rather than 100 proposals to evaluate every six months, plus the headache of trying to figure out what all the other panels had done with those same 200 proposals. Not to mention the lunacy of having to solicit multiple rounds of outside reviews, doubling everyone's review loads. All to distribute no more money.

One can spot duplicity via the current & pending. It would be foolhardy to submit duplicate proposals that neglect to mention each on the other C&P.

Anonymous said...

Another limitation (for the PI) when submitted duplicate proposals, is that reviews are not independent. Submitting twice is not like buying two lottery tickets rather than one, because proposal quality does matter. Submitting a mediocre proposal twice will not get it funded, and submitting an excellent proposal twice is no use if you have to turn down half the funding. I always try to have 2 things under review at a time, but two different things, say different approaches to the same fundamental question. Then when I get the reviews back I get feedback on different ways of tackling the problem I care about, and hopefully one gets funded. If a PI submits the same grant twice, I would consider them not only unethical but also foolish.

Angry Professor said...

@John V. - That's not what the program directors on my panels have indicated. In fact, the "throw very small amounts of money" statement is a direct quote from an NSF program director on whose panel I served in regards to a proposal I was primary on which had just received funding from NIH.

(I tried to fit more prepositions into that last sentence but ran out of steam.)

John Vidale said...

@angry prof

then I guess my remarks are not universal. I find it surprising, but try not to question facts.

Foreign and Female in Science said...

I did once submit a proposal to the same funding agency under two programs after discussing the issue with both program directors. The problem is that in year 1 we submitted to program A who said oh this work fits a bit in A but fits better in B. The next year we applied to program B to get the response of oh this work fits a bit in B but fits better in A...

EliRabett said...

Let me jump on John V here too. There are a lot of program directors at NSF who will take a proposal they want to fund, but which fell a bit short, and go hunt funding in other NSF programs. They don't have to go through a joint panel.

Sometimes they will look for joint funding for a higher rated proposal that they know they can get a buy in from a another program in order to fund a lower rated one they want to support.

Anonymous said...

The NIH link looks to refer to resubmissions of rejected applications or sending the same app to multiple branches of NIH.

Is category #3 (applying to multiple independent agencies for the same project) really an ethical issue? I'm a graduate student and we are always encouraged to apply to a wide variety of sources. Considering the small percentage of dissertation research apps which receive funding by e.g. NSF (to name just one of many), waiting out the 6+ months for rejection from one source before applying elsewhere would be academically suicidal to most PhD students.

In the happy event that multiple sources approve funding, it's the grantee's responsibility to explain how (and how much of) the money from each source will be used for the project. The unethical part, IMO, would be the failure to do that, not the applications themselves.

Anonymous said...

Hi, what happens if an agengy helps another agency to complete a grant application and both agencies end up submitting same parragraphs in the [rpgram description and other sections?

Anonymous said...

If you apply for a job or admission to a university, how many applications you submit? Applying to different places for a job would definitely increases the review burden. But would this be considered unethical? How different is this as opposed to the proposal submission?