Thursday, May 05, 2011

Possibly Well Written

How important is writing quality for the success of a grant proposal? I don't think you can get a grant proposal funded just because it is well written, but of course it helps if you can explain clearly what you want to do, why you want to do it, and how you are going to do it.

For reviewers, it's good if a proposal is written well enough that it isn't annoying to read. A proposal filled with typos and 2-page long paragraphs consisting of a multitude of unrelated points is a chore to read and makes you wonder whether the poor writing reflects something significant about how the research would be done, even if you know that there may or may not be any correlation.

I have found that it is fairly common for reviewers of my proposals to comment on the writing of the proposal. When I am reviewing a proposal, however, I tend to comment only if the proposal is extraordinarily difficult to read owing to writing problems; that is, the writing is so bad that I am not really sure what the PIs are trying to propose.

If a proposal is well written or moderately well written or not especially well written but I can still figure out what is going on, I don't tend to comment on the writing unless I can think of some specific constructive comment that might be helpful (e.g., for a new investigator). I comment on the writing of manuscripts submitted to journals, but what is relevant in a proposal review is different from what is relevant in a manuscript review.

I was thinking about this because I recently read the reviews of one of my proposals, and I noticed that 3 of 6 reviewers commented on the writing of the proposal:

The proposal is well written..

This is a well written and prepared proposal..

The proposal is not particularly well written..

Since neither of the positive comments about the writing said that the proposal was very well written, and the negative comment used the somewhat feeble description "not particularly", I am going to conclude that the writing was OK -- not great, but good enough. From the rest of the comments in those reviews, it seems that the first two liked the overall proposal anyway, and the third one found lots of little things to criticize -- nothing fatal (the grant was funded), but the reviewer clearly had some other ideas about how the research should be done. In that case, "not particularly well written" might mean "I would have written this proposal in a different way".

In another recent proposal that also led to a grant, two reviewers commented on the writing:

This proposal is very well written..

This proposal is well written..

OK, that's nice, but not relevant unless the reviewers took this into account in their overall ranking. There's no way to know if they did; see below for question about this.

But first, in the interests of bloggy pseudo-research, I need to do something unpleasant and re-read the reviews of a proposal that did not lead to a grant.. a rejected proposal. What, if anything, did reviewers say about the writing in my failed proposal?

Only one out of 6 reviewers mentioned anything about writing:

The proposal is well organized and well written..

Of course it is not possible to conclude anything from these few examples. The reviewers were likely different for each proposal, and who knows whether these reviewers make a habit of commenting on the writing.

I nevertheless stand by my rather obvious hypothesis, expressed in the first sentence of this post, that good writing won't get you a grant (if the proposed research isn't Excellent).

This leads me to some questions for readers who review proposals:

Do you factor how well written a proposal is (or isn't) into your overall proposal rating?

Do you typically mention the writing in your review? (always, never, only if the writing is notably good/bad?)

Do you think that good/bad writing could tip the scale for a proposal to be funded/not funded if the proposal is right on the very edge of the funding zone?


mOOm said...

The writing of the proposal could reflect on the quality of articles that will come out of it. That might be a scientifically relevant thing to pay attention to.

Anonymous said...

i feel it is difficult to separate out opinions of research from those of writing; certainly a well-written proposal or paper is more likely to convince you that the research is good, independent of the content. I find I've often fallen into the trap of thinking more highly (initally) of a paper's research if it was straightforward to read, and vice-versa, for proposals that are difficult to parse/exhausting to wade through.

OTOH as you pointed out, people who like the research are likely to transfer their liking to their opinion of the writing. Often "well written" implicitly means "the authors expressed opinions I agree very strongly with"

hc230 said...

I recently took part in a mock panel session (designed to teach junior researchers like me how the system works), in a European country. We looked at the reviews written (for genuine proposals) and had to decide which ones we would have funded. In one particular case, I was appalled by how poor the writing in one proposal was, but all the reviewers said that the science was excellent, and in the real world this proposal had been funded.

I do think that the writing is important though - if someone can't communicate what they want to do to a funding body, how will they communicate what they've done in a journal? I am frequently driven mad by poorly-written papers, and I don't see the point of doing science if you are incapable of telling anyone what you did. Surely communication matters!

Anonymous said...

My interpretation of these comments may be described by the example "This is a well written and prepared proposal". The "and prepared", included explicitly here, adds another dimension which is implicit in the other comments.

I.e. I would take "well written" to be synonymous with "case is well argued" or "proposed work is logically presented" and not just related to the writing style.

studyzone said...

Although I'm not yet a reviewer on my own, I have reviewed manuscripts/NSF proposals with my PI and good writing does help, particularly when there is a finer or more nuanced point to be made. However, bad writing doesn't necessarily sink a proposal, nor does good writing save one (I can attest to that - my NRSA proposal was unscored. 2 of the 3 reviewers commented it was one of the most clearly-written proposals they'd read - but there was no enthusiasm for the project, which suggests that my powers of salesmanship need a lot more work.)

Anonymous said...

I'd hate to think my writing put me in or out if I were on the border of funding. I generally do not comment on writing per se, because there are usually < 10 people who will ever read the proposal. However, the written form is usually the only method of evaluation (and is the first step to a possible oral presentation), so I WILL comment on the organization of a proposal. Organization is related to writing but it also reflects the way the research might be carried out and presented later.

In my field the program manager would make a decision on who gets funded, especially with who is on the border (that depends on specific $ available) so I doubt writing plays a big part of that.

I think it's important to point out when a proposal is particularly poorly written. If someone has a great idea, they should take suggestions and get help with the writing.

Anonymous said...

I appreciate good writing and will probably call it out. When one has 35+ proposals to read, those that make the job easier to do will be thanked.

Does it matter for the ranking of the proposal? Depends on what one means by "good." I will give equal weight to a perfectly penned proposal and one that has typos and minor grammatical error but remains easily readable.

On the other hand, we submitted a NSF proposal one year that was rejected. A year later, I was asked to review a proposal for the same research by another group. Being intimately familiar with the project, I could see areas where the authors had glossed over potential difficulties, but I was nevertheless struck by how the author's writing made the work seem so compelling, so necessary, so inevitable, and not just to me, but to the other panel members, who ranked it highly. That is the kind of good writing that can sway readers. (We did get our work funded too, but the 2nd group continues to do a much better job selling and publicizing their work. Our PI is a modest man and modest men are not rewarded.)

mathgirl said...

I don't have much experience, but I try not to be influenced by the quality of writing as long as the proposal can be read and has all the items is supposed to have (non obvious).

I recently reviewed a proposal that was very well-written except that the citations looked like [?] instead of [1],[2], etc (this is a common mistake if you use LaTeX and you don't compile enough times). Anyways, I did add a note in the review, because this was very annoying (I did recommend funding, as the proposal was, otherwise, excellent).

M. S. AtKisson said...

We used to joke in the proposal development office that seeing "This is a well-written proposal" was a death knell in reviews. Studyzone has a great point: good writing and good organization are not enough without a great idea with a solid, or, even better, an exciting approach. On the flip side, a great idea can be buried in a badly-written proposal, especially if the reviewer is tired, annoyed by one of their grad students, and not looking forward to a pile of proposals. :) I recommend this article from Science on the results of in-depth interviews and focus groups of NSF program officers, successful applicants, and unsuccessful applicants. The take-home is that highly successful grant writers think of every part of their proposal as a tool for communicating with the reviewers and with program. Good writing - effective communication - is a big part of that.

But the idea and approach ultimately drive success or failure. One reviewer told me a story of sitting in a panel. "So, the primary reviewer starts out saying, 'This was the best-written proposal I've ever seen. I couldn't wait to turn the page and find out what was gong to happen next.' He paused, and we waited for him to tell us what great research it was, but he said, 'Too bad it's a dumb idea.'"

Tom said...

Where I work, we've taken several writing courses. The feeling is that if we write our proposals well, then the actual science in the proposal (or manuscript) will be assessed and critiqued fairly. If the writing is horrible, the reviewer may be led down a rabbit hole that would not have existed if we'd done our proper job in writing the piece to begin with.

Anonymous said...

I frequently comment on the writing. I appreciate proposals that are easy to read, and I am certainly willing to consider that I may have a subconscious bias against poorly-written proposals. They make my life harder; if you're asking for my time to get money for your research, can you make it as easy as possible for me please?

I generally do not assume that good proposal writing = good paper writing (as some commenters above have suggested), because they're such different animals that I don't think that equality holds. I've read proposals that are written like papers, and even if the writing is good, I wouldn't say they're well-written proposals---they're well-written papers that got submitted as proposals.

Anonymous said...

I think your interpretation of what reviewers mean by 'good writing' is too narrow. When I have commented on writing, I mean I think the authors do a good job conveying the significance of the work, muster a clear, logical and compelling argument for why it should be done by their favored approach, and provide a thorough and scholarly outline of how this research fits into the bigger picture. It's about the scholarship and clear thinking, not typos and grammar. A bad idea won't come across in significance; a poor approach won't be helped by a clear but flawed argument. Lack of citations and poor, unclear descriptions of previous work and literature indicate a lack of scholarship. So the writing and the thinking are inseparable. Run-on sentences and long blocks of text make it harder to parse a good argument, but don't completely ruin it.

Allison said...

I definitely factor in the quality of writing. If someone isn't smart enough to string together a coherent sentence, they aren't smart enough to deserve funding. If you know that you are terrible at writing (not a forgivable offense, btw), you should at least be smart enough to have colleagues edit your work before submission.

Anonymous said...

When you write a grant proposal you are trying to sell the panel on the importance of your project. To do so you have to have a good project, AND you have to communicate it well. If this is done effectively, the reader will not be marveling at the writing quality, but rather at the quality of the research ideas. So reviewer comments on writing quality might not give a very accurate measure of writing quality.

Anonymous said...

I agree with several of the points made by others. It's far more likely for a good idea, well-presented, to be funded, than for a good idea, badly presented, to be funded. And sometimes even bad ideas, well-presented, can persuade tired and inattentive panels. (And good ideas, well-presented, can be shot down by an idiot on the panel.)

But once in a while, I come across a badly written proposal with huge gaping holes, and find that panel members say, "Oh, Dr. X knows what he/she is doing, so this is probably ok." I am willing to give minor brownie points for established track records, but huge gaping holes I will not allow, even if Dr. X won the Nobel Prize. If Dr. X really knows what he/she is doing, then Dr. X is being sloppy in proposal writing, and I hold him/her to the same standard that I would hold to someone relatively unknown.

I often comment on poor argumentation, e.g., "it is not clear how A, B, C" "the proposer does not explain why D, E, F". I think that's really what good writing is about.

Anonymous said...

My goal in writing a proposal is to make a compelling argument for the research AND to make life easy on the reviewers. Making life as easy as possible on the reviewers means that it has to be well written for its purpose - for a proposal, that means well organized, simple clear sentences, specific wording, straightforward thoughts and goals, not any extraneous material. That may not be the way you write the same science ideas for different venues. For every author that wants to save time by not editing their proposal there are 3+reviewers that have to waste time trying to figure out what they are proposing to do!

I do usually comment on it and probably subconsciously it enters my judgement - because a proposal that is easy to read puts me in a good mood and I'm less critical about the details. Proposals that are frustrating and complicated to read put me in a bad mood and I'm more inclined to nit-pick the details.

I specifically remember commenting on one proposal that I thought the writing was sloppy, but that despite that I really liked the science and I thought it should be funded. That proposal was funded, but I have to admit that I was a little disappointed in the sloppiness because it was written by one of the people in my field that I regard highly (one of the few women at the top in our research community).

EliRabett said...

As Eli said to a program manager, the proposal is a prospectus, if the PI can't explain to me what is to be done and why it is important, I am not going to rate it highly. It is not the job of the reviewer to go out and rewrite the proposal.

The last point always comes up in panel reviews. Eli the merciless.

Margot said...

When I review proposals, I place significant weight on the writing. Clear communication is specified, in way or another, in all the NSF rfps. It is nearly impossible to pose a sufficiently detailed plan to warrent a positive review without writing well.Further, in a resource-poor reviewing environment, we need to clearly distinguish "good" from "great"- Hardly anyone proposes an idea so ill-intentioned that it is not worth doing. In a 10% funding climate, reviewers need to use every criterion at their disposal to choose which of the many good ideas is worthy of support. Writing helps make this distinction.. Consequently, I nearly always comment on writing.

Anonymous said...

When I see the phrase "Principle Investigator" on a proposal, I really want to throw it against the nearest wall. Honest typos don't offend me, but how can anyone reach the career stage of being an independent researcher without knowing the difference between Principal and Principle?

Anonymous said...

Anonymous @ 9:48 - maybe they really do want to investigate principles? A philosopher gone to the hard sciences? ;)

yolio said...

I usually comment on the writing. This is in part because I don't see the matter as having a minimum threshold like you do. I think that there is a continuous linear relationship between how well written it is and how easy it is to read and evaluate. Also, I believe that the quality of writing is a function of effort. The way I see it, when a writer makes the effort to write well, then they have made my job of reading and evaluating easier. So I try to express my appreciation, or irritation in the case of poor writing.

I don't factor this directly into my evaluation of the document. As long as the document is intelligible, I will evaluate it on the science proposal alone. However, in my experience, poorly written documents are often sloppy in their thinking as well. Probably i am biased, but I can't help but taking bad writing as an omen of what is to come.

Anonymous said...

With each year getting more competitive with funding, I think reviewers are reaching for anything to help judge proposals. If there are two proposals that are essentially equal in the quality of science, and one is written better than the other that will help make the decision. In addition, the writing may reflect care and consideration to details that will carry a project through to publication.

Anonymous said...

In a recent large-scale research panel I was on, the finalists were asked the question in person, to explain their research to a State Senator. This is where almost everyone failed miserably. Similar to how overly technical the proposals were written, the researchers could not present the science in an understandable manner.