Every once in a while, there is an offensive statement in a review of one of my proposals. I am not talking about a "This research is not worth doing" type of statement, or even a "The PI has absolutely no qualifications to do this kind of research" type of statement. I mean statements that, in the view of the funding agency, inappropriately comment on an aspect of the proposal or the PI.
Some program directors at funding agencies will redact portions of reviews that they consider inappropriate. I know one program director who deletes such review comments and, in the specific place where the text has been deleted, inserts a statement that this has been done. He also includes an explanatory note that the comments were inappropriate and were not considered in the final decision about the proposal. The comments are not actually deleted from the system, but the PI can't see them.
I think that in some cases these comments are inappropriate because they are offensive, and in others they are inappropriate for technical reasons.
Another option that program directors seem to have is to strike-through the offending text. This leaves the text still visible to the PI but lets the PI know that the program director thought the comment was not appropriate. The overall ranking of the proposal by that reviewer may still be considered, even if particular comments are ignored.
I have seen both types of deletions of review comments in my proposals, but the strike-through method was used in the specific example I am going to mention today.
The specific example I wish to consider involved a review comment on the Broader Impacts section of one of my NSF proposals.
NSF's guidelines for what might be considered Broader Impacts include this statement:
How well does the proposed activity broaden the participation of underrepresented groups (e.g. gender, ethnicity... etc.).
In a one page description of the Broader Impacts of my proposed research, amidst a long summary list of activities of a Broader Impactish sort, was this item in a bulleted list (to be exact: the last item in the list):
and research support for a female faculty member (the PI).
The Broader Impacts part of my proposal was quite favorably reviewed by all but one reviewer. That last statement set off one of the proposal reviewers, who expressed severe disgust and revulsion for this kind of tactic. The program director had put a line through this strongly worded opinion about my use of my gender to get a grant. The reviewer also expressed a similar but less harshly worded opinion in other comments (not lined-through) in the same review, and his/her disgust seems to have affected the overall ranking, which was the lowest of the set.
Of course I didn't need to mention in the proposal that funding this proposal would support the research of an FSP. So why did I include it in my proposal?
I must admit that I didn't give it a lot of thought when I was writing the proposal. To the extent that I had a motive, I suppose I added that last item to my bulleted list summarizing all the Broader Impacts of the project because it made the list complete (note use of defensive font). If someone was pre-inclined to be suspicious of women PI's, I can see why they would be offended by the inclusion of such an obvious statement, although of course one would hope that such a reviewer could ignore their anger and revulsion and at least give an objective review of the Intellectual Merit of the proposal.
In discussing this situation with colleagues, I have heard different hypotheses for the reason behind the reviewer's high level of disgust, which went far beyond being uncomfortable about the concepts of gender diversity or Broader Impacts, but all these hypotheses involve the same central element: the reviewer, male or female, hates the idea that someone might get a grant (or job or other 'privilege') just because she is female, and not because of intrinsic merit, whatever that is.
Somewhere out there is a reviewer who ignored 99.9% of the proposal and got incensed about one item in a summary list in the Broader Impacts section. I hope that the program director will remember that this reviewer is hostile to proposals that mention gender diversity in their Broader Impact statements, at least when it is a woman PI mentioning this.
Perhaps I will leave such an obvious statement out of my proposals in the future, as it doesn't add anything substantive. Even so, most proposals that I review are by male PI's who are involving women students in research, and who mention their efforts to increase gender diversity in the sciences as part of their Broader Impacts statement. Perhaps this is OK with my hostile reviewer as long as these female students don't grow up to become FSPs who write their own proposals.
14 years ago
26 comments:
Wow. I'll have to put a link to this in today's post over at my place, which deals with ridiculous reviews.
I thought your complaint was more going to be about the reviewer not properly aligning their own criteria with that of the respective agency: it's NSF policy to promote funding "under-represented minorities".
Certainly I've played that game myself as a MSP, although I'm not convinced that these broader impact statements do anything other than provide some good P.R. for program managers as they defend funding decisions. I certainly don't believe it really moves decisions one way or another, current example notwithstanding.
I'm curious, though, you avoided the interesting question: What are your opinions whether gender or race should play a role in funding decisions?
"I hope that the program director will remember that this reviewer is hostile to proposals that mention gender diversity"...uneffingbelievable... the PD BETTER DAMN WELL remember to NOT fund the sexist a$$hole.
"Perhaps I will leave such an obvious statement out of my proposals in the future, as it doesn't add anything substantive."... I say HELL NO on that. In fact, my next NSF prop, I will add myself as a broader impact. More of us non-white-priviledged-males should be IMPACTED!
For anon 12:36... race and gender SHOULD play a role in federal funding decisions... there's this thing that's been around for a few decades now called Title IX.
Comrade PhysioProf wrote about deployment of a different inappropriate review criterion just recently at DoucheMonkey.
Well, I think anonymous 12:36 hit it right on the head. If encouraging diversity is a main goal of the program, then if PI is one of the underrepresented groups, should that be enough? Or does every PI should include a detailed plan to help *other* underrepresented minorities?
I personally believe that affirmative action has a larger role to play at high school and college levels, and should be gradually phased out at higher levels. If someone is a well-established, tenured faculty with solid reputation, who happens to be female, hispanic, african-american, come from impoverished background, first to attend college, etc. - why should their grant applications be treated any differently?
Don't you think your decision to use yourself as the female scientist who will be "helped" by the grant is at least a little a bit selfish? The original idea of the grant is to give money to PI so they can do research and educate *others*. By supporting yourself you are not doing anything to help others. Does that "broaden under-representation" in any way?
It also sounds as if you argue - I don't plan to help others, I plan to help myself. I know from reading your blog this is not the case, but this is what you essentially proposed.
I think the reviewer should have been civil, but I would have raised a question about whether you satisfied the NSF broader impact as well.
I agree with anonymous at 12:36 - the obvious problem seems to be that NSF has a position on encouraging gender diversity that this reviewer chose to ignore.
But I do hear where the reviewer is coming from -- I can't see why your gender, race, country of origin, whatever should matter - it should be about the work you're proposing and it's value, and whether you as a scientist are qualified to do it.
If anywhere, it seems to me that the place for funding agencies to give extra points for diversity would be at the student level, i.e. when granting graduate fellowships, to encourage young people from underrepresented groups to pursue careers in a particular field.
So, editors think that if you just cross out the particularly offensive bit, the rest of the review is probably ok and not biased? If someone says something offensive about the proposer, I think their entire review should be tossed in the trash.
and I'd never thought of mentioning my gender in the broader impacts of my NSF...hmmm.
Thanks for this post. It is depressing, but important.
I see no reason why "broader impact" can only include students or maybe postdocs. We all know women professors are still paid substantially less than men, on the whole (and most notably at the full professor level), and I'm sure the research compensation from universities is also very unequal. If female professors aren't properly funded for their research, they will not have the resources to achieve prestige and recognition on par with their male counterparts (on the whole). That should be obvious.
The reviewer was way out of line, and I wish, given the very obvious bias (I mean, they didn't even try to hide it), that their overall review was not considered. Shame on the program director for still considering it.
I agree with anonymous that the reviewer clearly thinks his opinions (it may have been a woman, but more often this attitude comes from men) are more important than NSF's official positions. You have no business reviewing proposals for foundation funding with a personal agenda that clearly contradicts the foundation's goals, if you can't set that aside.
I'm interested in the fact that this reviewer expressed disgust, and not anger at your statement. It so happens that my research topic is moral disgust - I don't suppose you would be willing to send me the text of that section of the review? I'd be happy to agree to any "terms of use" you would like, although it seems unlikely that your reviewer would ever by reading any psychology publications/attending any psychology seminars.
- Hanah
My PI has used my gender (and that of other students who left due to depression and isolation in his group) in grant proposals. I think this may support your theory.
I think that the reviewer's comment was out of line. I also think though that it is not appropriate to mention yourself in the broader impact section. Affirmative action is good, but the truth is that it doesn't actually change people's biases against women or minorities. I met a good number of white men who are quite angry at the fact that affirmative action exists. I suspect there are a lot of these people in the panels. If NSF wants to fund you and to take in consideration your gender, they will fund you regardless of what the reviewers say. The reviews should be rather positive though (you don't need to be top 1 or 2--but up to #5 you still have a good chance if you played your cards well), therefore I would increase my chances by not ticking off these hostile people that are on panels. If you didn't bring it up, he may have forgotten about the fact that affirmative action and female academics make him sick. I am an untenured FSP myself and I never mention my gender in my applications. I was lucky to get enough grant money (three grants, federal) without specifically pointing out that I am a FSP. So can you.
I think it would be easy to come up with a coherent argument for why broader impacts should be, well, broader than the particular PI. I don't know if that's compatible with NSF's policy on broader impacts but I can at least see the argument.
However, people who go beyond a coherent argument (e.g. a mild "I would prefer to see a statement that focuses on benefits to people other than the PI") and into an incoherent rage generally have issues that go way beyond some principled objection regarding the scope of the impacts.
I admit that I've been on the fence regarding when factors such as race or gender should or shouldn't be considered in a particular context. The people who fly into a rage over consideration of race and gender generally wind up steering me to the opposite side of the fence from them. And, honestly, while I may have ambivalent views, if I absolute have to pick a side of the fence, I know whose side I don't want to be on.
In a similar situation I did mention that I have a high proportion of female students; that I am part of a professional female association, and am particularly appreciated there as role model because of my being a FSP with 3 kids.
In answer to the many people wavering "should gender or race be relevant"; this is an interesting question, but it's not the referee's task to answer. The point is that here diversity support was one of the criteria, and I do think that it's easier to support diversity if you have experienced it on your own skin. So FSP's gender is relevant in this context.
Regardless of whether it makes any sense in 2008, or in certain corners of science, NSF set up the guidelines for receiving funding that include broader impacts. Theoretically (not in practice) the broader impacts are just as important as the intellectual merits of the proposal. There are five criteria that are considered "broader impacts:"
1. How well does the activity advance discovery and understanding while promoting teaching, training, and learning?
2. How well does the proposed activity broaden the participation of underrepresented groups (e.g., gender, ethnicity, disability, geographic, etc.)?
3. To what extent will it enhance the infrastructure for research and education, such as facilities, instrumentation, networks, and partnerships?
4. Will the results be disseminated broadly to enhance scientific and technological understanding?
5. What may be the benefits of the proposed activity to society?
(obtained from http://www.nsf.gov/pubs/2003/nsf032/032_3.htm#IIIA2)
#2 on this list involves broadening female (among other groups) participation, and FSP isn't in a field where there are abundant tenured full professor FSPs around. By being a well-known, visible, well-funded FSP, she is a role model for female science grad students and female science undergrads. While it's clear this reviewer had a short fuse, its equally clear he didn't read the guidelines for grant review before serving on this NSF panel. If the NSF rules say that you will be more likely to receive the grant if you can help foster gender diversity, then why wouldn't FSP mention it in the text??
[One has to write it in the actual proposal in order for the reviewers to check it off in their broader impacts section -- they don't receive the gender/ethnicity page we fill out, that's removed and reserved for NSF administrative purposes]
Maybe the NSF rules/guidelines will change, maybe they should change. But as they stand, you'd be foolish to not mention the ethnicities of the participating scientists, if they are underrepresented, when preparing an NSF grant. For a dissertation improvement grant, I once had to write a very awkward sentence, saying that I am female and my (male) PhD advisor is black. But if we hadn't done that, we wouldn't have had as strong of a broader impacts score. And intellectual merit, theoretically, is only 50% of the reason NSF grants are awarded.
For those on here who think the criteria should change, and gender or ethnic background of participating scientists shouldn't be any factor in awarding NSF grants, how do they feel about grant programs that are solely open to underrepresented minorities in science? Would they preserve these separate, typically lower $ grants (postdoc fellowships, research initiation grants) but remove the advantage in the regular grant pool?
For those of you that say that affirmative action is more important at lower levels of education: funding FSP's work influences students at those levels. If young women don't see someone who looks like them doing the work that they aspire to, then they won't think they can achieve that career. By funding an underrepresented minority, the NSF is encouraging current and future FSPs. That is a very broad impact indeed!
I don't think that FSP ever said she was listing herself under the Broader impact section with herself being the only "item" of a Broadish Impact sort, as many of the anonymous commenters seem to be implying. It was one bulleted item among many, and shouldn't have elicited a response of "severe disgust and revulsion" no matter what a person's ideas are on NSF's policies.
If there are still inequities in funding at FSP's level, then what is the difference in listing one's self and listing female students?
Well you could have been a bit more subtle about it.
On the whole tho Eli is with estraven. As a reader of proposals, I do take it into mild consideration when the proposer is a rare bird, but it is a minor plus. OTOH, if they carefully explain how they will/are leveraging their status to assist others it can become a huge plus
I've had the same experience as globalgirl. My former PI included my sex on his proposals because it showed diversity in his projects - this despite the fact that he couldn't seem to hand me wieldy things without running his hands across my chest.
"Perhaps this is OK with my hostile reviewer as long as these female students don't grow up to become FSPs who write their own proposals."
Exactly. It's as though MSPs can be the only ones allowed to benefit from that aspect of broader impacts. So as long as they have a token in their lab that should be justification for funding, but if an FSP dare to point out her F-ness, the horrors!
I wonder what the reviewer would have thought if that item had stated that the research team includes both genders (assuming that it is a team, and not just you on the project) and therefore gender diversity is maintained?
I wonder whether the program director notifies the reviewer that his/her review has been redacted? It seems the reviewer should be informed that these types of comments are out of line and not acceptable. Might give the person some pause for thought (or am I being overly optimistic)?
I can understand the reasoning for mentioning gender as it makes the application look more favorably. I don't think it is right, but I believe the issue should be taken up with the NSF as it appears they encourage that activity, and not with the application writer who is just attempting to play by the rules.
Of course a lot of the reviewer's emotion must come from feelings about gender issues or affirmative action. However, I do think your proposal may have looked like it was violating a key cultural norm for NSF proposals. (Not that this excuses the reviewer's behavior, but it may help explain it.)
NSF proposals typically spend a lot of time talking about how they will lead to intellectual progress in their field, benefits to society at large, opportunities for students and postdocs, etc. However, mentioning any benefits for the PI is strictly taboo. I've been on a lot of NSF panels, and every few years someone breaks this taboo (to say explicitly that getting a grant would really help them get tenure, or that their salary is too low and they really deserve summer support, etc.). This typically gets a horrified reaction from the panel, since it violates the polite fiction that PIs exist only to serve others.
Your comment about how one broader impact was providing research support for yourself is much more mild. I don't think it would bother most reviewers, unless they already had serious issues with women in science. However, I suspect that explicitly mentioning the benefit to yourself may have set off additional disgust in that crazy reviewer, and it may have released him from some of his inhibitions (since then he could justify it to himself by imagining what a terrible person you must be to write, or even think, such a thing).
I don't mean this as criticism of you or your proposal. I just want to try to explain where I think this person may be coming from.
as a female and an ethnic minority in a male-dominated physical sciences field, I have never tried to capitalize on my membership in under-represented groups. In graduate school and postdoc I never applied for fellowships that were aimed at females/minorities, in fact I deliberately stayed away from those and perhaps it was to my detriment (perhaps I would be more successful in my career now if I had made use of all the resources available to me). I don't fault those women/minorities who did take advantage of such programs, but I couldn't help but think I would feel patronized if I did get a fellowship/grant in large part because I was a woman and/or minotiry and wouldn't othewise have.
However, both my grad school and postdoc departments have used my picture on their promotional materials to advertise how 'diverse' they are, even though the truth is they are not diverse at all because I was the only one (or one of very few) who was female and/or ethnic minority.
However on the other hand I do feel that it's a shame that honestly stating that a proposal will benefit yourself (the PI) is taboo because what is wrong with being truthful?
I agree with grad student and lynne: This is fucked up for a reason you don't mention in your post. Hw can these people be considered objective and allowed to continue reviewing grants?
Just striking out one offensive comment doesn't strike out their obvious bias, which surely affects the overall score and the author's ability to improve future applications.
Should I be assuming this has happened to me without my knowing it (?). Are they even required to say that they've redacted the text, or is that at the program director's discretion?
This reminds me of a piece of news this week, that Obama will probably pick Larry Summers as one of his economic advisers. I think we need to send a message to these people that, while they might get a second chance to make up for being assholes, they should not be promoted to positions of power as if they aren't dangerously biased.
@anon 7:52,
FSP has broader impact as a ROLE MODEL. Just staying in business, she has a huge impact. So getting the grant alone, yes, it does make a huge difference.
@anon 7:55,
I disagree re: giving out more graduate fellowships without continuing to give support at higher levels. I got a graduate fellowship and a postdoc fellowship and now I can't get a faculty position. There's no point in bringing all these people into the pipeline if you're just going to dump them into a sewer halfway down the pipe.
@anon 10:27 who suggests we should, what, hide our gender to avoid pissing off these sexist assholes? Are you even listening to yourself?? What if, god forbid, your name in real life is clearly female? Aren't you worried they might NOTICE? How did you even get your job, just by keeping your head down? You're not part of the solution, FSPs like you are part of the PROBLEM.
Yes, I had something similar happen to me on an NSF proposal...I'm a historian of science, and I was attacked for doing work on a white male, and for the publisher the book was contracted with, because the books are expensive and inaccessible to the larger public (Yet the program specs are for academic monographs) When I resubmit, I'll mention that I've written articles about my work for National History Magazine, and given public lectures about it at the Royal Society. Maybe that will wake the reviewer up, who knows?
Post a Comment