Owing to the apparent youth of much of the academic segment of the blogosphere, it is not difficult to find posts by assistant professors describing their activities and thoughts during various stages of preparation of CAREER proposals, and their elation/dejection upon receiving news of NSF's decision.
My own CAREER grant expired long (long) ago, and now my primary experience with the CAREER program is as a reviewer of proposals and as an occasional mentor (in real life) to colleagues who are preparing their own such proposals.
Fellow reviewers of CAREER proposals (or whatever equivalent there might be in the NIH university): let's discuss how we go about reviewing these things.
If you have reviewed one or more CAREER proposals, do you:
1. Ignore the fact that these proposals are a bit different from regular NSF proposals, even though you know better, and review them much as you would any proposal (perhaps accounting somewhat for the relative career youth of the PI). That is, do you mostly ignore all the extra educational components of the proposal and focus on the Science?
2. Pay some attention to the education and/or outreach plan, but focus mostly on the Science?
3. Give serious (perhaps equal) thought to the plan for educational/outreach activities, provide detailed comments on these in your review, and factor in your opinion of these in your overall rating of the proposal?
- Other..?
From what I've seen through various intersections with the CAREER program over the years, different fields have different philosophies about these grants and how important they are. I think everyone agrees that it is a good thing to get these grants, owing in part to their being of longer duration (up to 5 years) than a typical grant (2-3 years). There seem, however, to be differences in practice about when to submit the proposal, e.g.:
The CAREER grant can/should be the first grant obtained by an assistant professor, and therefore should be applied for early, even if the individual has no previous grant track record,
vs.
A CAREER proposal should be submitted after there is some grant track record, even if after only one other NSF grant.
And I have heard rumors of fields or subfields in which a CAREER grant is essential for tenure at some institutions, but I have not seen evidence that these rumors apply to anyone I know or to any department at my university as long as there is otherwise a solid record of funding.
When I review a CAREER proposal, I certainly look at the required education/outreach parts and I like to see a sincere effort with this part of the proposal, but I must admit that I think it is asking a bit much of assistant professors to have a particularly sophisticated plan. It's great when someone really does have a creative and detailed plan, but, as long as there seems to have been a sincere effort with this part of the proposal, I don't penalize PIs whose broader impacts aren't awesome. That doesn't mean I don't value education or outreach; it just means that I think we have to be reasonable about expectations for already overburdened early-career faculty at research universities.
So, I guess I'm mostly a #2 in the list above, with the caveat that the education/outreach plan has to have some substance to it. After all, that is part of what distinguishes this grant from others, and partly what justifies the longer duration.
Another issue arises when reviewing CAREER proposals from faculty at different types of institutions. This is a general issue when reviewing proposals from, say, faculty at research universities vs. faculty at small liberal arts colleges, but it's a particular issue for CAREER proposals because the education activities might be more of an expected and valued part of the job for some faculty than for others. I actually don't think it's a major big-deal issue because we routinely deal with these types of differences n reviewing proposals, but perhaps someone disagrees with that opinion?
So, how do you review CAREER proposals?
14 years ago
19 comments:
I've had a CAREER award, and been a reviewer on a couple of CAREER award panels. Here's my take:
- regarding science, i of course want to see good science. i'm looking for a program that builds on what they currently are working on, but expands it into some new aspect. That way there is both in-hand expertise and new development involved.
- regarding education, i want to see some evidence that they are thinking about educational issues, beyond just "I'm going to teach a 100 level course, and give some public talks". Something interesting/creative is good, although I agree that I'm not expecting a new faculty member who has taught very little will suddenly have blazing insights into pedagogy (although if they do, great).
The biggest struggle I see is applicants trying to hit the right combination of science vs education. Too many applicants blow off the education part, and too many applicants set out a level of educational activity which would demand a HUGE amount of time. It should be in-between -- science is first priority, and then you want to do good educational activities at a reasonable level. For me, I'd say a reasonable effort is on the order of 15-20% or so of your CAREER grant effort.
I fall between #2 and #3. Yes, I check to see there is *some* substance to the education plan. But in my experience it is equally typical to see an assistant professor propose an education/outreach plan that, if executed as described, would take up so much of their time that there is no way they would be able to get their research program rolling as needed for tenure. I have probably downgraded just as many proposals for over-reaching on the education plan (proposing to revised an undergraduate curriculum at a research university, for example) as for under-reaching (superficial, tacked-on fluff).
To me, the CAREER is different enough from "regular" NSF proposals that wanna-be PIs should get their draft proposals, especially the education component, vetted by some senior colleagues who have had and --even better-- regularly review CAREERs. It takes some careful thought to hit the right balance: substance, and filling a need, without overwhelm.
I agree with FSP and the first two commenters. I take the education part of CAREER proposals seriously and comment on it as an important part of my review. But I weigh the research portion more heavily. And I too prefer a balanced educational component rather than an overly ambitious one.
What not to do (as far as I am concerned): I think teaching courses is part of every faculty member's job. Designing new courses is also, to an extent. Writing that you will design a new course on the special topic of your research field does not seem like an exciting idea to me. Rather, it's a nice thing that your department is letting you do because it's easier than teaching a big intro class. So you can mention it, but I like to see something a bit more. Outreach efforts (even very minimal ones) are what I like to see.
I have seen two very different types of CAREER Awards in my field (basic biomedical science). One type was awarded to two of my former postdocs, both at small undergraduate only institutions. These had only minimal preliminary data and thus were obviously judged primarily on their "other impacts". Both received them as the first grant. The other went to a colleague (recently tenured at the time) here at a top 25 R1 University. He had a load of preliminary data, including a substantial paper in a high profile journal. He also had a serious educational plan. His would have been judged very differently. For him, it was a second grant, in an area different from the RO1 he already had. It may be relevant that he got it renewed once, and then got parked just above the payline (once even with a verbal claim of funding) before giving up, feeling the message was this should be funded by NIH.
Mark P
The educational part should be strong, integrated with research, and should strengthen and support the research. However, you don't want the broader impact to be the focal point of your proposal -- the awesome research must be the focus. Utllizing existing NSF-funded programs and intitutional resources (e.g. for recruitment of minorities or summer students) is a good way to show you are serious while minimizing legwork.
Rule of thumb: for CAREER, broader impact 5 pages out of 15 (regular proposals 2 pages or so out of 15).
I also fall between #2 and #3. CAREER proposals are different from regular NSF proposals and should be treated as such. I know the PO's who have run the panels I've sat on take all aspects of CAREER awards very seriously.
It is a balancing act for the applicants - too much on the educational side and reviewers will penalize you 'cos you won't have sufficient time to follow through, too little and you're slammed for not staying within the intent of the award.
Would any people with reviewer experience care to offer advice for those of us in departments with no graduate program? I've published a few papers, including papers with undergraduate co-authors, and I'm doing science that has some people in my field intrigued and could lead to some important things. Any rough guidance on the extent to which the research plan vs. the education plan will matter?
My tentative education plan is to (1) convert some of my research simulations into educational simulations (basically, strip out some of the more abstract and technical parts, jazz up the graphics) and disseminate (with accompanying assignments used in my own classes) via some popular educational sites (e.g. nanoHUB) and (2) give some public lectures through an outreach program tied to a local chapter of a professional society (I'm currently active in this chapter, and could get a letter of support). Is this too much? Not enough?
My biggest worry, though, is actually on the science side: I would say that the problems I'm working on are important and their solution will lead to some useful tools, but when I first contemplated applying for a CAREER somebody asked what I would do after these problems are solved and the tools are developed, and admitted that I would probably have to change direction and start studying questions related to the things studied with the tools, rather than study questions that are extensions of what I was doing. He said that if it doesn't lay a foundation for a career-long program then it isn't a CAREER-worthy project.
Any advice?
I wish someone had given me better (any?) advice about CAREER awards. By the time I had figured out how valued they were, I had already become ineligible.
@Alex: your educational component sounds reasonable to me.
Research: I disagree that the research has to lay the foundation for your career. But it's a five-year proposal, you have to have plenty to keep you busy for five years. And it needs to sound like exciting and publishable science. Often, that implies that there's plenty to do once you're finished with this first grant, but not necessarily.
----
I realized I didn't answer part of FSP's original question: No, I don't think preliminary data (or a prior grant) are needed before writing a CAREER grant. My advice is the exact opposite. Write a CAREER grant as soon as possible so that you get feedback on it. Let the referees decide if you need preliminary data. Yes, you can only apply three times for a CAREER grant, so try to submit a grant that is at least credible. (Have an older colleague look it over to give you advice.) But there is no reason to delay. You got hired to a faculty position based on your awesome ideas, and your school gave you startup money for those ideas. So maybe NSF will give you a CAREER grant for those same ideas.
Regarding "proposing to revised an undergraduate curriculum at a research university": I revised an undergrad curriculum at an R1 university as an assistant professor, and created and taught several of the courses in the new curriculum. No one suggested that I apply for a CAREER award for it!
Of course, it was a new department and the few senior faculty had been hired from industry directly with tenure, so no one in the department knew anything about the tenure process. This may have contributed to my slower than usual academic progress. I eventually made full professor, but my step and salary are behind those hired 5 years later.
@Alex at 10:49:
What most distinguishes CAREER from other grants is scope. I like to think of it this way:
Someone gives you $400K to spend over 5 years, virtually no strings attached. You are totally allowed to dream big. Ask yourself: what is one big, important, exciting problem that you would use this money to solve? You don't want it nickel-and-dimed on small extensions of previous work. If there is one big problem question you are really excited about answering, which perhaps opens a small niche area just for you so to keep mining during the 5 years and beyond and where you can make a name for yourself, then that's your CAREER project. Painted with a broader brush than regular NSF proposals. A bit more detail here.
Your education part sounds good.
I agree with 'a physicist'.. I applied for the CAREER in my first year on the assumption it wouldn't get funded but the feedback would be useful. Well, it did get funded, and the feedback was useful (unlike many of my other grant reviews from foundations, etc). I have many colleagues who seem afraid to submit it since you only get 3x - that's still 1x more than any NIH grant, so to me it seemed worth the gamble (and obviously was).
In response to Alex - my proposal definitely has the potential to lead to other projects, and I have a friend who was dinged on having a project that was too self-contained. But, I don't think this has to truly be the project that makes your career - rather, like any grant I tried to give a picture of how the work I would do over the next 5 years would add to the community of science and how I could use it to lead to other work I would find interesting.
On how mine was reviewed - people definitely hit more on the science, but the educational plan did get discussed and deemed sufficient - likely meaning I had reviewers such as the other commenters who want to see something, but not too much to distract from doing science and getting tenure (I hope!)
Good luck to everyone putting one together - my advice is to read examples and have internal review, preferably by people who have one or have reviewed for this. That (somewhat painful) process improved mine drastically!
I have sat on a CAREER panel. The proposals were universally of high quality, all proposing great science but from wildly disparate subfields within the main discipline. Therefore a substantial portion of the evaluation fell upon the Educational Component as the portion of the proposal to differentiate great proposals from outstanding ones.
For instance, proposing to develop and teach a new course was not helpful to the applicants. It's a CAREER award, so all of the applicants are professors, and teaching is your job already. We were looking for someone who showed leadership in a creative and helpful way that stood out from the pack.
In the end the task was supremely depressing in that of our 50-some-odd great proposals, I think that they had money to fund 2. What I learned from having sat on the panel was that a CAREER isn't worth your time if your goal is to get funded. There's much better expected values for funding from other programs.
Anonymous 8/10/2010 01:09:00 AM: A 4% funding rate is pretty close to rate in the NSF program to which I send my proposals. Yes, it's depressing, but to not try because the odds are long is career suicide. Also, it's not a one shot deal. If the investigator can adequately address the reviewer and panel concerns, the odds should be at least somewhat higher for a resubmission.
Timing/grant record: I've reviewed CAREER proposals by both early and late applicants, and I don't think stage has any effect on how I evaluate the research component of the proposal. If the ideas and approaches are good, the investigator has a good track record, and the research has a reasonable chance of success, it's a good proposal. Where it can matter, at least for me, is the educational component. Fair or not, I tend to think that newer investigators are more likely to actually take the time to implement new and innovative educational activities than are those approaching the tenure decision.
Like Anonymous above, I applied for a CAREER not expecting to get funded but hoping for good feedback.
I actually got crappy feedback. It amounted to "not competitive with the other proposals" but with no indication as to why or how to make it better.
I found it completely demoralizing, and in fact didn't re-apply this year. I spent a lot of time and effort on the proposal. (Even though I didn't expect funding, I also didn't want to give a bad impression or waste one of my three chances to apply.) I feel like almost no time was put into the review, and I have no idea what to do to make it better next time (whenever that is).
Grrr... I'm frustrated all over again, just thinking about it.
Anonymous above said, "A 4% funding rate is pretty close to rate in the NSF program to which I send my proposals. Yes, it's depressing, but to not try because the odds are long is career suicide."
I don't get it. How is not getting rejected for a grant career suicide?
On the discussion of the need to apply for long-shot grants: When I told my father, a long-time K-12 teacher, about a prestigious grant I recently applied for, he was concerned that I know it was a long shot. My answer was this: "Everything I apply for these days is a long shot."
Grants (in my field, at least), faculty positions (especially of the move-up sort).... A person doesn't have to try every one. But if you don't try at least a few serious long shots, you're going nowhere.
I would say, write what you want to do. This is for five years. Write down what research and institutional change you would do in five years. If teaching is your passion, then the creation of a class is good. Show how your class will be analyzed and produce results that can be published. Talk to the people at your university who support teaching. Create a mentoring network across the sciences for women based on region of origin. Build a unique all-x lab. Your department WANTS you to get this reward. Use your writing this grant as leverage, to get support or at least feedback for your plans. Write your passion.
I was just awarded a CAREER grant this year, on its first submission. This was the first NSF proposal of mine to be funded, although I had 3 previous regular (and each very different) proposals without any funding.
Indeed MathTT, unfunded proposal are demoralizing, however I learned so much from the reviews. Some reviews (from those unfunded projects) thought the project was "excellent" while others thought they were just "fair." But I learned a lot from the less-than-glowing reviews, so much so that it allowed me to better tailor my CAREER proposal. Try again. I guess not this year, but next year if you are still eligible. Have 2-3 senior colleagues read it over and give you suggestions. I was hesitant to do this myself, for fear of looking weak to my senior colleagues. This, however, proved to be silly and my colleagues were supportive and helpful. And choosing colleagues not directly in your subfield but in your field in general is a good idea as this will help you write for a broader audience which it sounds like the review committee is comprised.
Post a Comment