Tuesday, November 13, 2007

On Failure

To follow on the last couple of posts on success, it might be timely to discuss (again) the topic of failure. In this case, I mean failure in an academic sense -- e.g., failing to obtain a degree or failing to do well in research (as a student, postdoc, or faculty member).

There are of course many scenarios, reasons, and types of people who don't do well in the academic ecosystem. The various types include:

- those who could do well if the powers-that-be weren't evil, biased, and/or lacking judgment, or who could do well if the academic system weren't so inflexible;
- those who could do well but are hindered severely by problems beyond their control (e.g., physical/mental illness or family issues);
- those who could do well if they tried/worked harder (that's a loaded statement, I know);
- those who are intellectually unable to do the work;
- those who are intellectually able but lack other essential elements (e.g., creativity; ability to complete a project, including writing)

I am not going to discuss the first two items today.

I have observed examples of the other three species throughout my academic career. They have always existed and will always exist. A challenge for faculty advisors is to identify them before accepting them as advisees. At this I have failed repeatedly, providing me with some expertise on the topic, as well as my own experience with failure.

It is important to note that I am not talking about people who could do well with good advising, mentoring, and a flexible research program -- I place them in the first two categories in my list.

It might seem to be easiest to identify in advance those who are intellectually unable to do research, but this is not always the case. Some of my best graduate students have had not-so-stellar undergraduate records, and some of my worst were undergrad stars.

Determining whether someone is going to be enthusiastic about their graduate/postdoctoral work is also difficult. Example: a graduate student who, as an undergrad, had done a research project and apparently done it well, with excellent reference letters from the undergrad research advisor; applied to grad school because of an apparently deep interest in research and excitement about an array of possible projects; arrived at grad school and suddenly lost all interest in research, appearing in the department occasionally to attend a class and teach a lab. I can't help but think of all the applicants we had to reject (owing to the limited number of grad positions available) the year this student was admitted; if we had known, of course we would have chosen differently.

Judging someone's ability to do research can also be difficult to do before that person is given a chance to demonstrate their abilities. However, giving someone a chance to get started with a research project to a sufficient level to judge their abilities fairly takes time. That, combined with time for courses, exams, and teaching labs (in some cases), means that you might not know for 2+ years whether a grad student is going to be able to do Ph.D. research. In the case of a 'failing' grad student, although the right decision may become more obvious with time, it also becomes more difficult to make.

I know of no fool-proof way to know in advance if someone has the ability and passion to do research. Genetic testing? Personality quiz? Crystal balls or tea leaves? I think it's going to have to continue to be an educated guess + trial-and-error kind of experience.

Failing is seldom an unemotional thing. I have never had a student who, after a year or two, just shrugged and said "oh well, I guess this research thing isn't for me but thanks anyway for trying to help me". Nor have I had a student who, after leaving the program (voluntarily or involuntarily), said "I am tormented by regret about all the money that was spent on me, including precious grant funds that you worked so hard to obtain and that I have essentially wasted because I didn't do the work I was paid to do. Here's the money back." [hands the department a check].

Some of my colleagues hope that there is a special circle of hell reserved for those who waste a substantial amount of everyone's time and money before quitting or failing. I am perhaps not so extreme, though I admit to having unkind thoughts about such people from time to time. Instead, I think it is better to wish our 'failures' well, and hope that they will succeed in their non-academic lives and be happy .. and perhaps also make a lot of money and send us that check.


Drugmonkey said...


(man that whole thing resonated)

to put a little back on the PI, let us not forget that in some cases we are willing to settle for half-a-loaf, yes? So the reluctance to be a hardcase and tell the 3rd yr grad student to quit is contaminated by self-interest...

Mr. B. said...

So true, so true...

At the undergrad level, I try to let students learn what is involved in doing research. Some of them realize that either they are not interested or not cut out for it.

Actually, I tell them that this makes me happy because I think these opportunities should help undergrads to make up their minds.

[Although as you point out, liking research as an ug is not an infallible guide.]

At the graduate level, it takes so long to discover whether a a student has the right stuff, that when you you finally do, it is hard to cut the cord if this is necessary. Also there is the matter of the huge financial investment that makes doing the right thing hard.

But that's why they pay us the big bucks, right?


Anonymous said...

I'm curious to hear your thoughts on the students who are hindered by things out of their control. It smacks of paranoia, but I always wonder if my advisor thinks it's that or the "I'd do better if I worked harder". (Although I suspect if he really thought that it was the latter, he wouldn't be helping get my PhD planned right now.)

James Annan said...

A major factor that you have missed in success is having a "successful" mentor/supervisor.

The number of people whose first paper in Nature does not have a co-author who has previously published there is vanishingly small (in my field, at least). The same is not true in other mainstream highly regarded journals which are run by scientists rather than journalists, but publishing in Nature seems to be the benchmark.

My advice to anyone who wants a career in academia is to choose their PhD supervisor carefully. It may not be the only, or even a very large factor, but it's certainly a helpful leg-up.

Of course from the POV of a supervisor trying to identify good students this is singularly useless advice!

Dr. Bad Ass said...

What about the graduate students who fail because they have absolutely zero interpersonal skills, they alienate everyone left and right, and no one will agree to serve as their chair OR to serve on their committee?

I say this only because I was the temporary advisor of such a student. I was so happy on the day I was able to "break up" with this student . . .

Anonymous said...

Nice post. Any correlation you can think of to students' age/emotional maturity at the time they enter grad school? The prefrontal cortex is still developing until age 25 or so...certainly if I hadn't taken some time off following undergrad to work in a lab and find out more about that life, I might have ended up like our recent rotation student, who was fresh out of undergrad and not only failed to understand the importance of showing his face in lab daily and talking with all of us; he also seemed emotionally unready to hear the advice we gave him ("You should try to interact more with the students and postdocs" etc etc.) Despite his obvious brilliance, he wasn't offered a spot in the lab.

Which brings me to--you've mentioned that students in your program don't rotate in labs their first year--would it give you a better chance to weigh them up as grad students if they did?

Anonymous said...

Ah, you have hit close to home for me.

I will be finishing my physical science PhD in less than a year. I love the research and am constantly excited by it. I am at the point where I can independently come up with ideas, design experiments, and write proposals that (often) get time at major facilities. I have ideas for directions that my research should go in in the near-term and long-term. So in that sense I have developed some creativity and independence. However, I do see that my skills in analyzing tricky data and interpreting data are lacking or at least have some way to go. It's sometimes hard to know if this is just an area that I need to develop further or a sign that I'm not cut out for it. It certainly hasn't stopped me from having some success and it hasn't stopped me from pursuing postdocs. (2 offers so far, will need to decide soon.) I have first author publications and my work is well received at conferences (of course, this is by people who do not work with me day to day. My advisor is very supportive and says he is happy with my (female, incidentally) work. I know there is another professor in our group who quite likes me personally but I know is not all that impressed by my work (this is by evidence-based inference).

My point in all this - I do have one - is that it would be really useful if someone (or plural) gave me a direct, frank assessment of whether I have the 'right stuff'. It can be hard to tell, even near the end of a PhD, even with some success. I'm not looking for encouragement or reassurances or validation. I am confident and secure enough. I am looking for the assessment of people with a lot of experience who might be able to see the forest while I am staring at the trees. Perhaps this is something that a lot of people experience near the end of the apprenticeship that is a PhD...

Thanks for sharing, FSP. It's good to hear the perspective on the other side.

ScienceGirl said...

I would be curious to find out what you consider "adequate advising." Uninterested students aside, if a proactive student is repeatedly unable to get feedback from their advisor, is the student set up for failure?

Anonymous said...

Nice post.

Our first year students do complete a 10 week rotation in the lab, during which time I can evaluate them and they can evaluate me and my lab. Except for extremely obvious bad cases - which we could all identify - I still find it to be very difficult to evaluate how "good" a student will be and whether a research career is a good match for them or not. It takes a couple of years and a couple of challenges for that to come out.

By the third or fourth year, it does become more obvious how well-suited a student is for an academic career as evidenced by their level of interest, their technical abilities and their intellectual capacity. In discussing their progress and exit strategies with them, I try to be frank in letting them know what I think it takes to be successful in academia and whether or not I think there are areas they will need to improve if they decide to pursue that career.

One might even say that sometimes my comments go so far as to discourage them, but I really believe that science has to be something they "have to do" if they want this career.

I do not enjoy training students who are not "cut-out" for this, but I can't seem to peg them early enough.

I find the issue of mentorship as raised in the comments quite interesting. One of my colleagues had very hands-on mentors and is also very hands on - to the extent of handling every single piece of his students' data in excruciating detail. In contrast, my postdoctoral advisor's attention span was 20 minutes, max, and that was if you had a publication quality manuscript in your hands. It was definitely sink-or-swim.

I can see benefits to both sides. Especially for students in their early years, a mentorship style that is more hands-on can give them a good foundation. On the other hand, students can come to depend on such an advisor's feedback and it can cripple their confidence and their ability to develop their own critical thinking skills. In contrast, too little mentorship can result in wallowing for too long. But the bottom line is that - as a science professor - no one (except the NIH study section - won't go there right now) tells me what to do or what is important. So I think this career is really sink-or-swim, and the earlier one gains survival skills, the better.

Anonymous said...

One the one hand I can see how a student can be a waste of money, if you could have had a better one for the same investment.

On the other hand, having a college graduate working 20hrs/week as a TA or RA for ~10K/year is a sweet deal, so I think it averages out.

I don't want to say grad students are being exploited. There is obviously an added value to working long hours for very low pay in grad school, but the "failures" are also the ones who are not getting that added value.

I wonder if some grad school drop outs out ther are waiting for a check for the wages they would have earned working in the industry had they not wasted 2-3 years in grad school ;)

Field Notes said...

I am one of your examples.

If a measure of success is actually getting a PhD and publishing an advisor-independent paper, then I am not a failure. But I still don't have a job so I feel like a failure.

I was one of those stellar undergrads who applied to grad school "because of a deep interest in research and excitement about an array of possible projects."

However when I got to graduate school I gradually lost interest, appeared in the department occasionally to attend a class or guest speaker, and met with my advisor sparingly.

There were a lot of reasons for this, but one of them was definitely the lack of rapport I had with my advisor. We did not have matching personal styles and as much as I tried to elicit a more personal relationship, my advisor kept everything strictly business-like. It didn't really work for me. By the time I realized it wasn't, it was too late to start over in another program.

A good advisor-advisee fit is crucial.

If I had it to do over again, I would have met my advisor first in person, visited the campus, etc.

Ms.PhD said...

I'm going to blog about this so as not to take up too much space here. An important topic, definitely.

Unbalanced Reaction said...

Re. RTG's comment: Rotations don't necessarily improve things. Although sucky rotation students are pretty obviously not going to be granted admission to our group, we have had a very good rotation student end up being really sucky. While this student was very productive during the rotation and even made enough progress to coauthor a paper, once they joined the group officially, all the focus went to classes (boo!) and TAing and virtually no work got done in the next two years.

Maybe you can't really tell how a grad student will turn out until s/he has been in a research group for X months??

Schlupp said...

First, I'd like to join sciencegirl in her question about the 'right' amount of supervision.

And then I have a second question about the time after PhD/postdoc. This seemst to be another step at least as big as undergrad-grad: During PhD and Postdoc, people can get through without having to find for themselves the problems to attack. Sure, creativity is needed for successful research at this stage, but I guess you need more as a PI. What is your experience about this step? Is success at the PhD stage a good predictor for success as a PI?

Anonymous said...

I need to voice a counter opinion (or this wouldn't be an academic discussion, would it?).

Many people go and get jobs that they later find out they are not suited to do (especially right out of college). When they leave said jobs (or are asked to leave) they are not obligated to return any salary/resources they made use of while employed.

We take risks when we hire new students. Sometimes (I believe the majority of the time) these risks pay off in the form of data, publications, etc. I am not saying it does not hurt when it does not work out, but I don't feel the person owes me anything (except perhaps a "thanks for the taking the chance/sorry it did not work out"). Unfortunately for us, that is price of doing business.

Female Science Professor said...

Yes, you're right of course, and I accept and understand the risks. That's why the reimbursement fantasy is just a fantasy. In extreme cases, though, I do feel that grad students/postdocs 'owe' me something. Not all students/postdocs work even the minimum number of hours they are obligated to do. I have one grad student who took the summer off without asking me about it in advance, although I was paying his salary from a grant. I have had RA's who had a very specific task to do but did not do it or even make a decent effort. I have had postdocs who, after a year or 2, moved on to another position without ever writing up their postdoctoral research. Those are the ones I have the unkind thoughts about.

EcoGeoFemme said...

What about the masters degree? Many of these problems could be addressed if questionable students were accepted into a masters program first. If they are successful during that, promote them to a doctoral program. Otherwise, there is an easy break when they finsih the masters. Or require all students to have a masters before attempting a PhD. Presumably if they have one, they have demonstrated their aptitude for research.

EcoGeoFemme said...

Ack, I think I just sumbitted the same comment twice. sorry.

Anonymous said...

But FSP, how is leaving after one year because you got bored different from leaving after one year because you decided that you'd rather spend time with your newborn? My reimbursement fantasy would be reserved for those who sincerely "use" you, and that means they have to have known that they were going to loose all interest in research, or were never really planning on doing any, but just wanted to move to the city to hang out with their girlfriend. If they sincerely lost interest, lost their way, got bored, found other things more interesting, but didn't know it was going to turn out like that, I don't think we should fantasize too much about getting reimbursed.

I do think we have to pull the plug sooner and that in my field we don't because people can sometimes be useful, and it's hard to pull the plug.

Female Science Professor said...

One year is fine -- I can deal with one year. More than that is not good.

I have done the MS option for 'underperforming' students, and that is definitely be a good option, but I still would rather know in advance if at all possible, as the 2 years of a lousy MS project can still be painful for all concerned.

Female Science Professor said...

An anonymous commenter wrote: ".. having a college graduate working 20hrs/week as a TA or RA for ~10K/year is a sweet deal."

In fact, it is not so sweet. First of all, with tuition + benefits + salary + indirect costs, you need to multiply that $10k by 5. Even if you only want to consider what the student actually takes home as pay, $10k is not anywhere near the real number. Then you need to consider the pesky fact that my time is worth something -- e.g., the time I spend advising. Then you need to consider the fact that not all RA's work 20 hours/week (those are the ones I have a problem with, as I've already noted), though of course the vast majority work >> 20 hours.

Ann said...

This is one of the hardest questions in science! I also have struggled with this repeatedly, and been both positively and negatively surprised many times by students surpassing/not meeting the expectations we had of them at admissions.

I have started going through a checklist before taking students.
After a 3 month trial, I ask myself if I see clear evidence of at least 4 out of the 7 following traits, else no! (I am a theoretical physicist, there are surely some other prerequisites for laboratory work).

1. Drive to do research
2. can judge correctness of own arguments and spot mistakes in mine
3. can judge importance of problem and relate to big picture
4. can calculate sensibly and efficiently, both algebra and numerics
5. quick to learn and understand
6. creative
7. can communicate effectivally orally and in writing, both to transmit and recieve.

Anonymous said...

FSP and commentators: Thank you a very interesting post.

Anonymous 1 year to go on your PhD: you asked for help on how to know if you have the right stuff. Of course there's no fail safe way to know for sure. And sometimes people who have the right stuff don't make it for reasons outside of their control. But I think your post suggests that you might have the right stuff - you seem to have an interest in research and drive and a decent CV. In my experience the following personal characteristics also tend to correlate with people who make it in academia: determined, good time management skills, relatively robust self-esteem, belief in your abilties, belief that you have every right to succeed in academia, hard worker, passionate about their area of research, desire to keep learning, not scared of steep learning curves, follow the motto "if at first you dont succeed try, try again".... If you have all or most of these qualities (other's might add more) I think you have a pretty good chance in academia. Note that I make no mention of emotional inteligence or social skills because, in my experience, they don't necessarily help that much in academia *grin*.

I hope that helps a little. Good luck!

Anonymous said...

and then you have the underperformer who gets a kick in the ass (metaphorically speaking of course) and turns out to be excellent...
One of my friends had a student like this - for a couple of years he was more involved in social activities than research - my friend had a serious talk to him about his remaining in the lab and that drove the student to find his niche and ended up co-authored quite a few good papers before he was done and becoming a leader in the lab...

Unknown said...

I suppose I was a half failure by the descriptions here. I left with a masters and a completed, successful project behind me, but I knew I was operating below my capabilities in terms of productivity. I think I could have stayed on for a phd, but I knew I lacked the focus to make it worth while.

When I left, I characterized what happened as a failure shared by myself and my advisor. I wasn't motivated enough or diligent enough about keeping myself busy and he wasn't active enough in laying out expectations and helping me find direction. I still think that's true overall.

It was not clear to me when I started grad school just how self-directed a student has to be. That required a lot of adjustment after an educational career where "what" I worked on was mostly dictated by others (though "how" was up to me). I think a different advisor might have eased that transition somewhat, just as another student would have dealt with it better.

Four years later I think I'm ready to go back, but at the time I left, I just wanted someone to tell me what to do for a little while.

Anonymous said...

"though of course the vast majority work >> 20 hours."

that's why I meant by "it averages out"

on the rest of your comment, I understand a grad students costs the PI much more than the grad student's stipend. I was trying to argue it from the other point of view, the point of view of the "failed" grad student. I realize these "failures" are a cost to PIs, but failing is not without a cost to them either. That's all I was saying.

Also, I was thinking about your classifications of people who are just not smart enough or not motivated enough. I wasn't really thinking about the lazy ones, though I guess I could argue that those get offset by the ones who put much more time in the lab than they officially have to (and I know they get a lot out of it too, in ways both quantifiable and unquantifiable, so please understand I'm not arguing anyone is getting exploited here. Not at all.)

Moreover, I was not talking about the grad students who are just sloppy in the lab, wasting not only resources and time, but also making one question whole sets of data. "Unkind thoughts" would be a mild way of talking about my thoughts on that type...

Janus Professor said...

Agree and disagree. Doing my graduate work at a demanding top-ten institution, I notice a combination of short-fallings from both advisers and advisees. The list in FSP's post covers much of what I observed in terms of advisees.

On the flip side, I also saw many problems with advisers. Particularly with my own. Our research group was giant >25 people, and our adviser was never around. Students that might have flourished, with a little push from their adviser, perished instead. In some cases, students in our group were not even given projects or research directions. We were expected to "make up" our own project. Unfortunately, since the adviser was never around, we didn't have a sounding board. Some degree of independence is fine, but there is such a thing as over doing it.

I think that students tend to succeed (i.e. stay in school) if the have some bit of self-motivation, and an communicative adviser.

Anonymous said...

This is a fascinating discussion. I'm curious to hear more about giving feedback to students who are not doing well. I teach undergrads only, so the issues are not as complicated. Mostly I'm evaluating whether they have "what it takes" to go on to law school or grad school. Still, I find it difficult to judge the students' potential (especially since they're so young and they're still maturing), and I find it difficult to give negative feedback. Actually, it's fairly easy to tell the slacker students that they're not going to get good reference letters or do well in grad school if they continue not to show up to class or do the reading. The students I have a real problem with are the ones who want to do well, who are trying, who believe they're going to go on to a great professional career, but who simply don't have the intellectual ability to handle grad school. How can I say what I want to say: "I'm sorry, but you just aren't good enough at this kind of intellectual work"? Is there some way to do this that won't crush a person?

chall said...

Sometimes I think that the major problem with leaving after 2 years or so it the fact that we percieve it as 'failing'. Noone wants to fail, not even the PhD student who has cried him/herslef to sleep the last mounths and tried to fit into the image that they thought they belong in.

I know, it might be a little too much but I honestly wonder what would happen if we had more of the conversations, in the beginning of the project, like "let's evaluate after X months and see if this leads anywhere" and unfortenatly I think if the person isn't getting result - no matter why - there is a need to solve the situation and salvage what can be the best.

And I understand the frustration of not being able to control your grants and results... I feel a little curiousity on why you would stay on a project you don't work with, you don't like and still you are there. Misplaced stubborness?

(I should be able to do this since i took the place from someone else?)

nice post though!

Anonymous said...

"Noone wants to fail, not even the PhD student who has cried him/herslef to sleep the last mounths"

It's not too much. Make that the last 2 years, with 2 left to go, and you have me.

I am trying to finish, not to be a failure, not to have wasted my advisor's time and my own. But I dread waking up every morning.

My failure is in interest more than ability. I don't really know what happened. I don't recognize myself any more.

Female Science Professor said...

ela - Have you talked to your advisor about this? (and anyone else who can help you). I would definitely want a student to talk to me if any of them felt the way you do.

Anonymous said...

It would be hard to talk to my advisor about it without getting all emo. He's very nice and well-meaning, but I don't want to start crying in front of him.

Besides, what could you do if a student told you this? I mean, if I doubted my ability, he could reassure me. But if I say "I just can't make myself care about this work any more?" There isn't really anything you could do to help with that, is there?

I know the answer: I should go find something I do care about. It's just that I've put so much into this. And I've always been doing it for the teaching, not for the research, so I still have hope that if I can manage to finish I can do something I do enjoy.

Female Science Professor said...

I don't know anything about your particular situation, but in my research environment, there is a lot of flexibility -- different students follow very different tracks in terms of research activities and time to degree. If it were possible to change something about a student's research plan in such a way as to improve a bad situation like yours, I would. However, even if that were not possible and I couldn't help a student care about research that they really hated, I could discuss ways for them to finish their degree most efficiently. I think it makes sense that you want to finish something in which you have a lot of time and effort invested. It may well lead to something much better for you.

Anonymous said...

To ela, I'm seconding FSP's good advice. Several of my friends who have come to the decision that research is not for them have been honest about this with (first) their advisors, and then their thesis committees. The solution is usually--get one short descriptive paper out, and graduate. It's much better to be honest and aim for a shorter thesis project than to go the route of losing interest, working less, getting fewer results (vicious cycle), and having your advisor get fed up and kick you out, which I've also seen happen. Not pretty. Definitely tell your advisor, who's the best person to get you on the fast track to graduation, even if it's a difficult conversation. Most profs have seen students cry before so don't beat yourself up about it; just try to stay calm enough that you get your concerns across, even if it's through some sniffles.

Anonymous said...

Thank you (both) for your advice. Who knew this was going to become a career counseling session? :) Maybe I will have a talk with my advisor about wanting to do this as efficiently as possible.

Kathryn H. said...

I found this post by searching "failing out of grad school". I'm about to finish my first semester of a PhD program. I have no advisor - I'm supposed to use the program director for advice, but as the director she is often too busy to meet with me. I was wondering if you or someone else reading this that has some experience mentoring would be willing to give me some advice about my current situation, which is failing.

Female Science Professor said...

Kathryn, I don't know anything about your department or field, so my advice is probably not useful, but questions that come to mind are:

- Are there particular professors you would like to work with? Can you talk to them directly and see if they are interested in advising you?

- Can you make an appointment with the graduate director? She is certainly busy, but if part of her job is to help students match with advisors, you need to talk with her.

Just because you have had a rough first semester doesn't mean you won't land on your feet with a bit of help.