Monday, February 25, 2013

Unchosen

A reader asks the perennial painful questions about why others were interviewed for a tenure-track faculty position and not them, despite their PhD from an excellent university and their apparently better* publication record compared to some being interviewed. There are no satisfying answers to these questions, of course, mostly because there is so much variability in the process, but in case it helps to have one (more) person's perspective on this much-discussed topic, here are some of my current thoughts on the situation.

(*"better" could indicate quantity or quality: more publications or publications in journals with higher impact factors)

Here are the reader's hypotheses, sent in an e-mail to me, for discussion:

1) Doesn't matter how much you have published, they will only look for Nature or Science in your CV;
2) You must have a PhD from a fancy US university, maybe Oxford and Cambridge are accepted too;
3) You got to suggest something really similar (almost overlapping) to what the people are doing in the department, even if they say that the search is broad and open to any topic.

My responses:

1) There may be a kernel of truth to this, but the statement is too extreme (the part about nothing else mattering). Having a Nature/Science paper is typically seen as a very good thing if the candidate has apparently been a major player in the published research, but the absence of such a paper doesn't mean a candidate will not get an interview.

The likelihood of a Nature/Science (N/S) paper depends in part on the subfield (topic) of the research, so in some cases the absence of such a paper is meaningless. Even within a single search, if the search is broad, there will be candidates in subfields that at least have a chance of publishing in N/S, and others that probably do not.

I can say unambiguously that indicating in an application that a manuscript has been (or, worse, will soon be) "submitted" to Nature or Science does not impress.

In my department, we do look at number of publications and journal quality, but we have interviewed some candidates on the basis of a high level of interest in the research and our optimism that important papers would be forthcoming. Some non-interviewed candidates may have more publications than some of those we invite to interview; there are many factors other than number of publications and journal prestige.

2) Faculty with PhDs from the "fancy" US universities are very well represented in STEM departments at US universities, but "must" is too strong a word in this hypothesis. We do look closely at successful and highly recommended graduates of particular research groups, but such research groups can be found at a wide range of institutions in the US and beyond.

I have seen numerous examples of pedigree-worship over the years, as well as the syndrome in which it is assumed that all students of Famous Professors must somehow have absorbed their advisor's awesomeness and must therefore be highly creative individuals as well. I am definitely not alone, however, in being interested in searching broadly and looking at each application carefully to try to get a good sense for the individual's accomplishments and potential.

Even if you apply to a pedigree-worshiping department and you got your PhD at a "non-fancy" university, any disadvantage that this may cause in some searches can be overcome by doing a postdoc in a top research group (in the US or in another country) and/or by working with collaborators at top-ranked departments (especially if they will write strong letters for you).

3) I also don't agree with this one, at least not based on my own experience. You may have to work harder to explain why your research is interesting and significant if there is no one with closely related expertise in the department to which you are applying, but I have seen great interest in candidates who can explain convincingly why we might want to go in a new (for us) direction in a field in which we have advertised broadly.

So, why didn't you get an interview (yet)? I don't know. The individual who wrote to me has an extremely strong academic record and has put together an impressive application (though I would lose the "in prep" part of the CV, keeping "submitted/in review" manuscripts in a separate list from those published or in press). The research statement in particular is excellent. There is no obvious reason why this person would not be seriously considered for a tenure-track position at any research university that advertises in their field, other than that the field is crowded with excellent candidates.

In that case, it may well be that a high-profile paper in a high-impact journal would make a big difference (especially if other candidates have this, but you do not). Perhaps at this high level of accomplishment, anything you can do to stand slightly ahead of your excellent peers makes all the difference.

My only advice (of admittedly limited use) is to keep doing what you're doing: interesting research, publishing in high-quality journals, attending conferences, giving talks. Stay visible, meet people, network, collaborate. I hope your various advisors/mentors are helping you, and I hope something good happens for you soon.

32 comments:

studyzone said...

When I was applying for jobs in my first career, I was passed over for many interviews, even though on paper, I met or exceeded the requirements for the position. One HR person went out of her way (not sure why) to suggest that I may want to re-evaluate who I asked to write my letters of recommendation - apparently, one of my letter-writers submitted a letter that was viewed as "luke-warm", and that was enough to sink my application. Now that I am writing letters for my undergraduate students, I am upfront when I feel I can write only at best a "luke-warm" letter.

Anonymous said...

Other reasons: Your application got lost. One of your advisers wrote you a weak letter compared to what was written for hir other advisees. You don't have the right subfield (fit) for what they're looking for. (This latter is very important. We pass up many wonderful people because we need someone who has already shown they can teach X, Y, or Z.)

qaz said...

I think it is imperative for applicants to understand that the ranking for these jobs are usually judged on fit, and that quality is a bar to cross. What most departments I know of do is something approximating the following: (1) group the candidates into a few categories (think "superb", "good enough", and "not qualified") - the superb group will still likely be larger than they can interview, and then (2) select from the superb group to interview based on fit, recommendation letters, etc.

The two keys are (1) to recognize that one cannot truly rank quality. One can only class them into groups. and (2) to recognize that departments are about fit. They are looking for someone who will fit well within their department and be a colleague for decades.

GMP said...

We are having a search right now, so I can give a few thoughts as well. The search is not entirely broad )("all areas of field") but equivalent to saying, for instance, "particle or nuclear physics" as opposed to all of physics.

We have received about 100 applicants and they are all very, very strong. Just getting to a list of 20-ish where we sent for letters was extremely hard -- some people were eliminated not based on their record, but based on redundancy with other candidates in the poor (ones with better record from the same small subfield or from even the same advisor; fields where we perceive little potential for funding; fields where needed equipment needed is not available; fields where the person would have poor chances of collaborating on campus...)

Of the 20-ish on the long list, we ended up inviting 5. We could have honestly invited any of the 20, they all look great. What made these 5 stand out was many strong publications, excellent pedigree accompanied by very strong letters, and appealing research statement.

Even so there is a degree of intangibility that I can't put my finger on but that oozes from the application. For instance, we interviewed one person with a spectacular record and letters who also interviewed at all the top places. We almost didn't interview another person with a spectacular record who got several interviews at good but not tippy-top places. Turns out, the latter interviewed much, much better than the former.

I have been trying to decide what it is that made the first one invited everywhere and the other fewer places. A great part is pedigree worship -- the first one's letter writers are all widely known and wrote extremely strong letters; also, the application was polished and the research statement seemed exciting. The latter had letters from equally spectacular people but who are not household names, just wildly successful in their own areas. Also, the candidate's research statement seemed kind of boring -- turns out it's great and creative work, it's just that the candidate is not a skilled showman but rather a matter-of-fact smart person; could have easily cost him the interview.

There are a lot of intangibles. Much of it is luck -- who is hiring in a given year, what they are looking for, who is endorsing you and how strongly, and do people know you.
I cannot emphasize how important it is to go to conferences and present. My postdoc got two interviews with very good schools this year, in both cases someone on the search cte had heard him talk at conferences last year (he gave some invited talks instead of me and some contributed).

Get comments on your application package. I cannot tell you how often the candidates don't ask for feedback and that's a mistake. Have anyone who is willing to look take a look and comment on your CV and statements (especially research, for R1 universities). For instance, multiple people should tell you what FSP says -- separate papers in the works from published ones.

Good luck!!!

Phillip Helbig said...

You are missing the obvious answer: Only candidates were interviewed who are romantically involved with and/or are blackmailing those who make the decision. Happens more often than you think.

cookingwithsolvents said...

I've told many this and will include it again here: one of the main disadvantages someone from a "non-glamour" place has to overcome is that peripheral knowledge about how the process works, what to avoid saying in proposals cover letters, etc, and other 'mentoring' **MAY** be harder to come by. You also may get more info as a jr. trainee about the directions that agencies are taking (or the field as a whole). Also, the research you are involved with **MAY** be part of what is taking your field in a new direction. Your colleagues **MAY** be more helpful with their comments on your application package (you got those, right?).

Once you've worked on the proposals, letter, and CV the best thing ANYONE can do is get smarter about careerism, how things work, research directions, and mentorship. A great way to do so is read this and other blogs which cover that stuff in detail. None of it will likely get you a job but neglecting it will definitely sink your application.

Also, be honest with yourself. Writing was one of my weak points, despite publishing a lot of papers in grad school. I worked on it in my postdoc. A lot. And my former boss was happy to see the progress when I shared my research proposals. Perhaps he even mentioned it in his letter, though I have no idea if he did. The point is that I acknowledged my weaknesses and improved them while staying productive in research. That type of professional and career growth is what committees look for, too.

Finally, I will add that for our position this year we interviewed 6-8 but easily could have brought in 15 and had 150+ applicants. Some variation is to be expected, unfortunately.

Anonymous said...

I know for a fact that the top universities will very rarely (never say never..) look at an application from a candidate who did not receive his/her PhD in one of the very top schools in the US or some selected places in Europe. I once heard the chair of the chemistry department of a well-known US institution say how proud he was of having thought outside the box when he chose to interview "Jose Mengano" (my version of Joe Doe in Spanish), who proved to be an outstanding and accomplished professor in his department. "Jose" has a degree from a South American institution, and not even his outstanding publication record in a top 5 US institution during his postdoc with one of the best cited chemists in the planet could help him get interviews. I got my degree in the same university and had no problems getting interviews in universities that rank below ~20, but I would not dream of shooting higher.
Regarding "You got to suggest something really similar (almost overlapping) to what the people are doing in the department, even if they say that the search is broad and open to any topic", the applicant who wrote this comment needs to understand that departments look for candidates that will fit well with their current initiatives while complementing their current strengths. We didn't interview many excellent candidates because we felt they would not have the proper environment to thrive. No department wants a new assistant professor who will have no one to talk to, or will not have access to students interested in their fields. It could be really depressing to be an assistant professor in a place where one is not a good fit, so my advice to the applicant is to try to apply to departments where his/her research will be viewed as a nice addition to their current strengths. We love reading cover letters of candidates that tell us directly why they think they would be a good fit in our department, mentioning names of faculty, center grants, etc.

Anonymous said...

As a related question (not knowing what this particular person's package looks like), I wonder if having too many publications in preparation looks bad rather than good - meaning, this person has the tendency to start things and then get distracted, which would not be a good attribute in an assistant professor who needs to know how to crank things out. Would this be less damning if you are not the first author on many of these (even adding some kind of statement like 'I contributed xyz experimentally' or something, to make it clear your part of the work is done?), or more damning because it still looks like you get involved in too many things?

a physicist said...

In my physics department, we look closely at the research proposal and look for a focus on science, as opposed to techniques. Or at least, if there's a really cool technique, we want to see evidence that the applicant is driven by science rather than just improving their technique. So we occasionally don't interview a candidate who has lots of top-notch publications if they seem too technique/application oriented without enough offsetting science.

Of course, an engineering or applied physics department might well have the opposite opinion. And I think it's highly likely other physics departments might also disagree with us. My comments are more to show that one can have an outstanding publication record and creative research plans and get all the right advice from your mentors, and yet still not fit with the department.

Anonymous said...

That's a good point about the letters. Some people just don't know how to write a convincing, supportive letter, even if they have a strong positive opinion about the candidate. We take into account the well known phenomenon of Americans using amazing adjectives and Europeans and others being more restrained in letters, but even considering this, some people are better letter writers than others, it's not fair to the candidate, and there's really not much that can be done about it except hope that hiring committees will blame the writer and not the candidate.

Anonymous said...

Anonymous 12:07 -- Just don't list manuscripts in preparation. You can describe works in progress in the research statement but the CV is not the place for listing imaginary papers.

Anonymous said...

3838I think we need to define what is meant by "fancy US university".

My department favors graduates from top ten universities, but in every interview cycle we have people from the top 30, and we have reached well below that for some rather outstanding candidates.

Now, top students tend to go to top universities so it shouldn't surprise anyone to see "fancy-US-universities" seemingly over-represented.

In terms of not getting an interview, having been in search committees there are so many reasons why your number might not have come up, including the fact that some middle-of-the-pack places purposely take a pass on top candidates as they have a very low success ration attracting those.

Anonymous said...

Letter writers can be weird. We've had cases where promotion candidates get four stellar letters plus a lukewarm one with no discernible reason for the lack of enthusiasm for the candidate. This leaves us wondering if there is some basis for that or if it's just that the letter writer doesn't know how to write a proper one.

Anonymous said...

There can be so many factors that are not at all apparent to the applicant. For example, one year we passed over an outstanding applicant because we could not accommodate some of the infrastructure the applicant would need. As silly as it sounds, it is not always about the qualifications.

Anonymous said...

No one mentioned "yeah, but we were really hoping for X Y or Z" where X Y and Z are combinations of gender and ethnicity?

I've seen it go both ways: once with a tenure decision and at least once with a hire.

Anonymous said...

The whole thing, from writing grant proposals to interviews (even once you get those) is one big tombola. Too many uncontrollable variables that could swing you either way and way too many excellent candidates in the pool. That's my honest (though a bit tired) summary of science after a draining year on the job market and interview trail.

Comrade Physioprof said...

A reader asks the perennial painful questions about why others were interviewed for a tenure-track faculty position and not them, despite their PhD from an excellent university and their apparently better* publication record compared to some being interviewed.

If they are talking literally about a single position that they applied for, then this is a stupid fucken question. There are a million different reasons--or non-reasons--why one might not get invited to interview at any single particular institution one applied to. It is a fool's game to even worry about that. Applying for entry-level tenure-track faculty jobbes is a statistical process, and you need to apply for as many jobbes as possible. In the biomedical sciences, this means applying for dozens and dozens of jobbes.

Now, if you are not getting *any* interviews at all after having applied for dozens and dozens of jobbes, while others are getting multiple interviews, then it makes sense to ask why.

Anonymous said...

What GMP said.

We are also having a search right now, and of the 100++ candidates...it has been really difficult to decide who to interview, as the top ~30 are all absolutely stellar. So stellar that we ended up inviting 10 - and even choosing those 10 was difficult. At the end of the day, we had to somehow rank those top 30 as best we could based on fit for the Institution, the Department, etc etc.

If you are in that top 30 pile, you will not get interviews everywhere you apply....but you ought to get interviews at some places (and not others). I know this answer may be frustrating in some ways, but look at it this way: it is possible that there is nothing 'wrong' with your application, but that it's just a very deep applicant field and that everyone needs to apply to more places to get x number of interviews.

That being said, if you aren't getting interviews even with a large n of applications, and everything else about your application seems stellar, I'd think hard about who you've asked to write your letters. Letters can make a big difference to a search committee, and that's the one thing about your application that you don't directly control.

Anonymous said...

Hi All,

I am the person who wrote to FSP about this. First of all, thank you all for the comments, I wanted to get an insider's view on the process and I definitely got it!

I thought I'd clarify some of the points raised here as well as provide more information.

1) Reference letters: this is a very good point, after all reference letters are the only part of your application that you cannot control! Still in my case I have asked these people for references before when applying for postdoctoral fellowships and I got all the ones I had applied for. So they must be pretty good... Still, will think about this again...

2) What is a fancy university? In this case I really meant fancy! MIT, Harvard, Caltech, Princeton and a few other. Basically top 5-6 in the US. In my case I got my PhD from a foreign university that ranks in the top 25 in the World, but I wonder how many people actually know that...

3) Infrastructure: that is a very good point, though it does not apply in my case. My research makes use of computer models, so all I need is to get access to a supercomputer, which is usually granted via NSF. The only money I would need is for consumables and to hire people

4) I have applied to ~ 15 positions in the States. So I have enough statistics :) I have received 7-8 rejections so far, the other applications are still under review.

5) Network: I do try to network as much as possible. I usually go to 3-4 big conferences per year and meet lot of people there. I also went to the career development seminars that are usually organised at these conferences.

6) I got people from all levels (student to Prof) to read my application. That was really useful, because each of them spotted something different. Nobody spotted the "in prep" publications though :)

All in all, based on your comments and FSP blog entry, it seems that t-t applications are a many-body problem with so many factors to take into account. I guess it's a matter of statistics, the more you apply the higher the chance that somebody notices you!

Thanks again for the feedback, will make some changes to my application package for when I apply next time.

Anonymous said...

9:05 am anon:

15 positions? That's it? In my experience, my interview invitation rate was around 10%. Offer rate was around 20%. As physioprof pointed out, you need to apply to "dozens and dozens" of jobs.

None of the other commenters mentioned money. If you have two applicants with equivalent qualifications, I think naturally a committee would choose the one who has hir own funding over the one who doesn't. Having your own funding beforehand is definitely a plus.

Anonymous said...

15 positions is not enough, period.

Last year I applied to 105 places, got 6 invites for onsite interviews. Declined one invitation and interviewed in 5 places. Then I got 3 offers letters -- the other 2 places contacted me after I had accepted my current position.

I had the luxury of choosing my best option and negotiating very good conditions, I am happy with my job.

Summary: you to work hard for it, apply, apply, and then apply some more.

Too many stars have to "align" for landing a successful position....

I have a very good friend in top-ranked US university, he submitted 200+ applications.....got a great position with an awesome start-up....

Anonymous said...

Dear Anon who wrote to FSP,

To be clear (re: letters), as someone else hinted at, it's not necessarily that one of your letter-writers doesn't think highly of you...but some people are not good letter writers. It's the difference between, "I most highly recommend X, as he/she is an excellent scientist who has an extremely bright future" and "I cannot possibly overstate how highly I recommend X. In my 30 years as a PI, he/she is quite simply the best scientist whom I have ever trained, and this includes several former trainees who are now Chairs of the top 5 Departments in my field. He/she is simply destined to be a star, and I can guarantee you will not find a more impressive person or scientist." You get the idea. Even if they think very highly of you, are they *good* at writing letters, per se? The first letter above should certainly get you a good postdoc position. But it might not get you that faculty interview...

Re: numbers of schools to apply to, it's impossible to generalize these things across subfields of science. I'd ask other people in your subfield how many they applied to and how many interview they got. In my subfield, I applied to less than 15 and got 4 interviews (including one at a top 5 school, the rest top ~20), fwiw.

Good luck!
Anon from 2/25/2013 11:21:00 PM

Anonymous said...

Warning: blunt statements ahead. Publication record is just one element for a strong candidate. There are a lot of candidates with strong publication records. We get candidates through with dozens of papers, many in good journals. What else can you show us?

Our department (at an R1) looks very carefully at "likelihood of funding". There are several aspects to this. Has the candidate demonstrated past ability to be funded, through graduate and/or postdoctoral fellowships? Is the candidate's proposed research a good obvious fit for one of the federal funding agencies? Are you bringing a project with you from your postdoc with preliminary data already collected (different expectations from different departments - so also a fit issue)? If their research proposal came to us in the grant review process, is it competitive for funding?

If you go to 3-4 conferences a year, have you won young investigator research awards for your posters, or have you been selected for talks that demonstrate real excitement for you and your work within your (sub)-field? Do you have any other external metrics that the committee can point to that demonstrate that you are exceptional, not just good?

Do you have evidence of a strong teaching record? At our R1, we are required to discuss this - a strong teaching record will not make up for a poor research record, but a poor teaching record will sink a good research proposal.

One potential warning sign for me is that you say you are using the same references for faculty positions that you did for your postdoctoral applications. If your proposed faculty research projects are within the field of your postdoctoral research, then more recent evaluations of your potential within your chosen research field might be more appropriate. A non-advisor letter from your graduate career has to be pretty spectacular to make up for the recency bias.

Your letters have to either say you are among the very best researchers that have come from your advisor's group, or your advisor has to be someone with a really strong history of successful proteges. If there are multiple postdocs on the search from your current lab, this can really count against you - because you are competing directly against your labmates for the openings in your subfield. Can your subfield support that?

Anonymous said...

"The truth is, son, it's a buyer's market
They can afford to pick and choose" -Billy Bragg

Anonymous said...

Beware that some of the statements made above are field-specific. In my field, having a manuscript listed as "submitted" to Nature or a peer-reviewed journal is considered to be virtually as good as a published paper, and may be included in the regular pub list on your CV. But not "in prep". Also, in my field, it does not make sense to apply to dozens and dozens of positions -- first, there aren't that many, and second, there really is no point applying to a place where it's pretty clear that you're not a great fit. That's a waste of time for everyone, and it will irritate your letter writers.

GMP said...

Regarding what Anon at 12:14 wrote: I don't think candidates should worry about irritating letter writers. When I agree to write a letter for someone, I assume I will write as many as they need and as often as they need, until the day they no longer have use for them (or one of us drops dead). Writing evaluation letters of all sorts is part of my job. Furthermore, recommendation letters show your commitment to support an individual and it is not an imposition. I think everyone is aware of how difficult it is to get a job, the last thing you want is for someone to hold back from applying broadly because they don't want to annoy letter writers.

Anonymous said...

God, I'm glad I decided against the tenure track years ago. I have a super job with good pay and stable employment at a university and the only thing I don't have is the ego-gratification of knowing I belong to the TT club. I don't miss it.

Anonymous said...

I wonder what it looks like when your postdoc advisor can't be bothered to write your letters on time. Anyone on hiring committees care to comment on this one?

Anonymous said...

@GMP: This is Anon 12:14. Fantastic that you are so magnanimous, but I do get annoyed when the applicant is applying for jobs that I know they won't get. I am fully committed to support the applicants, and I send letters wherever they want, but I do resent being asked to send dozens of letters to places where it's obvious the applicant doesn't have a prayer.

@Anon 11:37: It doesn't look good when your advisor doesn't submit a letter at all. Late may be inconsequential or may mean the letter cannot be included, depending on how the deadline is dealt with. But people also should send gentle reminders to their letter writers of approaching deadlines. If you gave them a list 2 months ago, a lot of people will honestly forget -- it doesn't mean they don't care or think highly of you. It just means they are super busy and have other urgent things in between.

Anonymous said...

I do resent being asked to send dozens of letters to places where it's obvious the applicant doesn't have a prayer.

Obvious to whom? It's tough from the perspective of the applicant to get a good sense of what our chances are. Remember this is largely a black box to many of us. We don't get feed back other than getting an interview or not.

Frank advice is tough to come by IME. I talk to plenty of TT folks in my area an no one wants to be the person to say "you're good but not that good". I practically had to drag an admission that funding is important out of my advisor.
Virtually everyone is told they're "competitive" and that things are very unpredictable. So why not apply for everything.

There is this at times a weird discrepancy between advice given to candidates and how they are actually treated.

EliRabett said...

Most jobs are not at R1s (anyone sending out 100-200 letters is looking at many lower ranked places) and outside of that, maybe even there, set up costs are a huge issue.

Make sure you tailor your project to the resources that will be available.

Teaching loads also differ. Your letter should take this into account.

Anonymous said...

I have served on numerous search committees, and sadly the idea of 'fit' usually has a gender connotation that discriminates against women because the vast majority of the search committee are men (or entirely men). In my experience, the people doing this don't realize their bias, and they can't always explain why they chose the male candidate over the female for an interview. I have also seen this happen to older candidates and to a lesser extent, because of race. In fact, I have seen this happen to female candidates who had Nature publications, and yet that is ignored in favor of some unmeasurable marker for the male candidate. I do not think my experience is untypical. I have seen this behavior in 4 different major universities in the USA.