Wednesday, February 17, 2010

Magic Number

How many papers do you need to get tenure in a Science field at a major research university today?

Let's start by considering the minimum number: What is the lowest number of peer-reviewed articles that you can publish and still get tenure today in a Science department at a major research university in the US?

How about 6? Could someone get tenure with 6 publications, as long as these papers were in respected journals and weren't Least Publishable Units?

(I am referring here to Science fields in which journal articles are the main form of publication. To extend the question to some Engineering departments, perhaps this could also include reviewed papers for highly selective conferences.)

I think 6 publications might be sufficient if the funding record was very strong and there were indications of an upward trajectory now that funding was plentiful and students were on track and the teaching record was good and there had been invitations to speak at conferences and other universities. Sometimes the system will give you the benefit of the doubt, even if your publication record is thin.

Institutions vary on policies about whether the relevant n is the number of publications that represent new work accomplished since the time of the tenure-track appointment or whether n is an all-inclusive number going back to the candidate's first publication as a student. Ideally, both will be good numbers, but even if all publications count towards tenure, n(tenure-track) is more important. In my musing about n = 6 above, I was assuming that the 6 was n(tenure-track) and that the total number was higher.

There is of course a lot of focus on n, but the quality of the papers and journals does matter. If you are a minor co-author on 30 papers in journals that will publish anything, or if you have attained a high n by providing bits of data that others use (but without much involvement by you in the overall research, interpretation, writing), your impressive numbers might not seem so impressive when scrutinized.

So, if you don't want the tenure evaluation to be more stressful than it already is, you shouldn't publish too few papers or too many of the wrong sort. You should therefore figure out the magic number of substantial papers expected for your institution and discipline and that is just right for the amount of funding you have and the number of students and/or postdocs you are advising.

What is that number in your field?

Note that the magic number is not the same as the aforementioned lowest number. The magic number is the number of publications that would make the attainment of tenure fairly certain but that is, at the same time, a number that can reasonably be accomplished by a human being who occasionally sleeps and eats and has the usual amounts and types of logistical problems with students, facilities, colleagues, life. Assume that all or most of the publications are in respected journals and are more substantial than least-publishable-unit types of papers, and are therefore not the kind you can just write in an afternoon in a cafe no matter how many double espressos you have.

Is n <> 10?

47 comments:

Anonymous said...

In my field, n/year between 2 and 3 is the magic target. So, if you don't go up early, that's 12ish by the paperwork deadlines for promotion. If you want to goup early, the number needs to be higher and thus the rate higher as well. I find that strange but it apparently takes an exceptional candidate to merit early consideration.

estraven said...

Is mathematics a science? In my subfield, a paper/year might be acceptable, or even somewhat less. In other subfields, I think 2/year is more common.
In a really good institution they will mostly be interested in the level of your top paper(s), not in the total number.
One good paper per year is definitely enough for tenure, and you can write it and still have a life (unless your life contains too many toddlers).

Anonymous said...

how do the people who are evaluating you judge if your papers were least publishable units or not?

I have been in research groups at both ends of the publishing extremes: in one of my groups, everyone published LPU type papers all the time. Everyone was always writing a new paper every other month it seemed. Therefore everyone in the group had a ton of publications, but all with practically the same title. I never liked this approach.

I've also been in other groups that were the exact opposite, where the culmination of 2 years worth of work is ONE all-inclusive and really high impact paper like in Nature or Science. And where it was discouraged to publish anything on that project afterward without at least another 2 years' worth of new results. (the timeframe wasn't the criteria, I'm just saying that the amount of new results expected before being allowed to publish again, was equivalent to about 2 years of work.) I have a higher regard for this philosophy because I think there's way too many crap papers out there (LPUs), and I think the LPU approach is all about sacrificing quality work for instant gratification. But obviously because the rest of the world doesn't share this philosophy you will get penalized if you only publish one paper every 2 years even if it's a really good paper.

IME, one will do better in one's career following the LPU model than the other extreme but I wish it were the other way round.

Anonymous said...

I think an average of 3-5 a year is expected at my institution. I've seen people get tenure with fewer very high profile papers that get a lot of press. It also seems to be OK to start out with fewer each year and ramp up to a much larger number (2 in year one, 4 in year 2, 8-10 the year before tenure).

What about first authored papers vs. papers coming out of your lab (post-docs/ grad students/ undergrads)? Does that matter at most institutions?

qaz said...

FSP - I'm curious whether your "magic number" counts second- and middle-authorships. (As compared to senior authorships - which appear as last-author in my field.)

In my observation, it depends greatly on the expected speed of production in your field. Some fields, one paper every two years is typical because the data takes a year and a half to collect. Other fields, two papers every year is typical. However, in my observation, there is no explicit magic number. It depends greatly on where the paper has been published and what you've contributed to it. For example, middle author publications are not worth much alone, but can count towards your number if you have a couple of really good senior author pubs. Similarly, first-author pubs with your former mentors, even if published while on the tenure track, don't count unless you have a couple of senior author pubs to go with them. And, of course, the distribution of journals makes a big difference as well.

Anonymous said...

I know of a recent case in physics with n=6. And some of those were in collaboration with the postdoc advisor. But the independent papers had very high impact, and the funding record was very good.

cookingwithsolvents said...

As always, thank you for a thoughtful post!

To what extent does "having a research program" figure into the # of papers equation? E.g. if you have 4 papers moving towards an overall goal and 2 others doing something else vs 2/2/2 of "random stuff".

I realize "research program" vs "research project" matters for funding, too, and comments on that are welcome as well.

Anonymous said...

At least where I am (molecular bioscience department at top private R1) n is MUCH more than 10. I would say its between 20 and 30 senior author papers.

Anonymous said...

At my place it's about 10.
But I find it so stupid, really. People in the inorganic field publish like crazy- they make a complex and that's it, whereas if you are a crystallographer it's so much harder, especially if you start with a new protein. I switched fields between grad school and postdoc, and my paper/yr count has decreased significantly, so I speak from direct experience.....

BTW I was a postdoc at top private R1, I saw two people getting tenure with 1-2 (Science) papers. Bean counting is plain wrong.

Anonymous said...

n >> 10 in my field. I wanted to say 10 per year is more like it, but that would be exaggerating. 5-7 per year would be pretty accurate.

Anonymous said...

In my field n=2/yr is the minimum. At that level, you and your students need to be the primary researchers on the work. But at my institution grant dollars are more important than publication numbers.

Anonymous said...

A major flaw in the counting system is that what looks like multiple measures is really just one. Number of students, number of papers, and grant amount are considered three separate metrics, but they are strongly correlated. The more money you raise, the more students and postdocs you can hire, and the more papers you can publish.

Ann said...

Now that I sit on the college's tenure advisory committee, I can say that at my institution the number and type of publications totally depends on subfield, and is highly variable. Also, quality trumps quantity. Also important is evidence of impact--citations, major talk invitations etc, and funding at whatever level is needed to do research.

Emil Chuck said...

While I don't think that you can put just a number "n" without putting in impact "f" for quality papers over time "t" in a simplistic sense, there is a number that seems to make sense for postdocs producing first-author papers in a study I did, which I am presenting at AAAS 2010 this weekend. :)

ME said...

In my field the number is 1-2/yr from your lab (e.g. you are the first or senior author on the paper). Closer to 1 if you are in an experimental area that is slower to publish and closer to 2/yr if you are a theoretical/modeling person who doesn't do experiments.

Anonymous said...

I have actually heard of n=1 in a biological department (I think more of a field study, multi-year type of biology than cellular/molecular biology)

I too hate the LPU approach; however, to get grants you need some kind of publication record and there is a preference for senior author papers by many reviewers.

In my field (biomedical eng), the n around 10 or 12 seems to be a norm, with many of them coming in the last 2 years before tenure

John V said...

In the last few years, I've seen more reliance on h-index and number of citations than number of papers. Both are available by clicking a button on Web of Science. Paper count is fairly useless.

Either measure avoids the problem of guessing whether anyone read the papers in question, and whether shingling is an issue, although the question of whether the citing papers agreed or disagreed with the work remains.

Both work better for senior than junior scientists because of the significant and variable time lags of recognizing good work, as well as problems of statistics of small numbers.

Anonymous said...

In my European university, the criteria for promotion from lecturer to senior lecturer (which I believe is roughly the same as going from TT to tenured in US, from what I can tell, although of course the concept of tenure does not exist on this side of the Atlantic) is at least 10 first author/last author papers, which have to be independent from your PhD work (i.e. your PhD supervisor should not be among the authors). Second/middle/whatever position author papers don't count into this number. You can apply for this promotion after 7 years of being a lecturer, so I think it's roughly on the same time scale as tenure in US. This is in an IT field, however, the promotion criteria are for the entire university, which is a general university (includes sciences, engineering, social sciences, humanities, arts) - I think for the humanities/social sciences the 10 papers can be replaced with a published monograph.

Doug Natelson said...

In my part of physics (condensed matter) I would say that 6 is on the low side, but if those 6 are good papers in serious journals, that could be ok. I can think of a high profile school that tenured someone at about that level, and I can think of another that denied tenure at a much higher publication number.

Anonymous said...

At my institution, the number is 10 papers in highly read journals (PRL, Science, Nature). That includes papers that you wrote in grad school and during your post-doc so people are generally not hired for tenure track positions unless they already have 4 or 5 publications. The idea is that then they can produce 1 paper a year and still make it to 10.

Anonymous said...

I'd say that at our broad biology dept. n = 1/year at least.

It's highly subfield dependent, though. Biomedicine tends to have more giant uber labs with 10+ grads/postdocs simply because there are a glut of postdocs and there's lots of funding in biomedicine. This leads to more publications belonging to the PI (even if the PI really didn't contribute anything much to a lot of the projects other than funding the postdoc).

For environmental, ecological, and evolutionary sciences labs are smaller, so there are fewer papers per year. Usually the PI is much more actively involved in the research, however, and there are many fewer authors/publication.

Sometimes I think a correction should be made for the number of authors on a pub. Like a publication with 10 authors counts as 0.1 for the last author compared to a single author paper by the same PI. :p After all, grants with multiple co-PI's count less towards tenure decisions for the co-PIs than would a single PI grant.

Anonymous said...

I am up for tenure next year. I have 21 papers published. I think this is a lot, but some of my colleagues criticize me for just publishing new data every month or two. Is this a problem for me? Do I need to start doing new work, or should I continue working on this specialized problem few care about? I distribute my data over a variety of journals, and I am mostly 1st author, but every paper of mine is very similar. If I have tenure I think I could feel relaxed and start to branch out more, but for now I will just continue to publish as much as I can.

Anonymous said...

In applied mathematics you can get tenure at most prestigious institutions with 2-3 (~25pp,1-2 authors) papers a year and a very strong funding record. Of course, everyone really must think you are the absolute top in your sub-field as well. (I know of people denied with 30+ papers in top journals but not seen as quite the very top of the top.)

Most departments will give tenure to someone with only 6ish papers (ie. one per year) if they are in strong journals as mathematicians publish less that others in the Faculty of Science as measured by paper count.

[Our dean, (a very theoretical physicist), recently calculated that mathematicians and mathematical physicists actually publish more pages per person than any other group in the Faculty despite having by far the lowest publication count.]

Unfortunately, because everyone person is so specialized within ever more specialized departments it is hard for Chairs, let alone Deans, to personally evaluate a publication record beyond counting and, maybe, recognizing some journal titles.

This unfortunate situation makes this is a sensible question. There was a time that the bar for tenure was (almost) paper count independent but that time has sadly(?) long passed.

Anonymous said...

John's comment points out a really disturbing trend -- because we cannot meaningfully measure the quality of our colleague's work we are moving to pointless metrics (paper counting) or those that can be gamed and are too heavily influenced by measures beyond the quality of the work (popularity count is particularly bad).

Citation count and h-index drive junior people to trendy areas at a time when they should really be moving in new directions. I don't see a fair, workable and externally justifiable solution to this mess. But, hey, maybe it doesn't matter -- Wired claims all advanced research will soon be done by AI anyway.

John V said...

Trendiness and popularity contests could be characterizations of measuring citation count and h-factors.

However, the alternative of polling whether one's colleagues consider work to be challenging and profound is an open invitation to a Lake-Wobegone attitude of everyone being above average, and people most effusively praising their friends at the expense of outsiders.

The futility of paper count is illustrated by the median citation count per paper being somewhat less than 2 (I think, my source is Nature 444, 1003-1004). Excepting very recent pubs, papers with 0-2 citations were mostly a waste of everyone's time.

Anonymous said...

(WIN! verification word: citable)

John,

I completely agree that paper count is scientifically useless, however it grew to prominence because it is so easy to do and so easy to use to justify ones existence externally. "Look Ms. Government official and Mr. Joe Public, our faculty members publish n+1 papers a year!"

I need to look up the reference on this but, there are already documented cases of people seriously gaming both citation counts and h-index (who was the physicist Elsevier journal editor who essentially required all published articles include him as an author and/or have loads of references to his work?) with groups citing each other in blocs to the exclusion of others.

I totally agree that these metrics are better than plain counting but they are not the panacea that administrators hope they are and Web of Science sells us they are.

Plus, we really need something to tell us how good or bad we are else many scientists would simply fall apart. (I am ashamed to admit my Sloan announcement changed my life much more than the money or the opportunities holding the fellowship brought.)

There are many driving factors behind these metrics -- accountability internally and externally, ranking in the competition for scarce resources, ranking for the puffing up of ego and so on. And this has all been done, arguably, to the detriment of the advancement of science. We all know people whose work would be much better if there were less of it., we would all be able to work better if we didn't have to referee (and even read) so much pointless CV padding work and of course we would be freer to pursue true curiosity driven research if a fruitless quest was not seen as exclusively a failure by those evaluating us.

There has been a shift from quality research to 'quality' publication for so long now that maybe it is time to swing the pendulum back the other way. Toss out the journal system as it is and start fresh with something freer but also verifiable and accountable.

Any good ideas?

(All this has been on my mind because we are about to deny our most promising assistant professor for not publishing enough. He is a wonderful academic in all ways except that he doesn't see the point in publishing much. I see his participation in the department and the broader community as a net gain for everyone but his publication record (no matter how you measure it) makes it hard to justify his ongoing existence to the Dean.)

Anonymous said...

Uhm, most of us can be above average with no chicanery at all. For instance, most everyone in the world has a higher than average number of legs.

However, given the exponential distribution of talent, most of us are below average.

SamanthaScientist said...

Oh my God, can someone please tell me what field everyone else is in? I'm in biophysics, and in comparison to what everyone else is saying, we NEVER publish.

OK, not never, but everyone talking about spending 2 years before publishing on a project. Try 7! 7 years without publishing on my project. And that's not a rarity in biophysics.

I can understand not wanting to publish some LPU, but 7 years? That's insane. I need to graduate and find a new lab, and possibly a new field.

Doug Natelson said...

Anon@1:28pm, I feel your pain regarding your colleague's situation. Still, disseminating results is, in some sense, the main point of doing science. If people only worked to satisfy their intellectual curiosity and never published their results, science would not function as an enterprise. Grinding out gratuitous low-impact papers is bad, but working within a cocoon is bad, too.

SamanthaScientist, 7 years with no publications is not normal in biophysics. Not hitting a *major* homerun in that time period isn't necessarily surprising, depending on the area, but that's different than no publications at all.

Alex said...

Assume that all or most of the publications are in respected journals and are more substantial than least-publishable-unit types of papers, and are therefore not the kind you can just write in an afternoon in a cafe no matter how many double espressos you have.

Hey! My best paper, with my best insights and published in a really good place, was written in 24 caffeine-fueled hours. I had to either write the paper or start preparing for the new quarter, and so the choice was obvious.

Kevin said...

"every paper of mine is very similar. "

If I was on a tenure committee and saw an applicant whose papers were just the same thing over and over, I'd vote against tenure. First, a faculty member should be interested in more than one thing (what if someone else solved the problem they were working on?). Second, a faculty member should recognize when a paper is publishable---repeated instances of LPUs is evidence of a very non-productive researcher.

Anonymous said...

If one is writting the paper as the corresponding author(starred) and can decide author order, is it best for tenure to be the first or last author?

Plague of Crickets said...

As people have said, it is completely dependent on the sub-field. An bean-counting, by itself, is just plain idiotic.

One thing that I didn't see mentioned was the external letters. Unlike within department evaluations, which are often made by people who don't really understand the importance of a candidate's research, the external reviewers do (at least they do if they're chosen correctly).

In my experience, if experts in the candidate's sub-discipline rave about the importance of a candidate's research, the bean counting is far less important. I've seen people receive tenure with only three in-house publications because the top people in the candidate's sub-discipline told us the research was that good. Conversely, if the external reviewers tell us the research is pretty pedestrian, can be a tough slog even with a ton of publications.

Of course, external reviewers have biases that can affect their evaluations. Ultimately it comes down to a subjective judgment of whether the candidate is doing enough interesting work. And what constitutes "enough" and "interesting" is almost impossible to quantify in a meaningful way.

Anonymous said...

Seems to me that external letters may favor TTs who did PDs/PhDs with the 'big wigs' of the field.
Take two TTs doing similar work, but one has a previous association with one of the experts in the field. Hhmmm...the admiration for the 'big wig' may unduly bias the letter in favour of that TT, no?

Anonymous said...

When our department was forcibly merged with another, the n number of publications for tenure for the whole new super department was based on the politically dominant department that was in a vastly different field with vastly different [and faster] procedures for generating data and getting publications. For members of our former department in a very different field, this number is unattainable with the resources available... so our n = screwed :')

Anonymous said...

The LPU issue is complicated. I agree that publishing a lot of 'trivia' doesn't make sense but does it make sense to do everything on a question before you publish on it? I'm a field biologist who also does experiments. In my work I might have some projects that involve sampling natural populations over many years and also projects where I do an experiment in 2-4 weeks and get an answer. I try to move back and forth between these things i.e. do the results from the experiments explain the population dynamics we see? But it doesn't make sense to me to wait for all these elements to come together to publish. Publish the experimental results and in a couple years when you have the data publish the field data and use the published experimental data to interpret the field patterns. Is this an LPU approach? Or just strategic? If everything has to come together in one piece can you get anything out in a timely way? Thoughts?

inBetween said...

six?! we won't even hire someone for an assist prof position with only 6 papers. I go with the 2-3 solid peer-reviewed papers a year, and an average over 3 is the best bet. So, with a postdoc and then 5 years of assist prof life, that would put you close to 20. I'm not sure I'd go up with much less than that... maybe my field is a much different beast.

Anonymous said...

often times people rush to publish LPUs out of fear of getting scooped

Anonymous said...

A good publication record is necessary but not sufficient for tenure. External research funding (especially federal agency money) is also important. And a good teaching record lacking significant student complaints (better with significant student compliments). And decent service. I think red flags would get raised with < 1 strong publication per year, but there's much more to look at than publication quantity.

Frank said...

"If I was on a tenure committee and saw an applicant whose papers were just the same thing over and over, I'd vote against tenure."

I agree. Not only does this make one sound like a broken record - if all your papers have practically the same title - but I notice such people who do this also tend to be the most aggressive in promoting themselves, pointing to their high number of publications. This combination is most unappealing.

Anonymous said...

As a UK based academic I know little detail of the US system. I know of course that tenure = promotion = more money = job security, but what happens if you don't (ever) get tenure?

What is the likelihood you actually get fired or your job description changes? (excluding cases of extrenme incompetence/gross mis conduct of course). Is there a withdrawal of access to departmental reseach facilities and money? Do you end up on a teacher-admin only career path by default. Or, if you bring in the money can you in effect just carry on "doing your stuff" and not worry about not getting tenure?

Anonymous said...

just to specify: my comments on 10 papers at my place, and 1-2 science being enough at a Ivy league, referred exclusively to the tenure track time. I have never heard anybody talk about previous work....
and the 1-2 papers were paradigm shifting Science papers (e.g., Xray structure of a protein that had eluded several major groups for 20 yrs)

Anonymous said...

ANON 07:26:00 AM,

No tenure == no job.

You are initially hired with a fixed term contract. You need to get tenure before that expires or you no longer have a job (you typically have a year to get your affairs in order before being let go). Many people that do not get tenure at their original school move directly to a tenured job at a lesser institution or leave academia altogether.

For instance, we have one member in our department who did not get tenure at MIT -- we hired him at the associate level (Reader) with tenure. The one person in the history of our department who did not get tenure is now teaching at a liberal arts college (a good one, but only limited opportunities for research).

To answer your next question (how likely is it to get tenure): it depends. Most universities try very hard to only hire people at Assistant level if they really believe they will definitely get tenure. Not getting tenure means there was a significant change in your predicted career trajectory. Some schools (like MIT & Princeton) pretty much never tenure assistant profs (in my area) -- people start there and leave after 4-6 years. Those schools also recruit very successful senior people directly to tenured positions.

Other places are in between. It requires a lot of effort to hire someone new and some level of departmental continuity is important for smooth functioning so you really don't want to keep cycling through assistant profs every few years if you don't have to.

Anonymous said...

It's also country specific.

I went to an American 'How to get a TT job' talk and was laughed at when asked if I could guess how many papers you were supposed to have to get on TT.

I had 4 times as many as required. They thought I was joking.

Anonymous said...

If you do not have tenure and have n papers then the number needed for tenure is >n.

It's Simple. You're fucked.

Anonymous said...

I'm no expert in statistics. How strong is a correlation of 0.5 deemed to be?

Anonymous said...

I'm one of three people hired in a single cohort in molecular biosciences at a top-25 public R1. We all have solid federal grant support. We all went up with 5-7 papers from our present positions where we are corresponding authors. All of these papers were in good journals (say, JBC to Cell). We all have additional collaborative papers on which we are not corresponding authors, and a few reviews. We all went up with a total of 18-25 research (data) papers on our CVs when grad and postdoc papers are included. In theory grad and postdoc work is not supposed to be considered. In practice, of course, it is considered. We were all tenured.